It feels like quite impossible to not know the D-A reaction for anyone who has touched on organic chemistry, but often it feels simple also complex if you dig deeper. We are going to talk about important things about this reaction from mechanism to stereochemistry.
The first thing you see in a D-A reaction is to realize that there are a diene and a dienophile, which is easy to distinguish as the diene has two double bonds. Next, we need to classify all D-A reactions into two categories, according to what are the Y group as shown above. Beware that is fairly common that the diene also brings with substituents but for the sake of simplified discussion, we only use Y group here as the way to discuss next.
1. Normal D-A reaction, with an electron-withdrawing Y group.
Often, Y here is a nitro group or a carbonyl group like ester, or a nitrile group, etc.
But why in such circumstances these reactions are called normal Diels Alder rather than the other way round? Well, now we have to mention the HOMO and LUMO theory. A quick answer is the LUMO prefers an electron-withdrawing Y rather than an electron-donating Y. since the former can lower the LUMO energy to help HOMO easily flow its electrons to LUMO.
Most of the examples listed inside the above links are the "normal D-A reaction", with the Y group being an electron-withdrawing group, but as we have mentioned, there might be additional substituents on the diene as well, but all you need to know is most substituents on diene are designed to increase its HOMO, just the opposite to Y is designed to lower LUMO so that the energy gap between HOMO and LUMO is narrowed so is it possible for the reaction to happen.
But why the Diels Allder reaction could occur, given the fact that some other HOMO-LUMO systems are not under just heating but require more hash conditions like UV irradiation?
This is a bit outside the scope of the Diels-Alder reaction, but we have to discuss it here, the conception is the symmetry of the molecular orbitals(MO). For the dienophile which has only one double bond, we need it as being the LUMO, as shown in the next image, there is a HOMO as well but it is not the one we need to let the reaction occur, the important thing to remember is the two vertical orbitals inside the LUMO on the right are not symmetry, so it requires to react with an unsymmetric HOMO from diene as well. And if you have a close look at the left HOMO involved, it is not symmetric in terms of splitting the four vertical orbitals from the right middle. That is the reason why D-A reaction could just happen under heating in most cases.
2. Reversed D-A reaction with an electron-donating Y group.
For a reversed DA reaction, the LUMO is from the left diene side, while the HOMO is from the right side dienophile
If the Y group becomes an electro-donating group, and at the same time if there are some electron-withdrawing groups on the diene side, it is possible such a "reversed D-A reaction" could occur, like what we have used as a cover image, obviously, the methoxyl group is an electron-donating Y group.
An example of reversed D-A reaction
But don't get it wrong that, the reversed DA reaction could just also occur under heating, not UV is further required, since the required HOMO and LUMO are already there just as in the normal DA, we don't have to create excited new HOMO as what would happen in a UV condition.
3. The stereochemistry of the DA reaction
The next thing we need to know is the stereochemistry of the DA reaction, perhaps, people have already quite familiar with the so-called "endo selectivity", which is mostly developed by Woodward. To better understand the reaction selectivity, the ultimate important thing you want to know is that the electro-withdrawing group tends to draw itself closer to the diene system, the two just could interact with each other in a "secondary molecular orbital" sense.
The image and note are from the below article.
Just be careful here, as different people will use different methods to define what is endo, they are ultimately the same but just too many groups or ways you could adopt to describe it, i.e, some people like to use the position of the Y group while some other prefer to use substituents from the diene, to help themself track what is endo and what is Exo, just don't be confused by these different standards. In the above example, the original article uses the location of the anhydride which is just the Y group, to define the endo and exo. It is kind they have to since there is no substituents on the diene.
image is from the above internet link: Endo vs Exo
As we have mentioned above, one explanation why endo is preferred even in fact it is not as less sterically hindered as exo, is that in the endo transition state, there is a potential "secondary molecular orbital" interaction, which helps lower the energy barriers between HOMO and LUMO. The next image show it better, that the rich electro from the diene kind of "flow" to the electron-withdrawing Y group.
images from the above internet ink : Endo vs Exo
Another thing to remember is that Diels-Alder is a reversible reaction, which means the product could just decompose and move back to starting materials. This is not just a fact you want to know, but this also contributes to why the endo product is more preferred instead of the actually more stable exo product are obtained. That is even the exo is more thermodynamically stable than endo, endo is kinetically controlled or endo could forms faster than exo, by the time or extent that further heating could form some exo, it is more likely to decompose what ever the product is back to starting material, so one might just obtain not much exo product but just endo instead.
Further reading
The stereochemistry will become more complex if there are both substituents on diene and dienophiles, the terms like diastereoselectivity and enantioselectivity, plus the endo and exo, would be enough to create a good confusion puzzle, so just be careful when using these terms.