Coolant burning smells sweet and isn't "smoky" it dissipates fairly fast, oil burning doesn't smell sweet and is smoky, sticks around as it burns.
Most likely valve cover gaskets need replacing.
I haven't done this on a 4.0l SOHC but reads like it is just common sense wrench work, outside of a longer socket extension no special tools are needed.
Simple how-to here: 4.0L SOHC Valve Cover Gasket Job - Some Notes - Ford Explorer and Ranger Forums "Serious Explorations"®
I would also replace your PCV Valve, the Positive Crankcase Ventilation(PCV) system uses the engines vacuum to keep Blow-by from polluting the air.
Blow-by comes down past the piston/rings each time a cylinder fires, it will create a pressure inside the crankcase and valve covers which can push oil out of these areas.
A working PCV valve keeps the blow-by pressure in the negative pressure area, so helps prevent leaks from starting.