No you don't need to pull the head but do need to pull the upper and lower intake manifolds.
Also as a heads up the 2002-2006 3.0ls had issues with exhaust valve seats, they would come loose preventing the valve from closing, causing valve train noise and misfiring that could come and go.
Loose rocker arm could be that and not the lifter.
Check height of other closed exhaust valves.
I doubt it will fit but get a new lifter and try it in the hole with pushrod removed.
I have a "magnet on a stick" that I use to retrieve nuts and bolts, it would hold the weight of a lifter with no problem.