If there is no gasoline leaking from the fuel lines then I would say yes, you will be looking at the fuel pump assembly.
It is an odd problem that it holds pressure when running???
Could be the lines in the tank or check valve, I would buy one to have on hand if problem can't be fixed once you inspect the assembly.
My '94 fuel pump is still working fine, 300,000
And there are alot of Rangers on the road compared to the fuel pump failures.
Yours is 11 years old, I doubt you would get that much from a non-OEM pump.
Have read of some 3rd party pumps that didn't last a year, they got a new pump under warranty but I would rather not have the hassle to R&R it again.