Welcome to the forum
Automotive Relay has no thermal protection circuit, so it doesn't shut off when hot, it can melt a wire or the plastic and shut off but it will not work again.
Fuel pump should draw 6 to 9 amps, it could be failing and drawing too many amps causing relay to heat up.
Fuel pump motor does have a thermal shut off, so will shut off if it gets too hot, it will reset once it cools down.
A relay has a coil of wire around a metal rod, when power passes thru that coil of wire the metal rod becomes a magnet and pulls down on the Contact bar, that passes the voltage to the fuel pump.
But that coil of wire is like a light bulbs coil of wire, or a toasters coil of wire, it does heat up just like a light bulb or toaster does, so a relay that is "on" for awhile will be warm to the touch, that is normal.
Corrosion in the relays socket can cause it to heat it up as well