For a square receiver hitch, if you don't want gunk buildup, spray some silicone on it - it's a dry lube. Like others here, I remove it when I'm not using it actively.
For my goose-neck ball, I use a light coat of lithium-rich bearing grease which I regularly wipe off and apply fresh, before...