I do the same for a living as well and have a pretty good size shop , it’s not as easy has you think to get it to fit that tight and you wouldn’t want it that tight! Tighter is not always better3 x 3 x 1/4 wall a500 structural tubing exists all day long. That would put the ID at 2.5" nominal +/- .02 tolerance. I do manufacturing and machining for a living. It's easily within Ford's ability to source better material.
The sound isn't that big of an issue. My WD Hitch is loud enough but the real problem is the slop causes excess wear as the shank has room to wiggle it will egg out the hole.
Mill to mull you will find that their Tolerances very a little bit I assume that you know that too. I’m not saying that you can’t find it that fits maybe a little better but it isn’t anything that’s going to the job any better and by the time that it does what you say it possibly could do you will not be around to see it more than likely.