I hear ya (Y'ALL !!!)
Ok, after you've got the spacer installed, you should have a slightly larger endshake than you want to end up with. It's easy if you have a sacrificial cylinder to use as a tool. I use an ASM Walker cylinder for Dragoons/ Walkers and an ASM Navy for all belt pistols. With 220 grit sandpaper, I stand the arbor in the appropriate cyl and sand the arbor down by moving the cylinder on the paper ( and checking often) until I get to my specific endshake - .002" max for cartridge, .0025" - .003" for cap guns.
Doing it that way (shortening the arbor a little more) will lengthen the wedge slot overall. That is where the set screw in the end of the arbor comes into play. Being able to "adjust" the slot length will allow you to keep your wedge and not have to buy an oversized wedge or make or have one made.
One thing I'll add is I would never use a set screw to be the correction of the arbor length. What you're doing is reducing the "connection" for the transmission of force to the diameter of the set screw. I'm not saying it's wrong if that's what you want to do, it just doesn't make any sense to me to do it that way.
I'll add some pictures. ( hopefully after this. They tend to go to the top of the post which takes them out of "context ".