I have been using the original Perko flag pole base with a Perko light pole I modified by putting a black ball on the end and adding Charlevoix clips. Although I have not had any problems with stress on the base fitting, I have been concerned about losing the flag and pole overboard in a strong wind. That is because the base and pole use a friction collar, rather than a threaded one.
Perko makes a heavy duty light pole base which has metal female threads, with a matching threaded collar on the accompanying light pole, and the required hole in the deck matches up with the existing hole for the original base. The swiveler sounds like a good idea, except I have a few flags of slightly different sizes, and the Charlevoix clips can be slid up or down the pole so I can use different flags. Also, changing the light pole base avoids the need to add another fitting to the transom area, and to drill another set of holes in the deck.