Yes, prior to ww2, the swastika was in widespread use, for instance:
As for "retarded" it was a deliberately chosen word. USSR (Father Sunshine, J.V.Stalin) did admit in the negotiations that Finnish insignia did not have any connection whatsoever with the German Reich. Also, it is bit exaggerated to call it "Allied" Control Commission because it consisted solely of Russians. Retardedness comes from those who initiated the change, and only in national insignia. I guess they just wanted to show that they won even they could not occupy the country. In the same went many pre-war organisations as "fascist" (like sport klubs, aviation clubs and so on, to pave on the eventual occupation).
We have plenty of active unit flags and citations which have the swastika on.
Finnish Air Force Flag
Finnish Defence Forces flag
But yes, there was the pretext that a roundel was more modern, and so be it.