Zeal Crown

Yes, some Loyalists / Tories ended up moving to Canada either because of harassment and pressure, though more moved back to Great Britain. Even after the Napoleonic Wars, after James Monroe stepped down the Election of 1824 was very controversial and caused what we called "partisan strife". And since you have a lot more of the common white male voting, there's a tendency to hear more of that "moving to Canada" comments go around.

Also, as said by Andrew Jackson himself: "If that gimpy, bald-headed, beslubbering prig is elected … I'm moving to Canada."

Huh, maybe you're right. I've never studied US history all that much, but it seems rather difficult to disprove anything you've said. I'd have to look more into that myself.