Sorry for being the wet blanket here, but my guess is that he's in the Crystal Empire because they want to keep him as a background pony they can bring up by going to the Crystal Empire. If he moved to Ponyville, they'd have to show how the others adapt to him, etc.
Back in non-soggy-sackcloth mode, since he was drawn to the Crystal Empire in the first place and has now been accepted there, by all the royals no less, he'd probably want to stay there. I do hope he visits Ponyville someday though.