Well, it's never been confirmed that only Fort Treehouse was destroyed, so it's possible there still was some offscreen collateral damage. Also, AFAIK, the library was placed in a somewhat empty area, so the debris might just not have flown away far enough (but I doubt that, considering stuff like grenade shrapnel can litterally fly hundreds of meters away).
I hope we'll see some more stuff about it in Season 5, because a charred tree in the middle of a town is indeed fucking weird. Also, Ponyville lacks a (public) library now. I hope they'll at least repair it and get a new librarian if Twalot isn't going to be the librarian again (which she isn't, because, y'know. Cold Crystal Treehouse that shouldn't be there is there),