Audacity is, in my opinion very useful, if used correctly.
If you're looking for on-the-go, then the Samson GoMic may be a good idea. It's relatively cheap, and it really portable and small. It comes with Cakewalk Music Creator LE. The mic runs for 35 USD on Amazon.
If you wanted to, you could go all-out and get yourself a ZoomH2n, which is also portable. It comes with Steinberg Wavelab 8 LE. This mic runs for 160 USD on Amazon.
I've had a blast with both of them, and both are amazing microphones. If you're doing mainly commentary, just go with the GoMic. If you're going for commentary and music recording, go with the H2n.
There are many other mics in-between these price ranges that will do just the job you want. For instance, the AudioTechnica 2020, which is a well-known popular mic that runs for 130 USD, or the AudioTechnica 2500(I have this one too, it's a pretty good mic, though not fantastic imo), which is usually overlooked, but almost has the same quality as the 2020, for half the price (66 USD).
Make sure to read reviews, learn about Frequency Response, and use this website: http://recordinghacks.com/microphones
Have fun!
Best,
Krenton