If you're handy with installing and uninstalling thermostats, then yeah you could put a nest in, keep the old one, then when you leave take the nest with and throw the old one back in! But as someone said, check with your landlord first.
It's a wonderful replacement for your crappy old thermostat, but my understanding is that it's not a simple plug & play type of thing.
You'd have to keep the old thermostat to reinstall it later, which may require keeping notes for uninstalling Nest and reinstalling the normal thermostat.
Additionally, you may want to check your rental agreement to determine whether or not your landlord is okay with you messing around with electrical equipment.
This product is awesome. You can put in a house, apartment, or rental. I love the total control you have from anywhere.
I live in a desert community so being able to turn on the a/c before I'm home by using my phone is great.
Being able to turn it off the same way if I forget to before I leave home.
Simple to install!
As long as you're fairly comfortable with DIY stuff it shouldn't be too bad to undo your current setup and keep track of the wires. You could use it while you're there, then again just refer to your notes to reinstall the old unit.