We had a tankless on the old house, installed it when we did the repipe. A few random thoughts:
The biggest wins are that it takes up less space and you get infinite hot water. No more cases of your teenage daughter taking a really long shower and using up everything in the tank.
Downsides are that it's more expensive, you wait a bit more for hot water, and it usually requires a different vent than a traditional water heater. It's either too hot or not hot enough or something, I forget which. Also, because it needs all the energy at the same time, you'll need a bigger gas line running to it -- ours was 1.25 inches, IIRC. I'd never go electric on something like that. Another downside is that since the brains require electricity run you don't have hot water when the power is out, even for a gas unit. (I guess you could use a UPS).
The biggest downside is that, according to the plumbers I've talked to, when they break they tend to be down for a long time. Traditional water heaters are simple, easy to troubleshoot, and the parts are in stock at the store down the street. Tankless units are a lot harder to troubleshoot and require parts that take several days to order, during which time you've got no hot water. The new house has a traditional tank for this reason.
As for energy savings -- that's a fantasy. You'll never pay back the additional purchase/installation cost through lower monthly energy bills before the tankless needs replacement.
Oh, it also needs to be descaled annually. Install it with the service valves, get 5 gallon bucket full of white vingegar, a pond pump and some hose remnants, disconnect it from the water system, turn off the gas (so the burner doesn't light) and pump the vinegar through it for half an hour.
If you live somewhere that it doesn't get that cold, you can mount it on an outside wall. Simplifies the venting, saves lots of space, and makes the descaling process less of a mess, but you need to run more pipes.