We have a small ~1200 ft^2 house, and even at that we've found one of the things you'll have to come to terms with is that you pretty much can't hire somebody to do any major project for less than ~$10k. Need a new roof? ~$10k. Need new windows? ~$10k. Need a new retaining wall? ~$10k. Need a new (small) bathroom? ~$15k, etc. Substquently, smaller projects like water heaters and carpet will generally seem to start in the $1k range.
Sure you can get it done cheaper, and we have, but usually that means sacrificing quality of materials, quality of craftsmanship, or both...Or you can try and figure out how to do it yourself and sacrifice copious amounts of what little free time you might otherwise have. It's a painful pill to swallow for what is commonly not actually very technically difficult work, but there is always going to be that price to pay at some level. And even without cheaping out, all but the best contractors will still leave imperfections behind that will bug you at first but if your at all detail oriented. Take solace that they will at the very least bug you somewhat less with time. There simply is no such thing as 'perfect' if you're really looking. One other tip we've learned the hard way is to always get at least 3 bids, and never go with the lowest one.
Having a house built to your specs would be neat, but baseline contractor grade stuff is typically the cheapest versions of everything. Thus, unless you're willing to pay quite a bit more to get everything upgraded from day one, within 3-5 years it's going to begin the cycle of constantly needing maintenance, just like any other house. Admittedly the older the house though, the more...uh...'interesting' past repairs/solutions you're likely to encounter. But that's all part of the 'character' of older houses. It's always fun having professionals come to quote a job, and watching them try to figure out WTF the previous owner(s) were thinking. LOL.
Many personal wealth gurus/books/seminars will tell you that your home is your single greatest asset. However, few will freely admit that it's also far and away your single greatest liability. This one sided 'investment' mentality is what gets people buying the most house they can afford. In most housing markets most of the time, when all is said and done over the years, you'll likely have made little (if any) more on your house than you would have sticking that same money in the stock market. The key is that you do still need a place to live anyways, so in the long run it is at least that much better than renting. So if you view buying your house from that more pragmatic perspective, merely a better long-term alternative to renting rather than an investment, you'll be more likely to buy below your means. Which will free up funds for pure investments.
Yes, if you buy things in the right place at the right time, and/or for the right price, real estate can be a great investment too...But that really shouldn't be the primary intent for most peoples personal home.
And if you look at how long you need to live there to 'break even', remember to also take into account the closing costs when buying, and even more importantly the RE agent commissions and any sales/excise taxes.