Do you mean the XS650? They used XS in the names of more than one Yamaha (XS1100, for one example.)
If I remember correctly, 1975 or later were considered to have the best chassis. I'd stay away from the 650 Specials, as they had a fake chopper styling (high handlebars, big rear tire, funky body panels) that don't lend themselves to cafe racer conversions well. For aftermarket cafe racer parts, check out Omar's: http://omarsdtr.com/manxnew1.html The simplest thing is just stick a set of clubman handlebars on a stock bike and call it done - I used to have a '74 650 and that was what I did. Looked great and was fun to ride. As far as pricing, $1/cc is a doable price for older bikes, if you keep your eyes open (although deals like that can be hard to find.) I bought my 650 for $300, which was a fluke - the guy just wanted to get it out of his garage and I was lucky enough to be the first person to respond to the classified ad. I just checked my local Craigslist and found two 650s, a Special listed for $1200 and a regular one a few weeks ago listed for $650.