Basically, you would need to have earned at least 40 credits (usually 10 years working in/making payments to the US) in order to be covered under US social security. There are provisions that allow adding foreign credits to US credits for people who have earned more than 6 but fewer than the 40 US credits and who because of working in multiple countries do not qualify under any one country's retirement system, but the formula gets complicated, and is subject to the various bilateral tax treaties.
As Gregv says, US social security is a tax, it's not payed into an account belonging to you - your payments go into the pot, to be split out among those drawing SS. The money paid in isn't yours per se.
For more information, see:
http://www.aca.ch/cadlinkj.htm
and also:
http://www.overseasdigest.com/odsamples/social22.html
Hopefully, there is something applicable to your individual situation there. Also bear in mind that the US has a Windfall Elimination Program - if you qualify for retirement benefits in both the US and a second country your US benefits may be reduced.
Your best bet would be to speak with the US Social Security Agency directly. I would suggest writing rather than calling the hotline, as your situation is somewhat out of the norm. The contact details are given in the first link.
---
Of course, all this supposes that US SS will still be around by the time you retire....

I'm hedging my bets - planning for my own retirement, and if by chance there is anything left in the US Social Security kitty by the time I retire that will be a nice extra. But I'm not counting on it.