We had our first ten months after moving here! Not quite how we'd planned it but we haven't regretted it at all. I was 26 when he was born too - the whole thing was an excellent experience (as much as giving birth can be when the baby's head seems to be the size of a watermelon thanks to your husband's lovely genes) but I hear that varies from canton to canton.
On the family side of things, I think the grandparents have visited us more often than they might have done if they hadn't become grandparents.

But we've made good friends and generally haven't missed having family nearby to help out.
We're also not really the going-out type so we haven't missed that at all. As for travel, google "round the world trip with kids" and you'll be amazed at how many families are not at all held back by one, two or even five children.

And like the others said, travelling with babies is easy, whether short or long trips. We took ours to New Zealand for a month when he was 18 months and it was fine. It just requires a wee bit more planning than you might otherwise do. But I don't think we currently travel much less than we would otherwise do, even now we have two over the age of two years. If you really want to travel, you'll find a way of doing it with the kids, and they'll enjoy it too.
Best of luck if you decide to go for it!