I sent a package to Vail, Colorado. It turns out there are places where US mail doesn’t deliver. I had paid the expensive postage but the packet was returned. Who is responsible for the non-delivery and possible refund?

That's weird. What was the reason for non-delivery? I thought USPS would deliver to any legit address. Rain, snow, dark of night and all that.

So did I but it doesn't and I've got the stamp to prove it.

If you sent your package with Swiss Post then you made a contract with them. You paid them for a service which you didn't receive. So the refund should come from the Swiss Post. They can follow this up with USPS and find out what happened.