Had a look at notice 704 and ended up phoning HMRC to get clarification (they directed me to Notice 703 as sale was an internet/mail-order sale rather than face-to-face purchase.
For anyone in similar situation the routine is as follows:
- Retailer charges VAT on invoice as goods are being shipped to EU (unless customer has relevant VAT number as discussed earlier)
- Customer upon exporting the goods themselves outwith the EU (in this case Switzerland), gets the invoice stamped by Border Control and sends copy to Retailer.
- Retailer if satisfied that goods have now become "Exports" can adjust the VAT and refunds customer.
Thanks all for your responses.