What they want is an affidavit, a sworn statement that you were either never married or no longer married. The notary is just an official means to verify your identity.
In theory you could do it in front of any public notary. But as we are speaking about bureaucracy we have to expect extra steps and stupid hurdles.
Better check with the Franch authorities if they are happy with an affidavit notarized by the US Ambassadeur to Switzerland, or an US Consul in Switzerland (the service is normally also offered in Zurich and Geneva, but both are closed due to COVID-19. Even Bern has only limited consular services), OR even better, if it can also be done in front of any French notary public.
See Affidavit of Civil Status
https://ch.usembassy.gov/u-s-citizen...s-we-notarize/
If it must be done at an US Ambassador/Consul also ask if you need an apostille as well on top of it. To get the apostille you have to send the notarized affidavit to the US Department of State
https://travel.state.gov/content/tra...tications.html