It seems that I was not fully right.
As this page
explains in the section "Détail des déductions relatives à l'impôt cantonal et communal incluses dans le barème de l'impôt à la source 2015
", some deductions are already included in the "À la source" tax, including contributions to 2nd pillar.
So for instance, "à la source" tax for 100k income will be much less
than "regular" tax for the same 100k (if you do the declaration), because of "deductions" already included in it.
It makes sense and explains everything.