I work full time as a consultant for one client and I am employed by a payroll company, that pays all my taxes, social deductions, pension, etc. and and invoices my client.

I would like to now switch to a permanent contract with another company, but still do a little bit of consulting for my current client. It will be one invoice per month for let's say 500-2000CHF.

What is the best way to handle this situation?
I heard that in Switzerland it is not possible to be self-employed with only one client?
Can I invoice the client without being self-employed?

Regarding "self-employed with only one client"...
If it's only one client, AHV considers them an employer and it's an employer's job to pay your AHV, insurance, etc.
When you have 2-3 clients, then YOU are responsible for all social payments.

This is only relevant when you register with AHV as self-employed - you need to show them that you have several clients (show a couple of invoices, etc.) But in practice, if you, say, had 2-3 clients, lost 2 and now have 1 -- being self-employed with one client is acceptable. Because you are already registered and they know who pays the bills. I hope it makes sense.

If you had other consulting clients in the past, I'd show them those invoices too and register as a self-employed with multiple clients, no matter active or not.