From my personal point of view it is very competitive world and as much as employer makes their own calculated offers to the stuff they hire, you also have a right to pursue better offers on the market. In order to prevent breach of conduct scenario the comapnies have codes of conducts put in place which clearly stipulate what they view as an infringement. I have gone through such routine online programme presenting multiple scenarios of how the laws can be violeted by emploees of "xyz" companies related to clients and competitors. You just had to answer the questionaire to pass it

.
I do not think the client company would take an open risk to hire you on the basis of knowledge of inside information as there clearly be some legal implications. Most likely they are interested in your skills, work style and ability to solve similar problems within their own working environement and organization. Many people have done that before and transferred.
There is also workplace factor coming in and I would be very open and honest with my previous employer plus workmates about my move to the new company. It is good to preserve good relationship, contacts, get references and not to burn bridges behind, especially in a niche market.