I have installed a few generators, and also considered one for my current house. I agree with Matt, it is much easier to have a generator big enough for the whole house load, rather than have separate breakout circuits in a sub panel that are handled by the generator. The transfer switch needs to be sized for the whole hose load. Older panels/houses were often 200A, newer house (larger) house tend to have higher amperage panels, my house service entrance is designed with a 200A main panel and an additional 100A breaker can be added if needed so I would need a 300A transfer switch. So the problem is that 200A automatic transfer switches are fairly common and they run $600-700. These are usually automatic type so they have a genset exerciser, sense line drop outs and also wait a specified time before before transfer back to house power and then have a cool down period for the generator. The common brands are Kohler, Honeywell, ASCO, Eaton, Winco, Generac, etc.. A manual transfer switch runs about $400. So at the 200A level it probably pays to go with an automatic. But above 200A the price of the automatic transfer switches in the 300-400A are 2-3X more expensive than the 200A. A manual 300-400A switch runs about $800, more if for a service entrance. If you cannot breakout the power after your main house breaker then you need a service entrance rated transfer switch and UL approved, which is very pricey. My last house I was able to put the transfer switch after the main house breaker, but many service entrance panels the main breaker is incorporated into the mains panel and you then need to use a service entrance transfer switch which is much more expensive. Not to be pessimistic, but if the panel is not rated correctly or is wired wrong and something happens, your insurance can deny coverage.
I would go with a transfer switch sized to your current main panel, a 400A is way overkill unless you have a large shop or heavy equipment, in witch case you would be better to just take care of the house power. A 200A transfer switch would be adequate for a 40kW genset. As far as what I recommend, the first 4 brands above, Generac is somewhat iffy in my experience. I would get the correct sized transfer switch and you need to determine if it can be installed, and if you need a service entrance type. At 200A, I would go with an automatic switch if the genset can be wired for auto start. The automatic systems also have sensors for things like low oil, overspeed/under speed, etc.