The settings in Visual Environment should not affect baking. However, if you set Ambient Mode to Dynamic and bake, it will not bake properly. Instead of baking a new result, it will repeat the last bake.

1) Open SampleScene
2) Clear baked lighting
3) In Lighting -> Environment, select Static Lighing sky:None
4) Bake. The three balls should be black (expected)
5) In Sky and Fog volume select Ambient Mode:Dynamic
6) In Lighting -> Environment, select Static Lighing sky: HDRISky
7) Clear bake
8) Bake. The result is black (not expected).
9) In Sky and Fog volume select Ambient Mode:Static
10) Clear
11) Bake again, Result is no longer black

Expected that in steps 8 and 12 the result should be the same. And the volume setting should not affect baking.

Note: if doesnt reproduce try switching to GPU baking.