Steps to reproduce:

1. Open attached project
2. Rebuild lightmaps for both scenes
3. Build and run project
4. In build press "A", this will Additively load scene "scene2"
5. Now press "S", this will call SetActiveScene on "scene2", notice that lightmaps of "scene1" become glithcy
6. Finally, press D to unload scene "scene1"
7. In built project's .exe folder there should be "log.txt" which recorded list of LightmapSettings.lightmaps throughout steps 4-6
8. Right at the top of that file, there's list which shows that there are 7 lightmaps
9. In the middle of the log file there should be line "Set scene2 as active scene!". Notice that Lightmaps list after this line only contains two lightmaps

Not reproducible in editor.

Reproduced with: 5.3.1f1, 5.3.4p3, 5.4.0b14