If you click the X (top right) after clicking Add Layer, it will still add a layer when it's expected that it doesn't do that.

Repro:

with Terrain Tools installed
0. add new Terrain object to hierarchy, select it and view in inspector
1. from terrain properties, expand Layers section
2. Click Add Layer to show the Select TerrainLayer popup window
3. Select a TerrainLayer, but don't double click it
4. Press X at the top right of the window

Expected:
That pressing X cancels the action of adding a layer.

Actual:
layer is still added even though you clicked "X"