Reproduction steps:
1. Create new poject
2. Go to Build Settings>Player Settings
3. In Other Settings panel, uncheck "Auto Graphics API for Windows"
4. In Graphics APIs for Windows list, leave only Direct3D9
5. In Resolution and Presentation panel, set "D3D9 Fullscreen Mode" to Exlusive Mode
6. Save scene and add it to Build settings
7. Build & Run
8. On the Resolution dialog, choose non native resolution (different from your monitors)
9. Wait for game to load and quit the game (ALT + F4)

Expected result: Application quits
Actual result: Application crashes

Bug does not occur if build uses DX11

Reproduced with: 5.3.6p6, 5.3.6p8, 5.4.1p2, 5.4.1p3, 5.4.2f2, 5.4.2p3, 5.5.0b10, 5.6.0a1
Not reproducible with: 5.3.6p5 5.4.0f3, 5.4.1f1, 5.4.1p1
Regression introduced in 5.3.6p6 AND 5.4.1p2 (The bug was fixed on 5.4.0f3 and appeared on 5.4.1p2 again)

Note: Executable is also crashing when changing resolution from non native to any other but this does not occur since 5.4.2p2 roughly

Tested on Windows 10 pro, Intel(R) Core(TM) i5-6400 CPU, Radeon RX 460 Graphics