[Test Runner]Clicking Rerun Failed button after Run all in play causes the test player to not close after running tests

To Reproduce:
1) Open Attached project
2) Open Test Runner
3) Click Run All to run all tests (some of the tests will fail, that's expected)
4) Click Run all in player (Standalone..)
5) As the build is happening start clicking on Rerun Failed to click it immediately once that becomes possible

Expected Result: Tests run, the launched player closes and you get the results
Actual results: Tests run, the launched player stops running. This causes weird console setting fighting in 19.1

Reproduced in: 2019.1.0b6, 2019.1.0a8, 2019.1.0a1
Does not reproduce in: 2018.3.6f1, 2019.2.0a7