Search Issue Tracker

Active

Under Consideration for 3.0.X

Votes

1

Found in [Package]

3.0.3

Issue ID

BUTFP-5

Regression

No

Allocations made in the SetUp() call are recorded in the performance log when using Performance Test Framework

Package: Performance Test Framework

-

How to reproduce:
1. Download the attached project "Profiling-GC-recorded-in-SetUp.zip"
2. Open Window > General > Test Runner
3. Run the test Tests.dll > ProfilingTest > WithAllocsDuringSetupAndNoAllocsDuringExecution_Profile_ShouldRecordZeroAllocs
4. Open Window > Analysis > Performance Test Report
5. Ensure the Test Report window has refreshed: Click "Refresh" or check "Auto refresh" in the top left
6. Observe the Time.GC() graph

Expected result: All values are 0
Actual result: All values are 201

Reproducible with: 1.3.3-preview, 2.8.1-preview (2020.3.36f1, 2021.3.4f1, 2022.1.5f1, 2022.2.0a17)

Comments (2)

  1. DwarfiusDan

    Sep 05, 2024 14:06

    I realized I made a mistake above, so here's the fixed resulting code:
    private double ExecuteSingleIteration()
    {
    m_Setup?.Invoke();

    if (m_GC) StartGCRecorder();

    var executionTime = m_Watch.Split();
    m_Action.Invoke();
    executionTime = m_Watch.Split() - executionTime;

    if (m_GC) EndGCRecorderAndMeasure(1);

    m_Cleanup?.Invoke();
    return executionTime;
    }

    private double ExecuteForIterations(int iterations)
    {
    var executionTime = 0.0D;
    for (var i = 0; i < iterations; i++)
    {
    executionTime += ExecuteSingleIteration();
    }
    return executionTime;
    }
    Also, be warned that likely why StartGCRecorder is not actively ran before every iteration by default is that it triggers GC.Collect - you'd need to patch more if you run many iterations (I did 512 iterations for 256 measurements, one run takes a whole minute instead of 1ms).

  2. DwarfiusDan

    Sep 05, 2024 13:22

    The fix for single frame tests is pretty straightforward(using 3.0.3 as reference):
    1) MethodMeasurement.cs, ExecuteForIterations - move StartGCRecorder and EndGCRecorderAndMeasure inside the else branch
    2) MethodMeasurement.cs, ExecuteForIterations - replace ExecuteActionWithCleanupSetup call with ExecuteSingleIteration

    That'll fix the GC tracking, but there's also an issues with SampleGroup tracking - they'll also include SetUp invocation

Add comment

Log in to post comment

All about bugs

View bugs we have successfully reproduced, and vote for the bugs you want to see fixed most urgently.