Developer Notes

ExecCmds="Automation" does not support queueing several Run requests.
Either RunFilter, RunAll or RunTest should be used.
You can set filter additionally to a Run request to further narrow down the test collection.
Also -testexit= is legacy and is not the best way to exit the editor after executing an Automation command.
Instead use:

-ExecCmds="Automation RunFilter Smoke;Quit"
Description

Repro Rate: 3/3

This issue was reproduced in //UE5/Release-5.1 at CL 23058290.

This is confirmed to not be a regression since the error also occurs on the previous live version. This is regression tested on //UE5/Release-5.0 at CL 20979098.

There seems to be something wrong with not finding any workers. Check the attached log for more details. Running the command only with a single "Automation RunTests Project" or "Automation RunFilter Smoke" works perfectly fine, the issue only occurs when combining the two in a single command.

 

Steps to Reproduce

Steps:

  1. Open cmd in the Workspace/Engine/Binaries/Win64 directory.
  2. Run the following command:
    • UnrealEditor.Exe [Project.uproject] -ExecCmds="Automation RunTests Project, Automation RunFilter Smoke" -unattended -nopause -NullRHI -testexit="Automation Test Queue Empty" -log -abslog=RunTests.log
    • Replace [Project.uproject] with the path to the uproject file in quotations.

Actual Result: The command is stuck in an endless loop and never starts any tests.

Expected Result: The Automation tool runs all Tests that match the "Project" Filter and then all tests that match the Smoke Filter and then exits.

Have Comments or More Details?

There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-170942 in the post.

1
Login to Vote

Fixed
ComponentUE - Foundation - Cpp Tools - Automation Tool
Affects Versions5.0.35.1
Target Fix5.2
Fix Commit23416127
Main Commit23416127
CreatedNov 23, 2022
ResolvedDec 6, 2022
UpdatedJan 18, 2023