Description

When following the documentation on setting up automation tests, the sample provided for "Simple Tests" does not successfully compile. One of two compiles errors is given based on the TFlag used

Affected Flags:

Editor Context - All AutomationTests must have exactly 1 filter type specified
Client Context - All AutomationTests must have exactly 1 filter type specified
Server Context - All AutomationTests must have exactly 1 filter type specified
ApplicationContextMask - All AutomationTests must have exactly 1 filter type specified
nonNullRHI - All AutomationTests must have exactly 1 filter type specified
Requires User - All AutomationTests must have exactly 1 filter type specified
Feature Mask - All AutomationTests must have exactly 1 filter type specified
Disabled - All AutomationTests must have exactly 1 filter type specified
Smoke Filter - AutomationTest has no application flag.  It shouldn't run.
Engine Filter - AutomationTest has no application flag.  It shouldn't run.
Product Filter - AutomationTest has no application flag.  It shouldn't run.
Perf Filter - AutomationTest has no application flag.  It shouldn't run.
Stress Filter - AutomationTest has no application flag.  It shouldn't run.
Filter Mask - All AutomationTests must have exactly 1 filter type specified

Regression:

  • No - same compile errors occurs with 4.14.3
Steps to Reproduce
  1. Open UE4 Editor (any project)
  2. Add code to project based on FunctionalTest (MyFunctionalTest)
  3. Add the following to MyFunctionalTest.cpp (copied from documentation page)
    #include "AutomationTest.h"
    
    IMPLEMENT_SIMPLE_AUTOMATION_TEST(FSetResTest, "Windows.SetResolution", EAutomationTestFlags::SmokeFilter)
    
    bool FSetResTest::RunTest(const FString& Parameters)
    {
    	FString MapName = TEXT("AutomationTest");
    	FEngineAutomationTestUtilities::LoadMap(MapName);
    
    	int32 ResX = GSystemSettings.ResX;
    	int32 ResY = GSystemSettings.ResY;
    	FString RestoreResolutionString = FString::Printf(TEXT("setres %dx%d"), ResX, ResY);
    
    	ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(2.0f));
    	ADD_LATENT_AUTOMATION_COMMAND(FExecStringLatentCommand(TEXT("setres 640x480")));
    	ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(2.0f));
    	ADD_LATENT_AUTOMATION_COMMAND(FExecStringLatentCommand(RestoreResolutionString));
    
    	return true;
    }
  4. Comment out all but "return true;"
  5. Compile

Result:
Compile fails based on EAutomationTestFlags provided

Expected:
Automation test function compiles and can be run from UFE.

Have Comments or More Details?

There's no existing public thread on this issue, so head over toAnswerHub just mention UE-45291 in the post.

0
Login to Vote

By Design
ComponentAutomation
Affects Versions4.14.34.15.24.16
CreatedMay 22, 2017
ResolvedJun 2, 2017
UpdatedApr 27, 2018