Testing¶

Summary: Project Eternal uses UE5's Automation Framework with BDD-style DEFINE_SPEC tests. Unit tests run synchronously with manual mocks, integration tests use LatentIt against a real backend. Shared helpers in TestHelpers.h create pre-configured items and save data. Tests live under Private/Tests/ and are stripped from packaged builds.

Table of Contents¶

Framework
Test Structure
Writing Tests
Shared Helpers
Mocking
Integration Tests
Proving Guard Tests Are Real (Mutation Testing)
Running Tests
Source References
Recent Changes

Framework¶

BDD-Style with DEFINE_SPEC¶

All tests use DEFINE_SPEC (Describe/It/BeforeEach/AfterEach) rather than IMPLEMENT_SIMPLE_AUTOMATION_TEST. This matches the BDD style already used by the ProceduralDungeon plugin in this project.

DEFINE_SPEC(FMyFeatureSpec,
    "ProjectEternal.Unit.MyFeature",
    EAutomationTestFlags_ApplicationContextMask | EAutomationTestFlags::ProductFilter)

void FMyFeatureSpec::Define()
{
    Describe("Feature behavior", [this]()
    {
        It("should do expected thing", [this]()
        {
            TestEqual("Value", Actual, Expected);
        });
    });
}

UE 5.8 Flag¶

Use EAutomationTestFlags_ApplicationContextMask (underscore, not ::) — this changed in UE 5.5.

Test Filter Naming¶

Pattern	Scope
`ProjectEternal.Unit.*`	Unit tests (no external dependencies)
`ProjectEternal.Integration.*`	Integration tests (requires running server)

Test Structure¶

Source/ProjectEternal/Private/Tests/
├── TestHelpers.h                              # Shared factory functions
├── Mocks/
│   ├── MockPersistenceApiService.h            # Mock for URemotePersistenceProvider
│   └── MockItemRegistry.h                     # Mock for item reconstruction
├── Persistence/
│   ├── PersistenceTypes.spec.cpp              # Data struct defaults + JSON round-trips
│   ├── PersistenceHelpers.spec.cpp            # CreateItemSaveState extraction
│   ├── ItemReconstruction.spec.cpp            # CreateItemFromSaveState restoration
│   ├── LocalPersistence.spec.cpp              # Local file provider (real I/O)
│   ├── RemotePersistence.spec.cpp             # Remote provider with mock API
│   └── ServerPersistence.spec.cpp             # Full HTTP integration tests
└── Replication/
    ├── ReplicationTestBase.cpp                # Base class: world/player access, wait helpers
    ├── ReplicationTestGameMode.cpp            # Test GameMode: skips persistence + travel
    ├── ContainerReplicationTest.cpp           # Add, remove, swap items
    ├── EquipmentReplicationTest.cpp           # Equip, unequip, replace, inventory flow
    ├── CrossSystemReplicationTest.cpp         # Cross-container registry moves
    ├── InventoryIsolationTest.cpp             # Player inventory isolation
    ├── QuestReplicationTest.cpp               # Quest tag add/remove
    ├── GlyphReplicationTest.cpp               # Stone plate equip flow
    ├── AttributeReplicationTest.cpp           # GAS attribute replication
    ├── CombatReplicationTest.cpp              # Hit severity, invincibility, combo counter
    ├── CraftingReplicationTest.cpp            # Crafting resource count
    ├── ExplorationReplicationTest.cpp         # Zone switch
    ├── WorldMapReplicationTest.cpp            # World map state
    ├── ContainerReplication.spec.cpp          # Unit: FastArray operations
    ├── EquipmentReplication.spec.cpp          # Unit: Equipment slot assignment
    └── ItemRegistryReplication.spec.cpp       # Unit: Registry item tracking

Headers live in Public/Tests/ following the standard Public/Private split.

Current Test Coverage¶

Unit Tests (DEFINE_SPEC)

Spec File	Tests	Type	What It Tests
PersistenceTypes	6	Unit	Struct defaults, JSON serialization round-trips
PersistenceHelpers	7	Unit	`CreateItemSaveState` — extracting save state from live items
ItemReconstruction	7	Unit	`CreateItemFromSaveState` — rebuilding items from save state
LocalPersistence	6	Unit	File-based provider: save, load, overwrite, delete, edge cases
RemotePersistence	8	Unit	Remote provider with mock API: delegation, error propagation, deserialization
ContainerReplication	24	Unit	FastArray add, remove, grid position, subobject registration
EquipmentReplication	5	Unit	Equipment slot assignment, GUID tracking
ItemRegistryReplication	9	Unit	Registry item tracking, cross-container moves
ServerPersistence	7	Integration	Full HTTP pipeline against real backend server

Functional Replication Tests (AFunctionalTest)

Test Class	System	What It Tests
`AContainerAddItemTest`	Inventory	Item add replicates to client
`AContainerRemoveItemTest`	Inventory	Item removal replicates
`AContainerSwapItemsTest`	Inventory	Position swap replicates for both items
`AInventoryIsolationTest`	Inventory	Player A's items don't leak to Player B
`AEquipItemTest`	Equipment	Equip populates client slot
`AUnequipItemTest`	Equipment	Unequip clears client slot
`AEquipReplaceItemTest`	Equipment	Swap replaces item in slot
`AEquipFromInventoryTest`	Equipment	Inventory → equipment cross-system
`AUnequipToInventoryTest`	Equipment	Equipment → inventory cross-system
`ACrossContainerMoveTest`	Cross-System	Registry-driven container move
`AQuestStartTest`	Quest	Quest tag add replicates
`AQuestCompleteTest`	Quest	Quest tag remove replicates
`AGlyphStonePlateEquipTest`	Glyph	Stone plate equip via real inventory flow
`AAttributeReplicationTest`	GAS	MaxHealth + BaseDamage replicate via ASC
`ACombatHitDirectionTest`	Combat	HitSeverity replicates on character
`ACombatInvincibilityTest`	Combat	bInvincible replicates on character
`AComboCounterReplicationTest`	Combo	ComboCounter replicates on character
`ACraftingResourceReplicationTest`	Crafting	Resource count replicates after AddResource
`AExplorationZoneReplicationTest`	Exploration	CurrentZoneID replicates after zone switch
`AWorldMapStateReplicationTest`	WorldMap	ReplicatedMapState.CurrentNodeID replicates

Writing Tests¶

Naming Convention¶

Spec class: F{System}{Feature}Spec Filter: "ProjectEternal.{Unit|Integration}.{System}.{Feature}"

UObject Lifetime¶

Tests create UObjects outside of a normal game world. Use TStrongObjectPtr to prevent garbage collection:

TStrongObjectPtr<UMyObject> Obj(NewObject<UMyObject>());
// Obj stays alive until TStrongObjectPtr goes out of scope

Shared State Across Lambdas¶

Describe/BeforeEach/It blocks are lambdas. To share UObject-owning pointers between them, wrap in TSharedRef:

Describe("SaveAndLoad", [this]()
{
    auto Provider = MakeShared<TStrongObjectPtr<ULocalPersistenceProvider>>();

    BeforeEach([this, Provider]()
    {
        *Provider = TStrongObjectPtr(NewObject<ULocalPersistenceProvider>());
    });

    It("should save data", [this, Provider]()
    {
        (*Provider)->SaveCharacter(Id, Data, Callback);
    });
});

Cleanup¶

Use AfterEach when tests create side effects (files, state):

AfterEach([this, CharacterId]()
{
    // Delete test save file
    IFileManager::Get().Delete(*GetSaveFilePath(*CharacterId));
});

Known Limitations¶

ensure() assertions do not suppress cleanly with AddExpectedError. Avoid writing tests that intentionally trigger ensure() failure paths. If a function guards a null path with ensure(), skip that edge case test and document why.

Shared Helpers¶

Private/Tests/TestHelpers.h provides inline factory functions in the EternalTestHelpers namespace:

Function	Returns	Fragment
`CreateTestBasicItem`	`UItemObject*`	None (minimal item)
`CreateTestEquipmentItem`	`UItemObject*`	`FEquipmentFragment` with CritChance prefix + BaseDamage implicit
`CreateTestConsumableItem`	`UItemObject*`	`FConsumableFragment` with configurable usages
`CreateTestStackableItem`	`UItemObject*`	`FStackableFragment` with configurable stack count
`BuildTestModifiers`	`FEquipmentModifiers`	One entry in each of 4 arrays (implicits, prefixes, suffixes, scaling)
`CreateTestStonePlateItem`	`UItemObject*`	`FStonePlateFragment` with 4x4 plate layout
`CreateTestCharacterSaveData`	`FCharacterSaveData`	Fully populated: items, inventory slots, equipment slots, quests, world map, exploration

All helpers use FGameplayTag::RequestGameplayTag(Name, false) for safe tag lookup during CDO construction.

Footgun: Mutate the Default Fragment, Don't Append a Second¶

FItemManifest's constructor already seeds a default FGridFragment, and GetFragmentOfType<T>() returns the first matching fragment. When a helper needs to set an item's grid footprint (e.g. CreateTestSizedItem), it must mutate the existing default via GetFragmentOfTypeMutable<FGridFragment>() — do not append a second FGridFragment. A footprint set on an appended fragment is silently ignored because lookups return the seeded first one. This exact bug made a "2x4" item secretly resolve as 1x1.

Do	Don't
`Manifest.GetFragmentOfTypeMutable<FGridFragment>()->Dimensions = Size;`	`Manifest.Fragments.Add(FGridFragment{...});` (ignored)

Related: merge match is on GetItemType() (a tag) — set it with SetItemType to a registered tag; empty tags never MatchesTagExact and will never merge.

Replication Functional Tests¶

Why Functional Tests?¶

Unit tests (DEFINE_SPEC) verify logic in isolation. Replication tests verify that state changes on the server propagate to clients correctly. These require a real multiplayer PIE session — they cannot run headless with -NullRHI.

Architecture¶

┌──────────────────────────────┐
│  AReplicationTestBase        │
│  (extends AFunctionalTest)   │
│  • World/player accessors    │
│  • WaitForReplication()      │
│  • bEnabled toggle           │
│  • Data asset item creation  │
└──────────────┬───────────────┘
               │ inherits
    ┌──────────┴──────────┐
    │ AEquipItemTest      │
    │ AQuestStartTest     │
    │ ACombatHitTest      │
    │ ... (20 test actors)│
    └─────────────────────┘

How They Work¶

Test actors are placed in ReplicationTestMap with AReplicationTestGameMode
PIE runs as Listen Server + 2 players
Each test auto-starts on the server after a staggered delay
Server mutates state, waits N frames for replication, verifies on client
Results logged to Output Log ([ReplicationTest] TestName: PASSED/FAILED)

Test GameMode¶

AReplicationTestGameMode inherits AEternalGameMode but: - Skips persistence (deferred auto-load disabled via SetDisabled()) - Skips world map travel (calls AGameMode::HandleStartingNewPlayer directly) - Skips SpawnManager gating (calls AGameModeBase::SpawnDefaultPawnFor directly) - Sets CharacterClassInfo on PlayerState from parent's field

Data Asset Support¶

Item-based tests have UPROPERTY(EditAnywhere) fields for UItemManifestDataAsset. Assign real production items in the editor for maximum coverage. Synthetic fallback items are used if no data asset is assigned (for CI without editor setup).

Adding New Replication Tests¶

Create header in Public/Tests/ inheriting AReplicationTestBase
Create implementation in Private/Tests/Replication/
Override RunReplicationTest() — mutate on server, WaitForReplication(), verify on client
Place actor in ReplicationTestMap
Assign data assets if the test uses items

Running Replication Tests¶

PIE only — set Number of Players to 2, Net Mode to Listen Server, press Play.

Mocking¶

Approach¶

Manual mocks via UCLASS inheritance — no external mocking framework. Mocks override virtual methods and expose captured state for assertions.

MockPersistenceApiService¶

Replaces real HTTP calls for URemotePersistenceProvider tests.

Configuration (set before test): - bShouldSucceed — control success/failure - MockStatusCode, MockErrorMessage — error details - bReturnMalformedJson — test deserialization error path

Captured state (inspect after test): - SaveCallCount, LoadCallCount — call counts - LastSaveCharacterId, LastSaveData — captured arguments

Realistic deserialization: On load, the mock serializes FCharacterSaveData to real JSON via FJsonObjectConverter, then deserializes into FApiResponse.Content. This exercises the same deserialization path as production code.

MockItemRegistry¶

Replaces asset-based item manifest lookup for CreateItemFromSaveState tests.

Registration helpers: RegisterBasicManifest, RegisterEquipmentManifest, RegisterConsumableManifest, RegisterStackableManifest — populate a TMap<FString, FItemManifest> instead of loading data assets.

Integration Tests¶

ServerPersistence.spec.cpp¶

Requires a running eternal-server at http://localhost:5065/ with PostgreSQL.

Key patterns: - Uses LatentIt with FDoneDelegate for async HTTP operations - Creates unique account + character per test to avoid collision - Wires up UEternalApiSubsystem with all three services against the real server - Verifies full HTTP save/load pipeline including JSON serialization through real network

Filter: ProjectEternal.Integration.Persistence

These tests are separated from unit tests so CI can run unit tests without a backend.

Proving Guard Tests Are Real (Mutation Testing)¶

A test that guards a behavior (a revert-on-fail, a count-0 rejection, a broadcast) is only meaningful if it can actually fail when that behavior breaks. A test that still passes after you sabotage the production code is tautological — it asserts nothing.

The bar: every guard test must be proven real by mutation before it is trusted.

Procedure¶

Break the guard — introduce a plausible mutation into the production code the test covers (e.g. delete the early-return that rejects a count of 0, remove the displaced-fit revert, skip the PostReplicatedChange broadcast).
Run the test — confirm it now FAILs. If it still passes, the assertion is not actually exercising the guard — fix the test.
Restore — revert the mutation and confirm the test passes again (PASS=n FAIL=0).

Guidance¶

Situation	Verdict
Test fails on a plausible mutation, passes when restored	Real — keep it
Test passes even with the guard removed	Tautological — rewrite until a mutation breaks it
Proving a broadcast fired	Bind a probe `UFUNCTION() void OnChanged(){ ++Count; }` via `AddDynamic` to the multicast and assert `Count == 1` — asserting "didn't crash" is not enough

This was applied to the itemization count-0 merge guard, the swap displaced-fit revert, and the PostReplicatedChange broadcast.

Running Tests¶

CI (Automated)¶

Unit and mock-based tests run automatically in PR validation. See CI/CD Pipeline for details.

CLI (Headless)¶

# Run all unit/mock tests (same filter used by CI)
UnrealEditor-Cmd.exe <project.uproject> \
    -ExecCmds="Automation RunTests ProjectEternal.Persistence;Quit" \
    -NullRHI -unattended

# Run specific test group
UnrealEditor-Cmd.exe <project.uproject> \
    -ExecCmds="Automation RunTests ProjectEternal.Persistence.Types;Quit" \
    -NullRHI -unattended

# Run integration tests (requires eternal-server running)
UnrealEditor-Cmd.exe <project.uproject> \
    -ExecCmds="Automation RunTests ProjectEternal.Integration;Quit" \
    -NullRHI -unattended

CLI (Headless Runner Script)¶

The wrapper script builds (closing the editor first), runs the requested filter headless, and prints a PASS=n FAIL=n summary parsed from LogAutomationController results — the preferred way to run tests from the terminal.

powershell -File Tools/run_tests.ps1 -Filter <name> [-NoBuild] [-EnginePath <path>]

Flag	Purpose
`-Filter <name>`	Test filter, by prefix — e.g. `ProjectEternal.Unit.Inventory`
`-NoBuild`	Skip the build step and run the already-compiled binaries
`-EnginePath <path>`	Explicit engine root

Engine path resolution order: -EnginePath argument → UE_ENGINE_PATH environment variable → error if neither is set.

Live Coding blocks UBT — the build step cannot run while the editor holds a Live Coding session. Close the editor before running with a build (force-kill is acceptable), or pass -NoBuild to run existing binaries.

Note: the runner lives at Tools/run_tests.ps1 (moved from Saved/).

A sibling runner, Tools/run_validation.ps1, drives the content-validation commandlet with the same flag/engine-resolution conventions — see Content Validation. Content-shaped test suites also exist: ProjectEternal.Content.Dungeon.GenerationRegression runs every UDomainDungeonConfig × 8 fixed seeds (topology mapping, reachability, same-seed determinism) via the normal test runner.

Editor¶

Session Frontend > Automation tab > filter by ProjectEternal.

Adding New Test Files¶

Create Private/Tests/{System}/{Feature}.spec.cpp
Use DEFINE_SPEC with appropriate filter path
Add shared helpers to TestHelpers.h if reusable
Add mocks to Private/Tests/Mocks/ if needed
No build system changes needed — UE discovers spec files automatically

Source References¶

File	Purpose
`Private/Tests/TestHelpers.h`	Shared item/data creation helpers
`Private/Tests/Mocks/MockPersistenceApiService.h`	Mock API service for remote provider tests
`Private/Tests/Mocks/MockItemRegistry.h`	Mock item registry for reconstruction tests
`Private/Tests/Persistence/*.spec.cpp`	All persistence test specs
`Public/Tests/ReplicationTestBase.h`	Functional test base class for replication tests
`Public/Tests/ReplicationTestGameMode.h`	Test GameMode for multiplayer PIE
`Private/Tests/Replication/*.cpp`	All replication test implementations
`Content/Maps/FunctionalTests/ReplicationTestMap`	Test map with placed test actors

C++ Style Guide — Code conventions
Content Validation — Save/push/PR gates for data assets; validation cores are spec-tested
CI/CD Pipeline — Automated test execution in PR validation
API Layer — Backend communication (tested by integration tests)
Replication Overview — Replication concepts tested by functional tests
Persistence architecture — ImplementationDocs/PersistenceLayer.md

Recent Changes¶

Date	Change	Reason
2026-07-08	Point to `run_validation.ps1` + note `Content.Dungeon.GenerationRegression` suite	Validation spine + B-8 regression suite shipped
2026-07-03	Add mutation-testing bar, headless runner (`Tools/run_tests.ps1`) CLI, TestHelpers default-fragment footgun	Promote automation-test patterns from session learnings
2026-03-14	Add replication functional test documentation	20 functional tests covering 11 replicated systems
2026-03	Initial documentation	Document testing patterns established with persistence layer