Day 12: The test harness, fakes and fixture contracts

Today is the test suite: the shared harness in test/support/, one widget test, one repository test, the fixture contract test, and a timer test under a fake clock. These are the files you will copy from when you add a feature, and the contract test is the first thing that goes red when the Phoenix side changes a payload. By the end you should be able to run any single test, read a failure, and know which fake to reach for.

Before you start

You need three things from earlier days. Providers and overrides from Day 5: a ProviderScope can swap what any provider returns, and that is the whole testing strategy here. Repositories, Dio and the interceptors from Day 6: buildDio adds the bearer token and turns errors into ApiException. The queue notifier from Day 8, which polls on a Timer.

Run the suite once before reading, so the first run’s compile time is out of the way:

cd footrace-flutter
flutter test

Expect a few hundred passing tests and a block of skipped ones. The skipped ones are the screenshot renderer, which only runs with FR_SCREENSHOTS set. The reference page for all of this is Testing.

The fakes and testApp

Open test/support/fakes.dart and scroll to FakeAuthRepository (app/test/support/fakes.dart:69-105).

class FakeAuthRepository implements AuthRepository {
  final calls = <String>[];

  Future<AuthResponse> Function(String email, String password) onLogin =
      (_, __) async => testAuth;
  Future<AuthResponse> Function(String name, String email, String password)
      onRegister = (_, __, ___) async => testAuth;
  Future<MeResponse> Function() onMe = () async => testMe;

  @override
  Future<AuthResponse> login({
    required String email,
    required String password,
  }) {
    calls.add('login:$email');
    return onLogin(email, password);
  }

Three Dart ideas are in play. implements AuthRepository treats the abstract class as an interface: the compiler demands every member, so adding a method to AuthRepository breaks this fake until you add it here too. That is Swift protocol conformance, not Kotlin subclassing. The on* fields are function-typed fields with default closures; a test changes behaviour by assigning a new one. And (_, __) async => testAuth is an async closure: async makes the return a Future, like an async arrow function in TypeScript. The _ and __ names are Dart’s convention for “unused parameter”; two underscores because, before Dart 3.7’s wildcard parameters, _ was an ordinary name and two parameters could not share it.

Every fake in the file follows that shape: default success, scriptable failure, a public record of calls. There are fakes for every repository, plus FakeRaceTracker, FakePermissionPlatform and FakeClock. No mocking library is involved; mocktail is listed in pubspec.yaml but nothing imports it.

Now go to the bottom of the file, testOverrides (app/test/support/fakes.dart:524-575). It returns a list of Overrides. Each line looks like:

authRepositoryProvider.overrideWithValue(auth ?? FakeAuthRepository()),

?? is null-coalescing: use the fake the test passed, or a fresh default one. Read down the list and notice what it covers: config, every repository, the socket, the race tracker, permissions, the in-memory token store and user cache, and locationProvider replaced by IdleLocationNotifier (:459-462), which never touches GPS. Anything not in this list would reach a real plugin, and the test engine has no plugins, so the call would throw a MissingPluginException. That is the reason for the rule in CLAUDE.md that widget tests extend this list rather than adding their own overrides.

testApp (app/test/support/fakes.dart:465-507) wraps that list in a ProviderScope around the real FootraceApp. So a widget test gets the real router, the real theme and the real screens, with only the edges replaced. In iOS terms, it is as if you could launch the whole app in a unit test with every URLSession, CLLocationManager and keychain call behind a protocol you had swapped.

Open test/features/auth/login_screen_test.dart.

Future<void> pumpLogin(WidgetTester tester) async {
  await tester.pumpWidget(testApp(auth: repo));
  await tester.pumpAndSettle();
  expect(find.byType(LoginScreen), findsOneWidget);
}

(app/test/features/auth/login_screen_test.dart:15-19.) No token is passed, so the in-memory token store is empty. The auth notifier settles on signed out, the go_router redirect from Day 4 sends the app to the login route, and the test asserts it got there. pumpWidget builds the first frame; pumpAndSettle keeps running frames until nothing is scheduled, which lets the async session restore and the redirect finish.

Then look at the failure case (:56-67):

repo.onLogin = (_, __) async => throw const ApiException(
      messages: ['Invalid email or password.'],
      statusCode: 404,
    );
await pumpLogin(tester);
await fill(tester, 'runner1@footrace.test', 'wrong-password-1');
await tester.tap(find.text('SIGN IN'));
await tester.pumpAndSettle();
expect(find.text('Invalid email or password.'), findsOneWidget);

find.text searches the widget tree for a Text with that string, the way an XCUITest query finds a label. Note that a tap does nothing visible until you pump: widget tests do not run frames on their own.

FakeApi and a repository test

The fakes above replace whole repositories. To test a repository itself you want the opposite: the real repository class on the real Dio, with only the network faked. Open test/support/fake_api_client.dart (app/test/support/fake_api_client.dart:26-42).

FakeApi({String? token = 'jwt', String baseUrl = 'http://api.test'})
    : dio = buildDio(baseUrl: baseUrl, readToken: () async => token) {
  adapter = DioAdapter(dio: dio, matcher: const UrlRequestMatcher());
  dio.interceptors.add(InterceptorsWrapper(onRequest: (options, handler) {
    calls.add(RecordedCall(
      options.method,
      options.path,
      options.data,
      Map<String, dynamic>.from(options.queryParameters),
      Map<String, dynamic>.from(options.headers),
    ));
    handler.next(options);
  }));
}

The part after : is an initializer list, which runs before the constructor body; it is how Dart sets a final field from a computed value, roughly Kotlin’s primary-constructor property initialisers. buildDio is the app’s own factory, so the token interceptor and the error interceptor are the production ones. DioAdapter from http_mock_adapter replaces the transport underneath, like stubbing URLProtocol. The extra interceptor records every request so the test can assert what was sent.

Now test/data/lobby_repository_test.dart (app/test/data/lobby_repository_test.dart:8-48):

setUp(() {
  api = FakeApi();
  repo = ApiLobbyRepository(api.dio);
});

test('the lobby lists everyone waiting', () async {
  api.getFixture('/api/lobby', 'lobby');
  final entries = await repo.list();
  expect(entries, hasLength(1));
  // ...
  expect(entry.displayName, 'Waiting Wanda');
});

late FakeApi api; at the top of main (:9) declares a non-nullable variable that is assigned before first use; setUp does the assigning. It is Kotlin’s lateinit var.

getFixture replies with test/fixtures/lobby.json, which is a copy of what backend/lib/footrace_web/controllers/api/lobby_controller.ex:12 renders. The error cases use failPost, which replies in the backend’s {"error": "..."} shape, and assert that the interceptor turned it into an ApiException with the right status (:50-59):

await expectLater(
  repo.join('req-1'),
  throwsA(isA<ApiException>().having((e) => e.statusCode, 'statusCode', 404)),
);

expectLater is the async form of expect: it returns a Future you await, needed when the thing under test is itself a Future. isA<T>().having(...) builds a matcher that checks the type and then a property; think of it as a typed pattern match in an assertion.

One test does not follow this pattern. test/data/auth_repository_test.dart builds its own Dio and replies with inline maps (app/test/data/auth_repository_test.dart:15-45). It was written before FakeApi existed, and CLAUDE.md’s rule that repository tests reply with a fixture is not applied there. Copy from the lobby test, not that one.

Fixtures and the contract test

Open test/fixtures/me.json. It is the body of GET /api/me as backend/lib/footrace_web/controllers/api/me_controller.ex:10 renders it: a user and a stats object with totalRaces, totalWins, winRate and a byRaceType list. The fixtures README (test/fixtures/README.md) explains the rules: copy from the controller or docs/channels.md, never from memory, and git fetch the backend first.

Now test/data/fixture_contract_test.dart. Each case is three lines (app/test/data/fixture_contract_test.dart:23-26):

test('GET /api/me', () {
  final source = fixture('me');
  expectRoundTrip(source, MeResponse.fromJson(source).toJson());
});

fixture('me') decodes the file into a Map<String, dynamic>, Dart’s untyped JSON object, like [String: Any] in Swift. MeResponse.fromJson is a hand-written factory constructor, no code generation. toJson() writes it back. expectRoundTrip in test/support/json_contract.dart walks the original and fails if any key is missing from the round trip (app/test/support/json_contract.dart:28-35):

for (final entry in source.entries) {
  final key = entry.key.toString();
  final child = path.isEmpty ? key : '$path.$key';
  if (ignore.contains(child)) continue;
  expect(out.containsKey(key), isTrue,
      reason: 'the model dropped "$child"');
  _compare(entry.value, out[key], child, ignore);
}

So the test does not ask “does the app parse this?” It asks “does the model keep everything the backend sends?” A model that silently reads null for a renamed key and defaults it to 0 would pass a parse test and fail this one, because toJson emits the old key and the fixture’s new key is missing.

Some fields are deliberately not kept, and those are listed by path with a reason. The lobby case (app/test/data/fixture_contract_test.dart:130-139):

expectRoundTrip(source, encoded, ignore: {
  // The app shows `username` under `name` and computes miles itself.
  '[0].username',
  '[0].raceType.miles',
  '[0].raceType.unit',
});

Paths use . for map keys and [i] for list indexes, built at json_contract.dart:31 and :45. Writing the ignore down is the point: dropping a field becomes a decision in a diff, not an accident.

The last case tests the matcher itself: it renames totalWins in the source text, then asserts that expectRoundTrip throws a TestFailure (:187-195). That is what you will reproduce by hand in a minute.

Timers under fake_async

The queue screen polls matchmaking every 2 seconds (app/lib/features/queue/queue_notifier.dart:72, :221). Testing that in real time would be slow and flaky. Open test/features/queue/queue_notifier_test.dart (app/test/features/queue/queue_notifier_test.dart:18-26):

void run(void Function(FakeAsync async, ProviderContainer c) body) {
  fakeAsync((async) {
    final c = ProviderContainer(overrides: testOverrides(matchmaking: repo));
    final sub = c.listen(queueProvider('five_km'), (_, __) {});
    body(async, c);
    sub.close();
    c.dispose();
  });
}

Inside fakeAsync, Timer, Future.delayed and the microtask queue all run on a virtual clock. Nothing advances until the test says so. If you have used RxJava’s TestScheduler or kotlinx-coroutines runTest with advanceTimeBy, this is the same tool.

Two details. There is no widget here: ProviderContainer is Riverpod’s container without Flutter, and testOverrides (not testApp) supplies the same fakes. And c.listen(...) keeps the provider alive: queueProvider is NotifierProvider.autoDispose.family (app/lib/features/queue/queue_notifier.dart:266), and an auto-dispose provider with no listener would be torn down between reads.

The first case (:28-55) scripts onPoll to answer null twice and then a match, and steps the clock:

c.read(queueProvider('five_km').notifier).start();
async.flushMicrotasks();
// ...
async.elapse(const Duration(seconds: 4));
expect(polls, 2);
expect(c.read(queueProvider('five_km')), isA<QueueWaiting>());

async.elapse(const Duration(seconds: 2));
expect(polls, 3);
final matched = c.read(queueProvider('five_km'));
expect(matched, isA<QueueMatched>());

flushMicrotasks runs pending async continuations without moving time, so the join call resolves and the state becomes QueueWaiting. elapse(4 s) fires two ticks of the 2-second timer. The test reads the state as a sealed-class value and checks its subtype with isA<QueueMatched>(), the same sealed-type idea as a Kotlin when over a sealed interface.

Widget tests do not use fakeAsync; testWidgets already runs on a fake clock that tester.pump(duration) advances.

Try it

Run single tests. One file, then one case by name, then a folder:

flutter test test/data/lobby_repository_test.dart
flutter test test/features/auth/login_screen_test.dart --name "wrong credentials"
flutter test test/features/queue

--name takes a regular expression matched against the full test name, including any group prefix.

Break a fixture. In test/fixtures/me.json, rename "winRate" to "win_rate" and save. Then:

flutter test test/data/fixture_contract_test.dart

GET /api/me should fail with a reason like the model dropped "stats.win_rate". Read the chain: UserStats.fromJson looks up json['winRate'], gets null, falls back to 0 (app/lib/data/models/user_stats.dart:24); toJson emits winRate: 0.0; the fixture’s win_rate has no counterpart. Every other test still passes, including the app’s widget tests, because they use testMe from fakes.dart, not the fixture. That is exactly the failure this test exists to catch: in the running app, the profile would show a 0 % win rate and nothing would crash.

Now try a type change instead: put "winRate" back, and change "totalWins": 10 to "totalWins": "10". This time the failure is a TypeError from as num? inside fromJson, not a contract message. Revert the file with git checkout test/fixtures/me.json when done.

Optional: move a timer. Change pollInterval in app/lib/features/queue/queue_notifier.dart:72 to 3 seconds and run flutter test test/features/queue/queue_notifier_test.dart. The expect(polls, 2) after four virtual seconds fails with 1. Revert it.

Check yourself

A widget test renders ProfileTab and the test fails with MissingPluginException. Where do you look first?
Why does FakeApi use the app’s buildDio instead of a plain Dio()?
The backend adds a new field streak to GET /api/me, and you copy the new payload into me.json without touching the model. What does the contract test do, and what are your two options?
In queue_notifier_test.dart, what would happen if you removed async.flushMicrotasks() after start()?
Why does testOverrides exist separately from testApp?

Answers

testOverrides in test/support/fakes.dart. Something the screen reads reaches a platform plugin that is not overridden there. The fix is a fake and a line in that list, not an ad-hoc override in the test.
So the repository test runs through the production interceptors: the bearer header and the error normalisation into ApiException. A plain Dio would skip both, and a test asserting a 404 becomes an ApiException would prove nothing.
It fails with the model dropped "user.streak" (or stats.streak, depending on where it lands). Either add the field to the model so it round-trips, or add its dotted path to that case’s ignore set with a comment saying the app does not use it.
The join call’s Future would not have resolved yet, so the state would still be QueueJoining (set synchronously at the top of start, app/lib/features/queue/queue_notifier.dart:96) and expect(waiting, isA<QueueWaiting>()) would fail. Timers would still fire on elapse, which also drains microtasks, but the assertion before it would already have failed.
Notifier tests have no widget tree. They build a bare ProviderContainer and need the same override list; testApp wraps that list in a ProviderScope for widget tests.

Day 13: Integration test, dev tools, sim GPS, release builds.