Testing

The app has three kinds of automated check, and they run in very different places:

Kind	Where	Runs on	Who runs it
Unit and widget tests	test/	the host, in a headless Flutter engine	make check, CI on every push
Screenshot renders	test/screenshots/	the host, same engine	by hand, with FR_SCREENSHOTS set
Integration test	integration_test/	an Android emulator, against a real backend	make integration, CI on demand

Everything in test/ runs under flutter test, which starts a test-only build of the Flutter engine (flutter_tester) on your Mac. There is no simulator, no emulator and no platform channels. A widget test renders the real widget tree into an offscreen surface and lets you tap and type into it, which is closer to an XCUITest than to a plain XCTest, except it runs in milliseconds and in-process. Anything that would reach the platform (GPS, keychain, the foreground service) has to be replaced, and most of this page is about how the repo does that.

Layout

test/ mirrors lib/: lib/data/repositories/lobby_repository.dart is tested by test/data/lobby_repository_test.dart, lib/features/queue/queue_notifier.dart by test/features/queue/queue_notifier_test.dart. The mirror is loose in one place: repository tests sit flat in test/data/ rather than under test/data/repositories/. There are 62 *_test.dart files at d1b602a.

test/
  support/          shared harness: fakes, FakeApi, fake socket, fixture loading
  fixtures/         real backend JSON payloads + README of backend quirks
  core/ data/ design/ features/ tracking/   mirrors lib/
  screenshots/      PNG renderer, skipped unless FR_SCREENSHOTS is set
integration_test/
  full_race_test.dart
tool/check_coverage.dart
scripts/integration.sh

flutter test with no arguments runs every file under test/ whose name ends in _test.dart. It does not look in integration_test/, which is why make check never touches the emulator.

flutter_test in five minutes

The package is flutter_test, which wraps Dart’s package:test. The shapes you need:

flutter_test	Closest thing you know
test('name', () { ... })	XCTest method, ExUnit test
group('name', () { ... })	describe block
setUp(() => ...)	setUp() / ExUnit setup
expect(actual, matcher)	XCTAssertEqual, but with composable matchers
testWidgets('name', (tester) async { ... })	a UI test with a driver handle
find.text('SIGN IN'), find.byType(LoginScreen)	XCUITest queries, Espresso onView(withText(...))
tester.pump() / tester.pumpAndSettle()	run one frame / run frames until nothing is scheduled

Matchers compose: throwsA(isA<ApiException>().having((e) => e.statusCode, 'status', 404)) reads as “throws an ApiException whose statusCode is 404”. You will see that shape in every repository test.

pump is the part with no native equivalent. A widget test does not run frames on its own. After a tap, nothing has rebuilt until you pump. pumpAndSettle keeps pumping until no frames are scheduled, which means it never returns on a screen with a looping animation. That is why useReducedMotion exists (below), and why the integration test writes its own settle.

The shared harness: test/support/

CLAUDE.md says widget tests extend this harness rather than adding ad-hoc overrides. The code bears that out: the feature tests import fakes.dart and nothing else from the platform side.

fakes.dart: one Fake per repository, and testApp

Every repository has a hand-written fake that implements the real interface. They share one pattern: public mutable handler fields, with a default that succeeds, and public lists that record what was called. FakeAuthRepository is the template (app/test/support/fakes.dart:69-105):

class FakeAuthRepository implements AuthRepository {
  final calls = <String>[];

  Future<AuthResponse> Function(String email, String password) onLogin =
      (_, __) async => testAuth;
  // ...
  @override
  Future<AuthResponse> login({required String email, required String password}) {
    calls.add('login:$email');
    return onLogin(email, password);
  }

A test that wants a failure assigns a new closure: repo.onLogin = (_, __) async => throw const ApiException(...). This is the same idea as a Swift protocol with a hand-rolled spy, and it is deliberately not a mocking library. Dart’s implements makes the compiler check that the fake covers every method of the interface, so a new repository method breaks the fake at compile time rather than at runtime.

testApp(...) (app/test/support/fakes.dart:465-507) returns the real FootraceApp wrapped in a ProviderScope whose overrides come from testOverrides(...) (app/test/support/fakes.dart:524-575). Each parameter is optional; anything you do not pass gets a default fake. The override list replaces:

• appConfigProvider with a fixed AppConfig (app/test/support/fakes.dart:545-547), so devTools is off unless the test asks.
• every repository provider with its fake.
• tokenStoreProvider and userCacheProvider with in-memory stores, seeded from the token and cachedUser parameters (app/test/support/fakes.dart:572-573). Passing token: 'jwt' is how a test starts signed in.
• socketTransportProvider with a FakeSocket, raceTrackerProvider with a FakeRaceTracker, permissionPlatformProvider with a FakePermissionPlatform.
• locationProvider with IdleLocationNotifier (app/test/support/fakes.dart:459-462), which never asks for permission or starts GPS.

If you have not read the Riverpod primer, the short version: a provider is a named slot the app reads through ref, and an override swaps what fills that slot for everything under one ProviderScope. It is dependency injection at the container, like registering test doubles in a Swinject or Koin container, except the app code never has to be written against the container.

socketTransportProvider is overridden twice, at app/test/support/fakes.dart:554 and again at :558. In Riverpod 2.6.1 (the locked version) the container stores overrides in a map keyed by provider, so the second one wins and the socket parameter is honoured. The first line is dead and harmless.

testOverrides exists for tests that have no widget tree: notifier tests build a bare ProviderContainer(overrides: testOverrides(matchmaking: repo)) and read providers from it directly (app/test/features/queue/queue_notifier_test.dart:20).

The rest of fakes.dart is small helpers: FakeClock (app/test/support/fakes.dart:409-414) overrides clockProvider with a hand-advanced now; usePhoneViewport sets a 1080 x 2400 surface so lazy lists build the rows a phone would show (:510-514); useReducedMotion turns on the platform’s “disable animations” flag so screens with a pulsing live dot can still pumpAndSettle (:518-522); neverAnswers holds the app in the auth unknown state (:453).

The rule in CLAUDE.md: a new repository gets a Fake<Name>Repository here and a parameter on both helpers.

fake_api_client.dart: FakeApi

Repository tests want something different: the real repository, the real Dio and the real interceptors, with only the socket at the bottom replaced. FakeApi does that (app/test/support/fake_api_client.dart:26-42):

FakeApi({String? token = 'jwt', String baseUrl = 'http://api.test'})
    : dio = buildDio(baseUrl: baseUrl, readToken: () async => token) {
  adapter = DioAdapter(dio: dio, matcher: const UrlRequestMatcher());
  dio.interceptors.add(InterceptorsWrapper(onRequest: (options, handler) {
    calls.add(RecordedCall(options.method, options.path, /* ... */));
    handler.next(options);
  }));
}

buildDio is the app’s own factory from lib/data/api/api_client.dart, so the bearer-token and error-normalising interceptors run exactly as in production (see Data layer). DioAdapter comes from http_mock_adapter and replaces Dio’s HTTP adapter, the layer that would open a socket. It is the Dio equivalent of a URLProtocol stub on iOS or OkHttp’s MockWebServer, minus the server.

Stubs match on method and URL only. What the app sent is asserted afterwards through api.lastCall and api.requestLog (app/test/support/fake_api_client.dart:48-50), which keeps the assertion in the test body instead of buried in a matcher. Reply helpers: get/post/patch/delete with any body, getFixture/postFixture with a file from test/fixtures/ (:56-63), and failGet/failPost that reply in the backend’s {"error": "..."} shape (:74-78).

Six repository tests use it. test/data/auth_repository_test.dart predates it: it builds its own Dio and DioAdapter and replies with inline maps (app/test/data/auth_repository_test.dart:15-45). That contradicts the CLAUDE.md convention that repository tests reply with a fixture, and is worth knowing if you copy a test as a starting point.

fake_socket.dart

FakeSocket and FakeChannel implement the app’s own SocketTransport and ChannelTransport interfaces (app/test/support/fake_socket.dart:7-89). A test scripts the join reply per topic (socket.joinReplies['race:race-1'] = ...), pushes server events with channel.emit('raceUpdate', payload), and answers client pushes by assigning channel.onPush. An unscripted topic throws ChannelException('not_found'), which is what the backend’s RaceChannel and WaitingChannel reply for an unknown race or race type (app/test/support/fake_socket.dart:84-85). The Realtime page covers the interfaces being faked.

fixtures.dart and json_contract.dart

fixture(name) reads and decodes test/fixtures/<name>.json (app/test/support/fixtures.dart:10-11). It uses a relative path, which works because flutter test always runs from the package root. fixtureSource returns the raw text so a test can mutate it before decoding (:14-20), and fixtureList pulls a list out from under a key (:23-29).

expectRoundTrip(source, encoded, ignore: {...}) is the matcher behind the contract test (app/test/support/json_contract.dart:14-19). It walks source and fails with “the model dropped <path>” for any key missing from encoded (:33-34). Extra keys in encoded are fine. Two tolerances: timestamps compare as instants, so ...Z and +00:00 are equal (:50-59), and 1200 equals 1200.0 (:60-63).

Fixtures and the contract test

test/fixtures/ holds one real backend payload per response the app consumes: me.json for GET /api/me (backend/lib/footrace_web/controllers/api/me_controller.ex:10), lobby.json for GET /api/lobby (backend/lib/footrace_web/controllers/api/lobby_controller.ex:12), the channel join reply and pushes, and so on. test/fixtures/README.md says where to copy each from and lists the backend’s real inconsistencies (snake_case leaks, started vs racing, three timestamp spellings) so they are not “fixed” by accident.

test/data/fixture_contract_test.dart has one case per fixture. Each decodes the fixture into its model with fromJson, encodes it back with toJson, and calls expectRoundTrip (app/test/data/fixture_contract_test.dart:23-26):

test('GET /api/me', () {
  final source = fixture('me');
  expectRoundTrip(source, MeResponse.fromJson(source).toJson());
});

When the backend renames a field and someone refreshes the fixture, the model no longer reads the new key, toJson emits the old one, and this test names the path. Fields the app deliberately does not keep go in ignore as dotted paths, with a comment saying why: the lobby case ignores [0].username, [0].raceType.miles and [0].raceType.unit (app/test/data/fixture_contract_test.dart:130-139), and the REST race cases ignore each competitor’s speed and status (:200-207). The last case breaks a fixture on purpose and asserts the matcher notices (:187-195), a test of the test.

This is not a schema check against the backend. It checks that the app’s models are lossless over the payloads in the folder, and is only as current as those files. CLAUDE.md and the fixtures README both insist on git fetch in ../footrace before copying, because an earlier set was copied from a stale checkout.

Widget tests

test/features/auth/login_screen_test.dart is a clean example. It boots the whole app on fakes and lets the router decide where to land (app/test/features/auth/login_screen_test.dart:15-19):

Future<void> pumpLogin(WidgetTester tester) async {
  await tester.pumpWidget(testApp(auth: repo));
  await tester.pumpAndSettle();
  expect(find.byType(LoginScreen), findsOneWidget);
}

No token is passed, so the in-memory token store is empty, the auth notifier resolves to signed out, and the go_router redirect sends the app to /login. The test is exercising the auth gate as a side effect. The cases then type, tap and assert on both the screen and the fake: an empty form shows validation text and repo.calls stays empty (:27-35); a good login lands on HomeShell (:46-54); a thrown ApiException shows its message (:56-67).

Design-system component tests do not need the app. They use pumpFr from test/design/components/pump.dart, which wraps one widget in a MaterialApp with the app theme on a 360 px canvas at a chosen text scale (app/test/design/components/pump.dart:7-31).

Time: fake_async

Anything driven by Timer is tested under fakeAsync, from the fake_async package. Inside the callback, Timer, Future.delayed and microtasks run on a virtual clock that only moves when the test calls async.elapse(...). RxJava’s TestScheduler and kotlinx’s runTest virtual time are the same idea.

The queue notifier polls matchmaking every 2 s with Timer.periodic (app/lib/features/queue/queue_notifier.dart:72, :221). Its test wraps a whole ProviderContainer in fakeAsync (app/test/features/queue/queue_notifier_test.dart:18-26) and then steps the clock: elapse 4 s, expect two polls; elapse 2 more, expect the match (:41-49). test/tracking/simulated_location_source_test.dart does the same for the GPS simulator, also passing the fake clock in as the simulator’s now (app/test/tracking/simulated_location_source_test.dart:25-48).

Note the split: fakeAsync is for test, not testWidgets. Widget tests already run on a fake clock that tester.pump(duration) advances.

mocktail

mocktail is in dev_dependencies (app/pubspec.yaml:41), but nothing under test/ or integration_test/ imports it at d1b602a. Every double is hand-written. The only “mock” in use is MockPlatformInterfaceMixin from plugin_platform_interface, which lets race_screen_test.dart and screenshots/race_test.dart install a fake wakelock_plus platform implementation (app/test/features/race/race_screen_test.dart:10-18).

Screenshots renderer

test/screenshots/ uses the widget-test engine to render screens to PNG. It asserts nothing. Every case is declared with skip: !enabled, where enabled is true only when FR_SCREENSHOTS names an output directory (app/test/screenshots/harness.dart:18-20, app/test/screenshots/screens_test.dart:134-142). So make check and CI see them as skipped.

FR_SCREENSHOTS=docs/screenshots/elb-1683 flutter test test/screenshots

The harness loads the bundled Plus Jakarta Sans and Space Grotesk files plus the Material icon font out of the Flutter SDK cache, because the test engine otherwise draws text in a placeholder font and icons as boxes (app/test/screenshots/harness.dart:24-54). It wraps testApp(...) in a RepaintBoundary and captures that with toImage at pixel ratio 2 (:56-68). The result is 1080 x 2400 PNGs that look like the phone, but are not emulator captures: no status bar, no system chrome, no real GPS. PRs that change a screen commit them under docs/screenshots/<story>/.

Coverage gate

make coverage runs flutter test --coverage, which writes coverage/lcov.info, then dart run tool/check_coverage.dart --min 70. The script is about 80 lines of plain Dart that parses lcov itself:

• excludes lib/dev/ and generated .g.dart / .freezed.dart files (app/tool/check_coverage.dart:10-12);
• prints the ten least covered files;
• exits 1 if total line coverage is below --min (:75-78).

The script’s own default is 60 (app/tool/check_coverage.dart:15), but both callers pass 70: COVERAGE_MIN ?= 70 in the Makefile (app/Makefile:23) and --min 70 in CI (app/.github/workflows/ci.yml:44-45). The CI comment calls it “a threshold, not a target”: it catches a branch that guts the suite.

Integration test

integration_test/full_race_test.dart runs the real app, with a real ProviderScope and no overrides (app/integration_test/full_race_test.dart:56), on an Android emulator against a real Phoenix backend. It registers a fresh account, races the steady bot over 100 m with simulated GPS, wins, and checks History and Profile; a second test leaves a race mid-way. It uses the same WidgetTester API as widget tests, but every wait is a poll with a deadline because a real backend is not deterministic (:259-272).

scripts/integration.sh (run by make integration) does the setup the test cannot: reset and seed the dev database, start mix phx.server on port 4001, boot the Pixel_9a AVD, install the APK with every runtime permission pre-granted, and run flutter test on the file with FAKE_GPS=true and DEV_TOOLS=true. Day 13 walks through both files.

CI

.github/workflows/ci.yml has four jobs:

Job	Trigger	What it runs
check	push to main, every PR	dart format check, flutter analyze --fatal-infos, flutter test --coverage, coverage gate at 70 %
build-android	same	flutter build apk --debug on JDK 17
storybook	same	flutter analyze and flutter build web in widgetbook/
integration	workflow_dispatch only	Postgres service, backend checkout, emulator, scripts/integration.sh --no-reset

Each job reads the Flutter version from .tool-versions with awk (app/.github/workflows/ci.yml:24-26), so asdf locally and CI never disagree. The check job also runs flutter pub get inside widgetbook/ before analyzing, because flutter analyze from the root picks up that package too (:35-37). There is no iOS job; the comment at :72-73 says a macOS runner costs ten times the Linux minutes. The build and release page covers the jobs from the build side.

Running things

make check                                          # what CI's check job runs, minus coverage
make coverage                                       # flutter test --coverage + the 70 % gate
flutter test test/data/fixture_contract_test.dart   # one file
flutter test test/features/auth --name "wrong credentials"   # by test name, within a folder
make integration                                    # emulator + backend; slow

Day 12 is the guided version of this page.