Pagination Library: internal design, caching, errors and testing

Behind the eight public symbols from the previous lesson is a small state machine, an in-flight registry, and a cancellation story. All three exist to survive a user flicking a list quickly.

This lesson designs those internals, then takes the question that decides whether real apps can adopt the library at all — does it own storage? — and finishes with the part that decides whether a consumer enjoys using it or resents it: errors, testing, and the follow-ups.

The state machine, per direction

Load state is not one value. A list can be appending at the bottom while refreshing at the top, and collapsing those into a single isLoading boolean is the first mistake.

Three independent directions, each with the same states:

                 ┌──────────────────────────────────┐   IDLE ───load──► LOADING ──success──► IDLE        │                     │  │                           │                     │  └──no more items──► END     │  (terminal for that direction)                     │                              │                     └──failure──► ERROR ──retry()──┘

refresh, prepend, and append each hold one of Idle, Loading, Error(e), or EndOfPagination. The consumer renders a footer spinner from append, a header spinner from prepend, and a full-screen state from refresh — three different user interface treatments from three separate values, which is impossible with one boolean.

Deduplication: the fast-scroll problem

The naive implementation asks for the next page whenever the visible range gets within the prefetch distance of the end. During a fast flick that condition is true on many consecutive frames, so the library fires five overlapping loads for the same key. Five requests, five responses, and pages appended in whatever order they return.

The fix is one rule: at most one load in flight per direction.

fun maybeAppend(key):    if appendState is Loading or EndOfPagination: return    if key == lastAppendedKey: return          // already have it    appendState = Loading    launch { handle(source.load(key, config.pageSize)) }

Two guards, and the fast-scroll storm is gone. Keeping a map of in-flight keys is a reasonable generalisation, but for a strictly ordered list the single-slot version is simpler and easier to reason about — and simplicity is a library requirement.

Cancellation

Three cases where work must stop:

Refresh supersedes everything. A refresh invalidates the current page set, so any in-flight append or prepend is cancelled — its result would belong to a list that no longer exists.

The consumer goes away. When the screen closes, all loads must be cancelled. That is why the consumer supplies the scope or lifecycle rather than the library inventing one: the library cannot know when the consumer's screen dies, and guessing produces leaks.

A load exceeds its usefulness. A page requested twenty seconds ago for a position the user has long scrolled past should be abandoned. This is optional and worth mentioning as a refinement rather than building by default.

Dropping pages

maxCachedPages bounds memory: with it set, loading a new page at one end drops one from the other and clears the key so it can be re-fetched on the way back. Without it, a user who scrolls through 5,000 items holds all 5,000 in memory.

The trade-off is real and worth stating: dropping pages means scrolling backwards can show a loading state for content the user has already seen. Default to unbounded and make it opt-in, because the surprising behaviour should be the one you asked for.

Cache and invalidation

Dropped pages have to come back from somewhere, which raises the question that decides whether real apps can adopt your library: does it own storage?

Why owning storage is the wrong default

A library that manages its own database seems helpful and excludes most serious consumers. Apps that need pagination usually already have a local database — it is their offline source of truth (Offline-first architecture) — and they need paged data and stored data to be the same data. If the library keeps a second copy, a reaction written to the app's database does not appear in the paged list, and now two caches disagree.

Compose instead: pages over the local store

The pattern that works, and it is worth drawing in the interview:

1. The consumer's PageSource reads pages from their local database.
2. A separate hook — call it a RemoteFetcher — is called by the library when it reaches the end of what the database holds. It fetches from the network and writes into the consumer's database.
3. The database change invalidates the current PageSource, and the library re-reads.

interface RemoteFetcher<Key> {    // called when the local source runs out at either end    fetch(direction: Append | Prepend | Refresh, lastKey: Key?): Result}

The library owns paging. The consumer owns storage. The list is offline-capable because its source is local, and the network is a background top-up — the offline-first architecture from Section 3, expressed as a library boundary.

Keep this optional. A consumer with no database supplies only a network-backed PageSource and never touches RemoteFetcher. Both cases use the same eight public symbols.

Refresh that does not destroy the user's position

The naive refresh clears the page store and reloads page one, so a user forty items down is thrown to the top. The news feed's Persistence and cache policy lesson has the same problem in an app; here it must be solved generically, because the library does not know what the items mean.

The mechanism is an anchor. Before refreshing, the library asks the consumer which item is currently visible, and after reloading it restores the position of that item's key.

config.anchorProvider = { visibleItemKey() }     // consumer supplies

If the anchored item has vanished from the refreshed data, fall back to the nearest surviving position and say so through the load state. The library cannot decide what "nearest" means for arbitrary data, so it does the mechanical part and gives the consumer a hook for the rest.

Invalidating a single item

After a mutation, an app wants one row updated, not the whole list reloaded. Two options:

The second is another argument for composing rather than owning. When the PageSource is an observable database query, a single-row update propagates without the library doing anything — which is the best kind of feature, the one you do not have to build.

Offline: with the composed design, offline is the consumer's local pages plus an append that fails. The library reports append = Error; the consumer renders "offline, showing saved items". The library never decides what offline looks like, and that is correct — it does not know.

Error handling, testing, and follow-ups

The last third of the design decides whether a consumer enjoys using this library or resents it.

Surface errors without dictating the user interface

The temptation is for the library to handle errors: show a snackbar, retry three times, insert an error row. Every one of those is a product decision the library is not entitled to make. One app wants a full-screen error, another a footer, a third a silent retry.

So the library does three things and no more: it puts the error in the load state for the direction it happened in, it keeps the pages already loaded, and it exposes retry(). Everything visual is the consumer's.

Partial failure is the normal case and must work. Pages one to four loaded, page five failed. The correct behaviour is that the list still shows the eighty items it has, append is Error, and retry() re-requests only page five. A library that discards loaded pages on a failed append is unusable on a mobile network, where a failed append is routine.

Retry that is the consumer's decision

Provide retry() and do not retry automatically by default. Automatic retry inside a library is invisible, hard to test, and compounds badly with the consumer's own network-layer retry from The network layer — three attempts in the library times three in the HTTP client is nine requests for one user action. If you offer automatic retry, make it opt-in, bounded, and documented as interacting with the transport's own policy.

Testing: yours and theirs

Your tests need a deterministic clock and scheduler so "loading" and "loaded" are observable states rather than a race, plus a fake PageSource that returns scripted pages, failures, and delays.

The consumer's tests matter more, and the design must serve them:

• The PageSource interface is the seam. A consumer implements a fake in five lines and tests their screen with no network.
• Streams must be assertable — a consumer must be able to collect emissions in order and assert on them.
• The scheduler is injectable. If the library hard-codes its own dispatcher, every consumer's test either sleeps or flakes.
• Ship a test artefact. A small companion module with a fake source and a state-assertion helper is a large adoption win for a modest cost.

The follow-ups

Prepend for bidirectional lists. A chat list pages upward into history and downward into new messages. Support it by giving LoadResult a prevKey from the start; adding it later changes a public type. Note the hard part honestly: prepending changes the offsets of everything below, so the consumer's list must be able to prepend without scrolling.

Placeholders when a total count is known. If the source knows there are 5,000 items, the library can present a list of 5,000 with nulls for unloaded positions, so the scrollbar is accurate and the list does not grow under the user. It costs nulls in the consumer's item type and requires stable positions — which cursor pagination cannot promise. Offer it only for sources that can count, and say why it does not work with cursors.

Threading. Loads run off the main thread; stream emissions arrive where the consumer asked for them. Never dictate that.