One successful request is not the whole dataset
An API integration reports success when fetching customers or products, yet users find that later records are missing. Check pagination before concluding that the source has no more data. A successful response may confirm only that one page request worked; it does not certify that the receiving system has retrieved every page.
Microsoft Graph documents server-side paging and returning @odata.nextLink when more results remain. Graph illustrates the mechanism here. Other APIs may use cursors, offsets or different field names, so read the actual endpoint contract before defining completion. One service’s convention should not be imposed on every API.
A 250-record example that retrieves only 100
Suppose a controlled test set contains 250 known unique IDs and the service returns pages of 100, 100 and 50 records. Code that reads only the first response obtains 100 correct records but misses 150. Correctness of retrieved records and completeness of the dataset are separate conditions, and both need checking. These numbers are a test example, not a page size supported by every API.
Even a total of 250 records requires duplicate and missing-ID checks: rereading a page can make a count look complete. Compare the expected and received ID sets, then validate business-critical fields such as status and document relationships. Keep failed items visible for investigation rather than deleting them to force totals to match.
Follow the documented next-page mechanism
For Microsoft Graph, use the full @odata.nextLink URL as documented rather than extracting a skiptoken and rebuilding the request. Do not declare completion simply because a page contains fewer records than requested or none at all. The documentation allows a page to contain zero records; the next-page signal still needs checking. Completion must follow the service’s contract.
Automatic URL following must stay within the configured service boundary. Validate the destination before sending credentials and never forward tokens to another domain through an untrusted link. Log run references, status and record counts for diagnosis without recording credentials or entire customer datasets merely for troubleshooting.
Checkpoint after durable processing
A checkpoint should correspond to data that has actually been stored successfully. Advancing it before storage risks skipping a page after a crash. Storing data without advancing the checkpoint can cause the same page to be read again. Design checkpointing together with duplicate-safe processing, such as an update by an appropriate business identifier rather than always inserting a new record.
When a page fails, keep the dataset marked incomplete and follow the service’s retry requirements. Do not skip the page merely to obtain a green job status. Expired tokens or unusable cursors need a documented restart or recovery procedure, with checks that previously stored records are not unintentionally inserted again.
All pages may still represent different moments
Other users may add, edit or delete records while retrieval is running. Reaching the final page does not by itself guarantee a single-time snapshot. Check whether the endpoint guarantees a snapshot or stable ordering. Use time filters, snapshot exports or delta mechanisms only where actually supported and under their documented conditions. A nextLink does not establish every form of consistency.
Before launch, test a partial last page, an empty page with a continuation link, repeated pages and mid-run failure in an authorized environment. Accept the integration using unique-ID counts, outstanding items and documented completion status. For Chawinnix integration support, prepare endpoint documentation, masked examples and acceptance criteria. The further-reading article on migration validation extends reconciliation checks after pagination is fixed.
