How to do Recursive API Calls based on Offset

There might be a case in which you want to load all the data from the API. For example, let's say you have a select field (customer) and the options come from the API.

And you have a use case in which you need to fetch all the data from the API. And the API does not return all the data in one go. Instead, it returns let's say 5 records at a time and have a nextOffset to indicate that we've more data to be fetched from the API.

Let's see how to handle such cases by recursively calling the API.

Here is an example of what we get from the API based on the offset query string.

GET /customers?offset=0

{
  "nextOffset":1,
  "results":[
    {"id":"customer1"},
    {"id":"customer2"},
    {"id":"customer3"},
    {"id":"customer4"},
    {"id":"customer5"}
  ]
}

GET /customers?offset=1

{
  "results":[
    {"id":"customer6"}
  ]
}

For such cases, we can handle it by recursively calling the API like this


const fetchAllCustomers = async (offset=0, allCustomers = []) => {
    const queryString = new URLSearchParams({
      offset: offset
    })
    const response = await fetch(`https://example.com/customers?${queryString}`)
    const data = response.json();
    const newCustomer = data.results;

    allCustomers = [
      ...allCustomers,
      // making sure we're adding only the unique record.
      // if you don't want to do it, then just do "...newCustomer" alone
      ...newCustomer.filter(
        (customer) =>
          !allCustomers.some((cust) => cust.id === customer.id)
      )
    ];

    if (data?.nextOffset) {
      return await this.fetchAllCustomers(result.nextOffset, allCustomers);
    }

    return allCustomers;
  }
}

Happy recursive records!