Skip to main content
Documentation

All About Pagination in NewsData.io News API

By August 29, 2023November 21st, 2025No Comments

Pagination helps reduce overall loading speed and ensures better crawling and indexing. In this article, we’ll explore the concept of pagination in NewsData.io.

How Pagination works

To paginate in Newsdata.io News API, follow the steps mentioned below:

https://newsdata.io/api/1/latest?apikey=YOUR_API_KEY&q=YOUR_QUERY

This request will retrieve 10 articles/page for free and 50 articles/page for paid users.

To go to the next page, you need to use the nextPage string value given at the bottom of the response. Use that nextPage string value in your query using this parameter: ‘page=Nextpagestring’.

https://newsdata.io/api/1/latest?apikey=YOUR_API_KEY&q=YOUR-QUERY&page=XXXPPPXXXXXXXXXX

This parameter will take you to the next page with a new set of articles.

For instance, my query is Football. The request parameter would be as follows:

https://newsdata.io/api/1/latest?apikey=YOUR_API_KEY&q=football

After entering the query, you will get the data related to your keyword.

Now, go to the bottom of the page, and you will find the “nextPage” with the specified number.

Copy that number and enter it in the query with the page parameter. This request will retrieve the next page.

You need to repeat this process manually to move to the next pages.

Methods to Paginate in Python

There are three methods to paginate in Python:

1. Manual Pagination

Manual pagination gives you full control over how you move through result pages. Each API response contains a nextPage token, which you pass back into the next request.

# -------------------------------------------
# Manual Pagination Example (Recommended when 
# you want complete control over page traversal)
# -------------------------------------------

from newsdataapi import NewsDataApiClient

# Initialize client with API key
api = NewsDataApiClient(apikey='YOUR_API_KEY')

next_page = None

while True:
    # Fetch the latest news with manual pagination
    response = api.latest_api(q='pizza', page=next_page)

    # Print or store the response
    print(response)

    # Get the next page token from the API response
    next_page = response.get('nextPage')

    # Stop when there are no more pages
    if next_page is None:
        break

When to Use:

  • When you want custom logic for stopping or processing pages.
  • When memory is a concern, and you prefer processing one page at a time.
  • When you want transparency and full control over the workflow.

2. Scroll

Scroll pagination fetches all matching results in a single response. It works like a bulk dump of data.

Use the max_result parameter to specify how many articles you want to retrieve in the scroll request.

# ---------------------------------------------------------
# Scroll Pagination Example
# This returns all results at once (be careful with memory!)
# ---------------------------------------------------------

from newsdataapi import NewsDataApiClient

# Initialize client with API key
api = NewsDataApiClient(apikey='YOUR_API_KEY')

# Fetch up to 1000 results in a single scroll request
response = api.latest_api(q='pizza', scroll=True, max_result=1000)

# Print or process all results at once
print(response)

When to Use:

  • When you want everything at once.
  • When the dataset is small or moderate.
  • When you need a fast, complete extraction.

3. Paginate

Paginate allows you to fetch a large amount of data; it provides you with results, maximum pages when you want large datasets.

This is the recommended method when working with large datasets. The API client automatically handles page traversal and returns a generator, allowing pages to be processed one by one.
Use max_pages to set the maximum number of pages the generator should retrieve.

# -------------------------------------------------------------
# Pagination using the "paginate" parameter
# This method is recommended for fetching large amounts of data.
# It returns a GENERATOR, allowing you to iterate through pages
# one by one without loading everything into memory at once.
# -------------------------------------------------------------

from newsdataapi import NewsDataApiClient

# Initialize the NewsDataApiClient with your API key
api = NewsDataApiClient(apikey='YOUR_API_KEY')

# Enable pagination and limit the number of pages to fetch
# Note: Setting paginate=True returns a generator object.
response = api.latest_api(q='pizza', paginate=True, max_pages=2)

# Iterate through each page of results from the generator
for page_data in response:
    print(page_data)   # Or process/store each page as needed

When to Use:

  • When you want large datasets.
  • When you want easy iteration without writing custom logic.
  • When memory efficiency matters.
  • Best for production workloads.

Leave a Reply

Close Menu