Crawl: the most reliable web crawler APISee what’s new
Zenrows
Talk to sales Start free

What are URL parameters?

URL parameters are key and value pairs added to the end of a URL to pass extra information to a page. They start after a ? and are separated by &, as in https://example.com/shoes?color=black&page=2. They are also called query parameters, query strings or URL query strings, and they control things like filters, sort order, pagination, search terms and campaign tracking.

For a crawler, parameters are the single largest source of duplicate URLs. The same page can be reached through dozens of parameter combinations, and telling the ones that change content from the ones that do not is a core crawling skill.

Anatomy of a URL with parameters

https://example.com/shoes?color=black&sort=price_asc&page=2&utm_source=newsletter
                         └──────────────── query string ────────────────────────┘

Each key=value pair is one parameter. Values containing spaces or special characters are percent-encoded, so a search for "running shoes" becomes q=running%20shoes or q=running+shoes.

Types of URL parameters

Type Examples Changes the content? What a crawler should do
Pagination page=2, offset=40, cursor=... Yes, different items Keep, and follow
Filtering color=black, size=10, brand=acme Yes, a subset Keep only the filters you need
Sorting sort=price_asc, order=desc Same items, new order Usually drop, or pick one
Search q=hoodie, query=... Yes Keep when searching is the point
Tracking utm_source, utm_campaign, gclid, fbclid No Drop
Session and state sessionid=..., sid=... No Drop

Why parameters cause crawling problems

  • Duplicate content. Tracking and session parameters create new URLs for the same page. Without deduplication, each one is fetched again.
  • Combinatorial explosion. Five filters with ten values each can produce thousands of URLs for one category. This is the faceted navigation problem.
  • Order sensitivity. ?a=1&b=2 and ?b=2&a=1 are the same request to almost every server but different strings to a naive crawler.
  • Traps. Parameters that increment forever, such as calendars, become a spider trap.

Handling URL parameters in a crawler

  1. Parse instead of string-matching. Use a URL library to split the query string into pairs.
  2. Drop known tracking keys: the utm_* family, gclid, fbclid, msclkid, ref and session IDs.
  3. Sort what remains so parameter order cannot create duplicates.
  4. Keep an allowlist per site of parameters that change content, such as page and the filters you care about. Allowlists age better than blocklists, because new tracking parameters appear all the time.
  5. Prefer the page's canonical URL as the final identity where it is trustworthy.

This is the parameter part of URL normalization.

What are UTM parameters?

UTM parameters are the tracking family: utm_source, utm_medium, utm_campaign, utm_term and utm_content. Marketing tools read them to attribute a visit to a campaign. They never change what the page shows, which makes them the safest parameters for a crawler to strip.

How Zenrows Crawl handles this

Zenrows Crawl drops duplicate links as it crawls, so each URL appears in its results once. Include and exclude patterns let you keep only the URLs you want, such as those containing /product/, and leave parameter variants such as sort or filter URLs out with exclude patterns. See the Crawl endpoint reference.

Frequently asked questions

What is the difference between URL parameters and query parameters?

Nothing in everyday use. Both mean the key and value pairs after the ?. Some frameworks also use "path parameters" for values inside the path, such as /users/42, which are a different thing.

What is a query string?

The whole part of a URL after the ? and before any #. It contains all the query parameters.

Do URL parameters affect SEO?

They can, by creating many URLs for the same content. Canonical tags, consistent internal linking and keeping tracking parameters out of internal links prevent most problems.

How do I remove UTM parameters from a URL?

Parse the URL, delete every key starting with utm_, and rebuild it. In Python, urllib.parse with parse_qsl and urlencode does this in a few lines.

Go deeper on the blog

In the docs

Last updated: Oct 9, 2026

Get reliable web data in minutes.

Free plan, 5,000 credits every month, no credit card required.