What are URL parameters?
URL parameters are key and value pairs added to the end of a URL to pass extra information to a page. They start after a ? and are separated by &, as in https://example.com/shoes?color=black&page=2. They are also called query parameters, query strings or URL query strings, and they control things like filters, sort order, pagination, search terms and campaign tracking.
For a crawler, parameters are the single largest source of duplicate URLs. The same page can be reached through dozens of parameter combinations, and telling the ones that change content from the ones that do not is a core crawling skill.
Anatomy of a URL with parameters
https://example.com/shoes?color=black&sort=price_asc&page=2&utm_source=newsletter
└──────────────── query string ────────────────────────┘
Each key=value pair is one parameter. Values containing spaces or special characters are percent-encoded, so a search for "running shoes" becomes q=running%20shoes or q=running+shoes.
Types of URL parameters
| Type | Examples | Changes the content? | What a crawler should do |
|---|---|---|---|
| Pagination | page=2, offset=40, cursor=... |
Yes, different items | Keep, and follow |
| Filtering | color=black, size=10, brand=acme |
Yes, a subset | Keep only the filters you need |
| Sorting | sort=price_asc, order=desc |
Same items, new order | Usually drop, or pick one |
| Search | q=hoodie, query=... |
Yes | Keep when searching is the point |
| Tracking | utm_source, utm_campaign, gclid, fbclid |
No | Drop |
| Session and state | sessionid=..., sid=... |
No | Drop |
Why parameters cause crawling problems
- Duplicate content. Tracking and session parameters create new URLs for the same page. Without deduplication, each one is fetched again.
- Combinatorial explosion. Five filters with ten values each can produce thousands of URLs for one category. This is the faceted navigation problem.
- Order sensitivity.
?a=1&b=2and?b=2&a=1are the same request to almost every server but different strings to a naive crawler. - Traps. Parameters that increment forever, such as calendars, become a spider trap.
Handling URL parameters in a crawler
- Parse instead of string-matching. Use a URL library to split the query string into pairs.
- Drop known tracking keys: the
utm_*family,gclid,fbclid,msclkid,refand session IDs. - Sort what remains so parameter order cannot create duplicates.
- Keep an allowlist per site of parameters that change content, such as
pageand the filters you care about. Allowlists age better than blocklists, because new tracking parameters appear all the time. - Prefer the page's canonical URL as the final identity where it is trustworthy.
This is the parameter part of URL normalization.
What are UTM parameters?
UTM parameters are the tracking family: utm_source, utm_medium, utm_campaign, utm_term and utm_content. Marketing tools read them to attribute a visit to a campaign. They never change what the page shows, which makes them the safest parameters for a crawler to strip.
How Zenrows Crawl handles this
Zenrows Crawl drops duplicate links as it crawls, so each URL appears in its results once. Include and exclude patterns let you keep only the URLs you want, such as those containing /product/, and leave parameter variants such as sort or filter URLs out with exclude patterns. See the Crawl endpoint reference.
Frequently asked questions
What is the difference between URL parameters and query parameters?
Nothing in everyday use. Both mean the key and value pairs after the ?. Some frameworks also use "path parameters" for values inside the path, such as /users/42, which are a different thing.
What is a query string?
The whole part of a URL after the ? and before any #. It contains all the query parameters.
Do URL parameters affect SEO?
They can, by creating many URLs for the same content. Canonical tags, consistent internal linking and keeping tracking parameters out of internal links prevent most problems.
How do I remove UTM parameters from a URL?
Parse the URL, delete every key starting with utm_, and rebuild it. In Python, urllib.parse with parse_qsl and urlencode does this in a few lines.
Go deeper on the blog
In the docs
Last updated: Oct 9, 2026