What is a 504 error?

A 504 Gateway Timeout means a proxy or load balancer forwarded your request and then waited longer than it was willing to for a response. Unlike a 502, where the upstream answered with something unusable, a 504 means it did not answer in time at all.

The distinction is useful: a 502 suggests something is broken, a 504 suggests something is slow. Slow things sometimes succeed on a second attempt.

Where the time goes in a scraping request

A scraping request has more hops than an ordinary one, and each adds latency:

1. Your client to the proxy.
2. The proxy to its exit node.
3. The exit node to the site's edge.
4. The site's edge to its application servers.
5. If you are rendering, the browser waiting for scripts, fonts and network calls to settle.

A timeout at any of those surfaces as a 504 from whichever gateway ran out of patience first. Rendering is the step that most often pushes a request over the line, because a page that takes four seconds to become interactive in a browser takes at least that long in a headless one.

Reducing the time rather than raising the limit

Raising your own timeout does nothing when the gateway enforcing it is the site's. The productive moves shorten the request:

• Do not render unless the page needs it. Rendering is the single largest contributor. Check whether the content arrives in the initial HTML first.
• Block resources you will not use. Images, fonts, media and stylesheets have to be fetched before a browser considers the page loaded, and none of them matter to a text extraction.
• Wait for a specific element instead of a fixed delay. A selector-based wait finishes as soon as the content is there, while a fixed delay always costs its full duration.
• Lower concurrency. If 504s appear only at peak parallelism, the chain is saturated and every request is being slowed by the others.

A note on partial success

A 504 does not always mean nothing happened. The upstream may have processed the request completely and simply answered too late for the gateway. For read-only scraping that is harmless, but if a request had a side effect, such as submitting a form during a browser session, retrying blindly can repeat it.

Where Zenrows fits

Two parameters do most of the work here. wait_for waits on a CSS selector and returns as soon as the content exists, rather than burning a fixed wait on every page. Block resources drops images, fonts and stylesheets before they are fetched, which cuts page load time substantially on media-heavy sites.