Quick summary of what’s happening right now:
We encountered a 504 Timeout error when calling Outscraper, which might suggests that Outscraper is down. This issue affects new businesses.
hello @noerrorapi,
The HTTP 504 Gateway Timeout indicates that the server did not get a response within the dedicated time. In other words, Outscraper did not export the data within the time of the request. It can happen when you are using multiple queries within one request, extracting a large amount of data at once (such as reviews and comments), or in cases where Outscraper might be overloaded with requests.
To make your system bulletproof from this, you can consider the following actions:
- Use retries. Expect that some number of scraping requests might return an error or timeout. Usually, trying one more time solves the issue.
- Use async requests. A good practice is to send async requests and start checking the results after the estimated execution time. Check out this Python implementation as an example.
- Use a webhook to receive results once it’s ready.