Plain HTML
The response will be plain HTML if you do not explicitly specify the output types with your request. This applies to both whether JavaScript rendering is enabled or disabled. Example request:Markdown
ScrapeAutomate allows you to generate markdowns from a webpage very easily. By including themarkdown=true
parameter in your request, the response will be returned in Markdown format. This is useful for extracting content for LLMs or simplifying web data for better readability.
You can use this with both javascript rendering enabled or disabled, but you might need to use javascript rendering when loading dynamic content from a webpage.
Remember, ScrapeAutomate automatically removes unnecessary elements, such as navbars, from the body content.
https://example.com/
Screenshot
To capture a screenshot of the webpage, we provide two query parameters. Screenshots are currently available only in PNG format:screenshot
: Capture a screenshot of the visible area of the webpage.screenshot_full_page
: It captures a screenshot of the entire webpage, including the parts that are not visible.
Screenshot of Visible Area
To capture a screenshot of the visible area, include thescreenshot
query parameter and set it to true
with a request. The response will contain a png
image.
Full Page Screenshot
To capture a screenshot of the entire page, set thescreenshot_full_page
parameter to true
. This will take a full-page screenshot of the webpage and return it as part of the API response.
Webhook
If you prefer to have your responses sent to a webhook instead of receiving them directly, you can utilize our Webhook feature. To use the webhook you need to simply include yourwebhook_url
in the query parameters, and the API will send the response to that webhook. You can receive all types of responses, including HTML, Markdown, and screenshots.
1. Can I use multiple output types?
1. Can I use multiple output types?
Currently, we do not support multiple output types. However, in the near
future, you will have the ability to select multiple output types!
2. Can I use a webhook with API workflows?
2. Can I use a webhook with API workflows?
Yes, you can use a webhook with workflows. Just make sure to set the webhook
when creating or edit a workflow.
3. How can I ensure that the entire webpage has loaded before returning an response?
3. How can I ensure that the entire webpage has loaded before returning an response?
By default, we already wait for the entire webpage to load before returning a
response!