Generate rotating user-agents and proxy IPs for scraping APIs

by Adam ABDELMOUMNI · 93 views · created 2026-02-23 · Document Extraction

What this workflow does

This workflow has 12 nodes and runs manually, when you click “Execute workflow”. It uses only n8n's built-in nodes. It also calls useragentstring.com and cloudflare.com over HTTP.

How it runs

Trigger: Manually, when you click “Execute workflow”.

1. user-agents (HTTP Request): calls useragentstring.com
2. SET your proxy connection details here (Edit Fields (Set)): sets 3 fields: proxy_username, proxy_password, proxy_port
3. Extract user-agent values (HTML): parses or builds HTML
4. Merge: merges the data from its input branches
5. Split Out: splits a list into separate items
6. Check used IP/user-agent with cloudflare (HTTP Request): calls cloudflare.com (POST)
7. Targeted API (HTTP Request): calls an HTTP API
8. clean the return to line in user-agent (Edit Fields (Set)): sets 1 field: clean_user-agent
9. IP address and user-agent used (Edit Fields (Set)): sets 2 fields: Ip_address, user-agent
10. random sort (Sort): sorts the items

…plus 1 more step. See the full node list below.

HTTP calls to: useragentstring.com, cloudflare.com

Scraping pages by hand? Firecrawl is a simpler option (10% off for new users).

Before you upgrade to n8n 3.0

Nodes used (8)

• HTTP Requestn8n-nodes-base.httpRequest
• Edit Fields (Set)n8n-nodes-base.set
• Sortn8n-nodes-base.sort
• HTMLn8n-nodes-base.html
• Split Outn8n-nodes-base.splitOut
• Manual Triggern8n-nodes-base.manualTrigger
• Mergen8n-nodes-base.merge
• Limitn8n-nodes-base.limit

From the author

✨What it does Useful for scraping only/API data. This workflow will give you the ability to bypass the IP address limitation control of some publicly available APIs by using a different couple of "user-agent" and "IP address" for each call to the targeted…

View the original on n8n.io → Check your own copy