Scrape MMC doctor records by APC year and log matches to Google Sheets
Ready for 3.0What this workflow does
This workflow has 17 nodes and runs manually, when you click “Execute workflow”. Along the way it connects Google Sheets.
How it runs
Trigger: Manually, when you click “Execute workflow”.
- Set Year and Target (Edit Fields (Set)): sets 2 fields: targetYear, dailyTarget
- Load Previous Progress (Code): runs custom JavaScript code
- Fetch Doctor Page (HTTP Request): calls an HTTP API
- Parse Doctor Rows (Code): runs custom JavaScript code
- Check More Rows (If): checks a condition and branches into true / false
- Fetch Doctor Details (HTTP Request): calls an HTTP API
- Proceed to Next Page (Code): runs custom JavaScript code
- Match Check & Extract Data (Code): runs custom JavaScript code
- Check All Pages Processed (If): checks a condition and branches into true / false
- Evaluate Match (If): checks a condition and branches into true / false
…plus 4 more steps. See the full node list below.
Services it uses
Before you upgrade to n8n 3.0
Nothing in this workflow is removed in n8n 3.0, so it should import and run after the upgrade.
Nodes used (8)
- Manual Trigger
n8n-nodes-base.manualTrigger - Edit Fields (Set)
n8n-nodes-base.set - Code
n8n-nodes-base.code - HTTP Request
n8n-nodes-base.httpRequest - If
n8n-nodes-base.if - Google Sheets
n8n-nodes-base.googleSheets - No Operation, do nothing
n8n-nodes-base.noOp - Wait
n8n-nodes-base.wait
From the author
Quick overview This workflow manually scrapes the MMC MeRITS registered doctor directory, filters doctors by a target APC year, and writes matched records to Google Sheets while rate-limiting requests and saving a resume state so it can continue from the last…