Scrape and summarize posts of a news site without RSS feed using AI and save them to a NocoDB
Needs changesWhat this workflow does
This workflow has 18 nodes and runs on a schedule (every week at 04:00). Along the way it connects NocoDB. It also calls colt.net over HTTP.
How it runs
Trigger: On a schedule (every week at 04:00).
- Retrieve the web page for further processsing (HTTP Request): calls colt.net
- Extract the HTML with the right css class (HTML): parses or builds HTML
- Extract date (HTML): parses or builds HTML
- Create single link items (Item Lists): reshapes item lists (legacy Item Lists node)
- Create single date items (Item Lists): reshapes item lists (legacy Item Lists node)
- Merge: merges the data from its input branches
- Select posts of last 7 days (Code): runs custom JavaScript code
- Merge: merges the data from its input branches
- HTTP Request: calls an HTTP API
- Summary (OpenAI): work with chat
…plus 7 more steps. See the full node list below.
Services it uses
HTTP calls to: colt.net
Before you upgrade to n8n 3.0
2 things in this workflow are removed in n8n 3.0. Fix them before you upgrade:
- OpenAI (legacy) in Summary, Keywords: replace with OpenAI node (AI section).
- Item Lists (legacy) in Create single link items, Create single date items: replace with Split Out, Aggregate, Sort, Limit, Remove Duplicates or Summarize (match the operation used).
| Removed in 3.0 | Replace with |
|---|---|
| OpenAI (legacy) Summary, Keywords | OpenAI node (AI section) |
| Item Lists (legacy) Create single link items, Create single date items | Split Out, Aggregate, Sort, Limit, Remove Duplicates or Summarize (match the operation used) |
Nodes used (9)
- HTML
n8n-nodes-base.html - OpenAI
n8n-nodes-base.openAi - Edit Fields (Set)
n8n-nodes-base.set - Merge
n8n-nodes-base.merge - Code
n8n-nodes-base.code - HTTP Request
n8n-nodes-base.httpRequest - Schedule Trigger
n8n-nodes-base.scheduleTrigger - NocoDB
n8n-nodes-base.nocoDb - Item Lists
n8n-nodes-base.itemLists
From the author
The News Site from Colt, a telecom company, does not offer an RSS feed, therefore web scraping is the choice to extract and process the news. The goal is to get only the newest posts, a summary of each post and their respective (technical) keywords. Note that…