Information overload
Filtering out that noise is relatively easy with tools such as Large Language Models. Instead of manually scanning countless sources every morning, you can use LLMs to filter news articles based on explicitly described interests and professional needs.
In this blog, I will show you a solution that filters noise from an IT-related RSS news feed, where I essentially let the LLM do the decision making: is an article worth reading or not? Please note: my evaluation is not so much about the specific application (filtering noise from an IT news feed), but mainly about the concept: using an LLM for a non-critical decision. This concept can easily be translated to many other domains, and I hope it inspires you to apply it to your personal or business needs.
The script
Let's start with the interesting part: the program itself. I wrote the script in Python, and it looks like this:
#!/usr/bin/env python3
from datetime import datetime, timedelta
from email.utils import parsedate_to_datetime
from openai import OpenAI
from os import getenv
import json
import requests
import subprocess
import xml.etree.ElementTree as ET
FEED_URL = "https://www.security.nl/rss/headlines.xml"
MODEL = "Claude-Haiku-3.5"
with open("system_prompt.txt", "r") as f:
SYSTEM_PROMPT = f.read()
def get_feed():
"""Get RSS feed"""
response = requests.get(FEED_URL)
response.raise_for_status()
return response.content
def parse(feed_content: bytes):
"""Parse RRS feed"""
root = ET.fromstring(feed_content)
channel = root.find("channel")
title = channel.findtext("title", default="(untitled)")
link = channel.findtext("link", default="")
description = channel.findtext("description", default="")
pub_date = channel.findtext("pubDate")
last_build = channel.findtext("lastBuildDate")
return {
"channel": {
"title": title,
"link": link,
"description": description,
"pubDate": pub_date,
"lastBuildDate": last_build,
},
"items": [
{
"title": item.findtext("title", default="(no title)").strip(),
"link": item.findtext("link", default="").strip(),
"description": item.findtext("description", default="").strip(),
"pubDate": item.findtext("pubDate", default="").strip(),
}
for item in channel.findall("item")
],
}
def extract_recent_items(feed, days_back: int = 0):
"""Check if the publication date is within the specified number of days back from today."""
recent_items = []
for item in feed["items"]:
dt = parsedate_to_datetime(item["pubDate"])
now = datetime.now()
today = now.date()
cutoff_date = today - timedelta(days=days_back)
item_date = dt.date()
if item_date >= cutoff_date:
recent_items.append(item)
return recent_items
def evaluate(items, filter=True):
"""Evaluate all items"""
api_key = getenv("POE_API_KEY")
client = OpenAI(api_key=api_key, base_url=getenv("POE_BASEURL"))
evaluations = []
for i, it in enumerate(items):
title = it["title"]
completion = client.chat.completions.create(
model=MODEL,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": title},
],
)
out = completion.choices[0].message.content
data = json.loads(out)
data["title"] = title
data["link"] = it["link"]
evaluations.append(data)
return evaluations
return evaluations
def notify(results):
"""Notify about all results that are noteworthy"""
for r in results:
if r["decision"] == "yes":
command = [
"/usr/local/bin/telegram.py",
f"=== {r['title']} ===\n\n{r['reasoning']}\n\n{r['link']}==========="
"".strip(),
]
subprocess.run(command, check=True, text=True, capture_output=True)
def main():
feed_content = get_feed()
items = parse(feed_content)
recent_items = extract_recent_items(items)
results = evaluate(recent_items)
notify(results)
if __name__ == "__main__":
main()
Copy code
The workflow of the script is as follows:
- Get an example RSS news feed; I use www.security.nl, a Dutch site that publishes IT-related news.
- Parse the feed to extract all news items.
- Filter only today's items.
- Evaluate the items with an LLM to determine whether they are important.
- Send a notification about the news article if the LLM determines that it is interesting.
Each step has its own function, and it is the main() function that calls all these functions.
Runtime!
Retrieve news items
The script starts by retrieving the RSS feed using the get_feed() function. This sends an HTTP request to the RSS endpoint of the site I am using and returns the response content.
Then parse() the feed content and all feed items are extracted. A single news item (an item in the items[“items”] list) looks like this, for example:
{
"title": "Ziekenhuis ontslaat medewerkers wegens ongeoorloofd inzien patiëntendossiers",
"link": "https://www.security.nl/posting/908552/Ziekenhuis+ontslaat+medewerkers+wegens+ongeoorloofd+inzien+pati%C3%ABntendossiers?channel=rss",
"description": "Het Albert Schweitzer Ziekenhuis heeft twee medewerkers ontslagen die elfhonderd patiëntendossiers ongeoorloofd inzagen. Het ...",
"pubDate": "Thu, 09 Oct 2025 17:10:37 +0200"
}
Copy code
To filter only today's items, the feed goes through extract_recent_items(), which by default only returns items from the last day. I do this so that I can schedule the script daily without seeing duplicate articles. If you want to expand this further, you can keep track of the status (for example, in a database) to record which items have already been processed. This allows you to run the script more than once a day.
Model selection, system prompt, and sending the message
With today's items filtered, an LLM can evaluate whether the article is worth reading, based on a system prompt (which I show below). The filtering is done in the evaluate() function. I use www.poe.com to interact with LLMs. Poe is a Quora company that recently introduced an OpenAI-compatible API. This allows me to use the OpenAI Python SDK to talk to LLMs via Poe.
As you can see at the beginning of the code, I am using Claude Haiku 3.5. This choice is based on cost and energy efficiency. Claude Haiku is a smaller model with fewer parameters. It was therefore cheaper to train, and inference is also cheaper, in terms of both energy and money. The trade-off is capability, but since we don't need enormous complexity here, this is more than sufficient.
The system prompt used for filtering by the LLM is read from a local file (system_prompt.txt). Below is an English translation of the prompt (I implemented it myself in Dutch because I use a Dutch news source and wanted my results in Dutch as well):
You are an IT news filter for a Linux consultant. Assess news headlines for relevance to Cloud consultancy work.
CONSULTANT PROFILE:
- Cloud consultant
- Works with: Ansible, Terraform, HashiCorp Vault, Python, AI/LLMs, Linux distributions, Kubernetes
- Has limited time - only let critical items through
FILTER CRITERIA:
YES (let through):
- Security issues in the above technologies or Linux
- Breaking changes/major releases of the above tools
- Major cloud/infrastructure outages
- Enterprise Linux regulations/compliance
- Zero-days, CVEs, critical patches
NO (filter out):
- Consumer tech, gaming, mobile apps
- Marketing announcements, tutorials, opinions
- Minor updates, conferences, startup news
- Regional news that doesn't have major impact on the IT industry
- Non-Dutch national news that doesn't have major impact on the IT industry
WHEN IN DOUBT: Filter out (be conservative)
OUTPUT:
Always return JSON with this structure:
{
"decision": "yes" | "no",
"reasoning": "brief explanation why relevant/not relevant for Linux consultant",
"other": "optional additional context if relevant"
}
No additional text is possible outside this structure. Do not provide explanations outside the JSON.
Focus on: Would this directly impact the consultant's work or clients?
Copy code
This prompt is also largely AI-generated. All I did was add some specific technologies that I often work with, and add the rule that the model should never give explanations outside the JSON, which it started doing while testing the prompt. For example, the raw output of the LLM for a single news item looks like this:
{
"decision": "no",
"reasoning": "Geen directe technische of IT infrastructuur relevantie voor Linux consultancy werk",
"other": "Privacy incident, meer geschikt voor HR of compliance afdelingen"
}
Copy code
In this format, downstream code can easily act based on the decisions made by the LLM.
Notifications
Finally, to actually receive notifications about interesting articles, the filtered items go through the notify() function. This uses a local tool that I wrote myself to send notifications via Telegram. I won't discuss this tool here, but I will show it to complete the script. You can, of course, replace it with any other notification method.
Translate this concept to other applications
It is not difficult to see how these techniques translate into business applications. Retailers use LLMs (and other forms of Natural Language Processing) for inventory management decisions, analysis of customer reviews, and more. In the legal sector, LLMs are used for contract analysis and legal research. Trading and hedge funds even use LLMs to support trading strategies and risk assessments.
Conclusion
The above is one way to use LLMs for non-critical decisions. By letting an LLM make decisions based on user input (the system prompt), you save a lot of time and effort and filter out noise from a news feed. It is a practical example, but the underlying ideas are easily applicable in many other domains.
Are you curious about how LLMs, or AI in general, can help with your personal or business needs? Or has this inspired you to implement LLMs for a personal or professional use case? Please feel free to contact us, we would love to hear from you.
