bot_errors — Telegram bot for monitoring parser errors
A Telegram bot written in Python that monitors an error table in PostgreSQL and sends each new entry to subscribers in real-time.
Slide 1 of 2
DESCRIPTION
The bot tracks the appearance of new rows in the parser.errors table (a log of external parser failures from the marketplace) and sends them to subscribers on Telegram. It is implemented using pure requests: a custom long polling getUpdates loop with offset management, timeout extension, and retries on network failures.
Multithreaded architecture
The architecture is multithreaded: the main thread processes commands (/start, /stop), while a background daemon thread polls the database. The list of subscribers is stored in memory in a set under threading.Lock. The connection to PostgreSQL (psycopg2, DictCursor) is configured with TCP keepalive for long-lived processes.
Self-configuration to the table
A non-trivial part is the self-configuration to the table structure. The bot queries information_schema.columns to find the integer key column (id or error_id of any integer/serial type), using it as a monotonic cursor. On the first run, it sets the baseline last_notified_id = MAX(id) in the singleton table bots.bot_errors_state to avoid sending historical
TOOLS USED
- Python
- Telegram Bot API
- requests
- PostgreSQL
- psycopg2
- SQL
- threading
- Git
RESULT
- Monitoring of the parser.errors table with real-time dispatch of new entries has been implemented.
- Manual long polling over requests with offset management and retries on network errors has been implemented.
- A multithreaded scheme has been built: the background thread polls the database, while the main one processes commands; subscribers are stored in a thread-safe manner.
- Auto-detection of the key column in the table through information_schema and a baseline cursor to exclude historical records on the first run has been implemented.
- Pagination of new row selection and stable storage of the cursor position in the singleton table have been implemented.
AI-ASSISTANT
Ask a question about this project