Multi-Broker Data Platform (2026)
Why it exists: Demonstrate end-to-end data engineering depth: automated multi-source ingestion, medallion warehouse architecture, dbt-based transformation, and a live BI layer ā built production-style from scratch.
How it works: Three public APIs (CoinGecko, US Treasury, GitHub Events) feed raw data into a PostgreSQL Bronze layer via Apache Airflow. dbt models (orchestrated via Astronomer Cosmos) transform data through Silver and Gold layers using window functions, incremental strategies, and star-schema dimensional modeling. Downstream: a FastAPI read layer serves Gold table data as JSON endpoints; Metabase dashboards connect directly to the Gold schema for live BI reporting. Asset-based cascade triggering ensures Silver and Gold transforms only fire after successful upstream ingestion ā no blind temporal scheduling.
Tech used: Python, Apache Airflow 3, dbt Core, Astronomer Cosmos, PostgreSQL 16, FastAPI, SQLAlchemy Core, Metabase, Docker.
View on GitHub ā
E-commerce & Yandex Delivery Integration (2024-2026)
Why it exists: Enable online ordering and last-mile delivery for a federal drug store chain through Yandex Delivery, handling ~1,000 orders per day across the integration.
How it works: Designed and built a backend service from scratch integrating the internal order management system with the Yandex Delivery API. The core challenge was reliability: external delivery APIs produce inconsistent callback behavior ā duplicate status events, missing confirmations, and undocumented edge cases on the vendor side. Built RabbitMQ-based retry workers with idempotent processing to handle all failure classes without corrupting order state. The hardest real-world problem wasn't technical - overnight business requirement changes forced full redesign of the order flow mid-project, requiring the integration logic to be restructured without downtime to the live ordering system.
Tech used: Python, Django, PostgreSQL, RabbitMQ, REST APIs.
Retail Data Warehouse (OLTP ā DWH) (2024-2025)
Why it exists: Give management, regional offices, and connected 1C systems a single source of truth for sales, stock, pharmacy performance, and financial reporting - replacing fragmented per-store queries against live OLTP tables.
How it works: Designed and maintained a PostgreSQL DWH covering 50+ fact and dimension tables spanning the full pharmacy retail domain: employee data, store metadata, inventory, orders, and pre-aggregated analytical tables. At 400M+ rows across core fact tables, partitioning was applied to keep query performance stable as data grew. Wrote complex reporting SQL - CTEs, window functions, multi-level aggregates - consumed by BI dashboards and internal 1C system integrations. The most persistent challenge was data integrity: OLTP systems in retail accumulate inconsistencies over time, and reconciliation work (ensuring DWH totals matched source system records) was a recurring, non-trivial part of maintaining the warehouse.
Tech used: PostgreSQL, SQL, Python.
ERP & Legacy System Integration (2023-2024)
Why it exists: Replace a brittle internal Java-based legacy system (built by a previous developer, largely undocumented) with clean Python services, without disrupting live pharmacy retail operations during the transition.
How it works: Executed a hard cutover migration of ~20 tables covering the full pharmacy retail domain - product catalog, pricing, stock, and operational data. The migration took 3 months, with the majority of that time spent on data quality: the legacy PostgreSQL database contained years of accumulated junk data - malformed records, broken references, missing values with no source to reload from. Where data could be reconstructed, it was reloaded from available sources. Where it couldn't, records had to be inferred, validated against business rules, and filled in manually - a process requiring close collaboration with business stakeholders to confirm correctness. Built API-based communication between legacy and new services with RabbitMQ async processing to prevent blocking during the transition period.
Tech used: Python, Java (legacy), PostgreSQL, RabbitMQ.
Web Data Parsing & Competitor Monitoring (2023-2026)
Why it exists: Track competitor pricing and product availability across 6 pharmacy retail websites to inform pricing decisions and market positioning.
How it works: Built and maintained automated scrapers for 6 full competitor websites running on weekly schedules. The core reliability challenge with long-running scrapers is layout drift - competitor sites update their HTML structure without notice, silently breaking naive scrapers. Built change detection logic that identifies when a page structure deviates from the expected pattern and flags it for review rather than silently producing corrupted data. Cleaned and normalized output loads into OLAP cubes for analytical consumption, enabling pricing and availability comparisons across competitors at the product level.
Tech used: Python, BeautifulSoup, PostgreSQL.
