Multi-Broker Data Platform ā Portfolio Project (2026)
Why it exists: Demonstrate end-to-end data engineering depth: automated multi-source ingestion, medallion warehouse architecture, dbt-based transformation, and a live BI layer ā built production-style from scratch.
How it works: Three public APIs (CoinGecko, US Treasury, GitHub Events) feed raw data into a PostgreSQL Bronze layer via Apache Airflow. dbt models (orchestrated via Astronomer Cosmos) transform data through Silver and Gold layers using window functions, incremental strategies, and star-schema dimensional modeling. Downstream: a FastAPI read layer serves Gold table data as JSON endpoints; Metabase dashboards connect directly to the Gold schema for live BI reporting. Asset-based cascade triggering ensures Silver and Gold transforms only fire after successful upstream ingestion ā no blind temporal scheduling.
Tech used: Python, Apache Airflow 3, dbt Core, Astronomer Cosmos, PostgreSQL 16, FastAPI, SQLAlchemy Core, Metabase, Docker.
View on GitHub ā
E-commerce & Yandex Delivery Integration (2024-2026)
Why it exists: Enable online ordering and delivery for a federal drug store chain through marketplace and delivery partners.
How it works: Backend service integrating internal order system with Yandex Delivery API; handled order status synchronization, delivery updates, and failure retries via RabbitMQ workers. Designed to tolerate inconsistent external callbacks and network errors.
Tech used: Python, Django, PostgreSQL, RabbitMQ, REST APIs.
Retail Data Warehouse (OLTP ā DWH) (2024-2025)
Why it exists: Provide management and regional offices with consolidated sales, stock, and pharmacy performance reports.
How it works: PostgreSQL-based DWH with multiple large fact tables (400M+ rows). Partitioning applied to manage growth. Complex SQL (CTEs, aggregates, window functions) used for reporting and monthly financial reconciliation.
Tech used: PostgreSQL, SQL, Python.
ERP & Legacy System Integration (2023-2024)
Why it exists: Synchronize product catalog, pricing, and stock data between legacy Java-based systems and new Python services.
How it works: Gradual migration from Java modules to Python services; API-based communication and async processing via RabbitMQ to avoid blocking critical retail operations.
Tech used: Python, Java (legacy), PostgreSQL, RabbitMQ.
Web Data Parsing & Competitor Monitoring (2023-2026)
Why it exists: Collect competitor pricing and product availability data for analytics and pricing adjustments.
How it works: Automated parsing tools with scheduled runs; data cleaned and loaded into internal reporting systems. Focused on resilience against layout changes and incomplete data.
Tech used: Python, BeautifulSoup, PostgreSQL.
Infrastructure & Server Configuration (2023-2026)
Why it exists: Maintain stable deployment and operation of backend services across staging and production environments.
How it works: Dockerized services; server configuration and monitoring setup; ensured reliable deployments and minimized downtime during updates.
Tech used: Docker, Linux, Nginx, PostgreSQL.
