Ilia Kopkov

Scope: High-throughput backend ingestion service and analytical data platform handling 600k+ daily transactions with zero-idle compute and strict latency SLAs.

• Async Ingestion & Serverless Compute: Engineered FastAPI event collection endpoints (asyncio, Pydantic, uvicorn, aiobotocore) paired with AWS Lambda and S3 raw sinks for seamless auto-scaling under burst traffic.
• Database & API Optimization: Designed data access layers and tuned complex PostgreSQL/Cloud DWH queries (multi-table JOINs, CTEs, execution plan tuning via EXPLAIN ANALYZE, Redis cache-aside), cutting DB load by 20% and reducing API response times from 1.5s to <200ms.
• ELT Orchestration: Built automated batch and streaming pipelines using Apache Airflow, dbt Core, and AWS Glue (Pandas) for raw event cataloging, deduplication, and downstream DWH modeling.
• Observability & Quality Gates: Developed custom Python Airflow operators with structured JSON logging, Prometheus/Grafana metrics, Sentry error tracking, and automated CI/CD test suites (pytest, dbt test at 80%+ coverage), cutting production incidents by 15% and lowering MTTR.

• Python
• FastAPI
• asyncio
• Pydantic
• PostgreSQL
• Redis
• AWS Lambda
• Amazon S3
• AWS Glue
• Apache Airflow
• dbt Core
• Terraform
• Prometheus & Grafana
• pytest

Scope: Production-grade data infrastructure platform automating multi-source external API ingestion (CoinGecko, US Treasury, GitHub), layered Medallion transformations, and low-latency REST metric delivery.

• Low-Latency Read Layer: Built an asynchronous REST API using FastAPI, Pydantic, and SQLAlchemy Core over PostgreSQL (with asyncpg connection pooling and Alembic migrations) to serve analytical aggregates without ORM serialization overhead.
• Event-Driven Orchestration: Containerized Apache Airflow 3 (Docker Compose) utilizing asset-based cascade triggers, custom Python ingestion operators (httpx/requests, Pandas, exponential backoff), and Astronomer Cosmos for modular dbt execution.
• Layered DWH Architecture: Designed a Medallion DWH (Bronze → Silver → Gold) in PostgreSQL with incremental materializations and automated data retention policies (48h Bronze, 7d Silver, 28d Gold).
• CI/CD & Data Quality Framework: Enforced automated schema contracts, uniqueness, freshness, and non-nullability constraints via dbt test and pytest.

• Python
• FastAPI
• SQLAlchemy Core
• Pydantic
• PostgreSQL
• Apache Airflow 3
• Astronomer Cosmos
• dbt Core
• Pandas
• Docker Compose
• GitHub Actions
• pytest

Scope: Asynchronous backend integration service bridging internal ERP systems with external delivery aggregators (Yandex Eats/Delivery API), unlocking a digital sales channel driving ~10% of total company revenue.

• Event-Driven State Sync: Engineered non-blocking message pipelines using RabbitMQ (aio-pika workers) and Django REST Framework to process high-volume order state transitions and inventory updates in real time.
• Fault Tolerance & Idempotency: Mitigated flaky third-party network dependencies by implementing Dead-Letter Queues (DLQ), exponential backoff retries, rate-limiting, and idempotent PostgreSQL UPSERT updates for duplicate webhooks, achieving 99.9% delivery reliability.
• Contract Validation & Exchange Schemas: Defined strict OpenAPI specifications, Pydantic payload validators, and PostgreSQL + dbt data exchange views to eliminate order state desynchronization across vendors.
• Legacy Migration & Deployment: Migrated legacy Java/SQL sync modules into containerized Python microservices (Docker, pytest, GitLab CI/CD) with end-to-end cross-system data validation.

• Python
• Django REST Framework
• RabbitMQ
• aio-pika
• Celery
• Pydantic
• PostgreSQL
• dbt Core
• OpenAPI
• Docker
• pytest

Scope: Centralized PostgreSQL Data Warehouse handling 400M+ row fact tables for daily financial reconciliation across 2,000+ pharmacies, paired with an automated multi-platform pricing ingestion engine.

• High-Volume PostgreSQL Tuning: Architected declarative table partitioning, composite/partial indexing, batch loading (COPY / bulk inserts), and window function optimizations to maintain strict daily SLA compliance on 400M+ row tables.
• Dimensional Modeling & DQ: Built incremental dbt Core models (SCD Type 2 historical tracking, Kimball star schema) and automated Data Quality validation suites (dbt test, custom SQL reconciliation assertions) to match live ERP totals.
• Resilient Scraping Ingestion: Engineered distributed web-scraping pipelines across 6 competitor platforms using Python (aiohttp, BeautifulSoup4, Selenium) and Pandas, featuring automated DOM layout-drift detection and rate-limit handling.

• Python
• PostgreSQL (400M+ rows)
• dbt Core (SCD2)
• SQL Optimization
• Table Partitioning
• Pandas
• aiohttp
• BeautifulSoup4 & Selenium
• Docker

Scope: Distributed backend services and real-time streaming pipelines across a 15+ microservice architecture processing 200k+ daily transactions for billing and parcel locker networks.

• Kafka Event Streaming: Built real-time data pipelines using Apache Kafka (processing up to 5k events/sec, consumer groups, offset management) and Pandas to synchronize core logistics, billing, and postmat state.
• Database Latency Reduction: Optimized high-traffic PostgreSQL read/write workloads via multi-table JOIN refactoring, window functions, batching, and index tuning, cutting ingestion and retrieval latency from ~45 mins to <2 mins.
• Microservice APIs: Developed and containerized 40+ Django REST Framework endpoints with Docker, implementing rate-limiting, exponential backoff retries, and circuit-breaking logic for 99%+ cross-service delivery reliability.

• Python
• Django REST Framework
• Apache Kafka
• PostgreSQL
• Pandas
• Microservices
• Docker
• REST APIs