Michael P.

Led the development of the 'Customer 360 & Order Intelligence' data platform, creating end-to-end ELT pipelines in Snowflake for comprehensive customer, order, and fulfillment analytics across various business functions. Architected an analytics layer in Snowflake and dbt, focused on transformation models with SQL-first development and enforced CI/CD practices to enhance efficiency. Managed domain-based data pipelines that sourced data from MSSQL systems, SaaS tools, and APIs using Fivetran, ensuring schema evolution without disruption. Implemented Databricks SQL notebooks and PySpark jobs for large-scale transformation, exploratory analysis, and validation tasks, orchestrated through Airflow. Introduced data quality checks and anomaly detection systems, utilizing dbt tests and Python utilities for improved pipeline issue detection. Enhanced Snowflake performance and cost efficiencies via clustering and query refactoring. Developed near-real-time ingestion pipelines with Kafka, significantly decreasing data latency. Supported Reverse ETL workflows by preparing datasets for customer-facing and marketing systems, maintaining metric consistency. Packaged data utilities using Docker, collaborating with platform teams for Kubernetes deployment. Worked alongside platform and DevOps teams in managing data infrastructure utilizing Terraform and AWS. Engaged in cross-cloud analytics validation across AWS and Azure environments to maintain dataset accuracy. Provided mentorship to junior data engineers, conducted code reviews, and collaborated with stakeholders to create dimensional models for analysis.

Designed scalable batch and streaming data pipelines to ingest and process high-volume event data using Kafka and Spark, facilitating near-real-time analytics for the marketplace. Developed and refined large-scale dimensional models in Hive and Presto, enabling effective experimentation and reporting. Conducted data profiling and schema validation to manage event changes and reduce data discrepancies. Collaborated with data scientists and product teams on projects involving pricing and marketplace health metrics, while enhancing pipeline reliability through various optimization techniques.

Designed and managed enterprise ETL pipelines with SSIS to integrate data across manufacturing, finance, and supply-chain sectors from Oracle and SQL Server. Developed and optimized star and snowflake schemas to support analytics and reporting needs. Handled schema updates and source-system changes while minimizing disruption to reporting. Automated data validation and reconciliation processes using SQL and Python, improving data quality and reducing manual efforts. Worked closely with business stakeholders to create well-documented data models that aligned with reporting requirements.

Created SQL-based analytical datasets and reports for finance and operations teams. Conducted ad-hoc analyses to identify data quality issues and supported the development of dashboards and executive reports. Collaborated with engineers to clarify metric definitions and enhance reporting logic, laying the groundwork for future data engineering responsibilities.