Rohith B.
Senior Data Engineer
Open to offersNew to PlatformAbout
Rohith B. is a seasoned Data Engineer with over five years of experience transforming complex data into actionable insights. Specializing in building scalable and reusable data pipelines, Rohith excels in ensuring high-quality data delivery for analytics and business decision-making. With extensive expertise in ETL/ELT development and real-time data processing, he has a strong command of Python, SQL, and Apache Spark, and is adept with cloud platforms like AWS and Azure. Serving in prominent roles with organizations such as AgFirst and Costco, he has architected ETL pipelines processing massive volumes of data and developed real-time ingestion systems using Apache Kafka and Azure Event Hubs. Committed to data integrity, Rohith has led initiatives in data quality validation, efficient data storage solutions, and collaboration with cross-functional teams to deliver analytics-ready datasets. He holds a Master's degree in Information and Technology from Wilmington University and a Bachelor's in Computer Science Engineering from India's St. Martin’s Engineering College.
Experience
Data Engineer
AgFirst · 2024 — PresentDesigned and implemented scalable ETL pipelines utilizing Python, SQL, and Airflow to process large financial datasets. Optimized data pipelines for high availability and efficient delivery to business users and analytics. Developed real-time ingestion workflows with Apache Kafka to facilitate near real-time analytics. Created transformation workflows in PySpark to clean and aggregate data in distributed settings. Designed dimensional data models for BI reporting, and improved SQL query performance in Snowflake. Established data quality frameworks and built standard components for data ingestion. Partnered with data science teams to provide feature-engineered datasets for machine learning applications. Collaborated with stakeholders to translate requirements into technical solutions and maintain data integrity through validation and testing.Data Engineer
Costco · 2024 — 2024Constructed scalable ETL pipelines using Azure Data Factory and Databricks for the ingestion of POS and logistics data. Implemented transformations in PySpark with Delta Lake and the medallion architecture model. Developed real-time streaming pipelines utilizing Azure Event Hubs and Stream Analytics. Optimized Spark workloads through effective partitioning and caching practices. Automated infrastructure management and deployments with Terraform and Azure DevOps CI/CD. Ensured secure data solutions using RBAC and data masking. Managed data integration from multiple sources and provided troubleshooting support for internal users. Collaborated with cross-functional teams to establish scalable data models for analytics.Data Analyst (Data Engineering Focus)
Healthkart · 2020 — 2023Created ETL pipelines with AWS Glue and PySpark to process clickstream and sales data. Designed a data lake architecture in Amazon S3 deploying partitioned Parquet datasets. Facilitated ad-hoc data requests and reporting needs for business teams. Maintained data consistency and integrity across various systems and pipelines. Developed SQL transformations in Redshift to empower analytics and reporting functionalities. Implemented automated monitoring through CloudWatch and Lambda, optimizing query performance while minimizing data latency. Enhanced ETL pipeline performance by refining data partitioning and indexing strategies, supporting a transition to more automated reporting processes.
Skills & Expertise
Education
- Master in Information and TechnologyWilmington University
- Bachelor in Computer science engineeringSt. Martin’s Engineering College