002ACH - Data Integration Engineer

Overview

We are seeking an experienced Data Integration Engineer with strong hands-on expertise in Amazon Redshift, AWS Glue, and large-scale data integration pipelines. The ideal candidate will design, build, and optimize data workflows that enable smooth data movement across various enterprise platforms.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue, PySpark, and Lambda.

• Implement and optimize data models and schemas in Amazon Redshift for high-performance analytics.

• Integrate data from multiple sources (APIs, databases, SaaS applications, streaming systems) into centralized data platforms.

• Automate ingestion, transformation, validation, and orchestration processes.

• Work closely with data engineers, architects, and business teams to define data integration strategy and mapping rules.

• Ensure data quality, accuracy, and consistency across systems.

• Implement monitoring, logging, and error-handling for data pipelines.

• Optimize query performance, cluster tuning, and workload management in Redshift.

• Troubleshoot pipeline issues and ensure high availability of integrations.

• Maintain and enforce data governance, security, and compliance standards.

Required Skills

• Strong hands-on experience with Amazon Redshift (Spectrum, Workload Management, Query Optimization).

• Deep knowledge in AWS Glue (ETL jobs, crawlers, catalog, PySpark).

• Excellent experience in Python, SQL, PySpark, and data transformation techniques.

• Strong understanding of data warehousing concepts and best practices.

• Experience in integrating data from Salesforce, ERP, RDBMS, APIs, and cloud data sources.

• Knowledge of AWS services such as S3, Lambda, Step Functions, Athena, CloudWatch.

• Experience working with structured, semi-structured (JSON, Parquet), and unstructured data.

• Strong debugging, performance tuning, and problem-solving skills.