Principal Data Engineer

Accountabilities: • Design scalable and reliable data architectures, leading complex technical decisions related to data pipelines, ETL processes, data migration, performance, scalability, and platform reliability.

• Review technical implementations delivered by engineering team members and external resources, providing guidance on engineering standards and best practices.

• Analyze existing systems and identify opportunities to improve data processing, architecture, performance, and reliability.

• Create high-level technical designs covering ETL processes, data dictionaries, metadata, file layouts, data flows, validation, transformation, and loading requirements.

• Gather and analyze business and application requirements, identify gaps or inconsistencies, and translate them into effective technical solutions.

• Plan and execute Agile sprints while coordinating development, testing, deployment, and other phases of the software development lifecycle.

• Develop technical documentation for parallel jobs, batch processing, sequencers, and data migration activities.

• Participate in architecture, code, and design reviews with network, infrastructure, and solutions engineering teams to ensure alignment with enterprise standards.

• Lead and mentor junior engineers, promoting strong engineering practices and supporting technical growth across the team.

Requirements:

• Bachelor’s degree or foreign equivalent in Computer Science, Electronic Engineering, or a related field, followed by at least 7 years of progressive experience as a Senior Data Engineer, IT Consultant, or in a related role.

• At least 7 years of experience building data pipelines, data warehouse solutions, and frameworks supporting end-user reporting requirements.

• Strong experience gathering requirements, creating software designs, and developing efficient solutions within an Agile software development lifecycle.

• Proven ability to build data pipelines using diverse sources, including flat files, databases, APIs, and streaming applications.

• Hands-on experience building ETL and data warehousing solutions with IBM DataStage, Talend, and ODI.

• Strong experience designing data enrichment frameworks and processing large datasets using technologies such as Spark, Python, Talend, DataStage, ODI, Unix, PL/SQL, Teradata, AWS, and Java.

• Experience designing, developing, and automating ETL processes for data validation and reconciliation across multiple source systems.

• Proficiency with programming and scripting languages including Python, Unix, Java, JavaScript, C, and C++, including experience retrieving data from REST APIs and working with security keys.

• Extensive experience with AWS services, including S3, Glue, Athena, Lake Formation, Lambda, EC2, RDS, EMR, Redshift, and IAM, particularly for data migration and cloud-based processing.

• Experience working with cloud environments including AWS, Azure, and GCP and integrating cloud-based data with downstream systems.

• Experience developing and integrating APIs using REST web services.

• Familiarity with CI/CD tools such as Bitbucket, GitHub, and Jenkins.

• Strong technical leadership, communication, problem-solving, mentoring, and cross-functional collaboration skills.

• Ability to work remotely from any location within the United States.

Benefits:

• 100% remote work permitted from any US location.

• Full-time schedule of 40 hours per week.

• Additional compensation and benefits details were not specified in the source job description.

How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1