AWS Data Automation Engineer

AWS Data Engineer – PySpark / Data Automation
Location: Remote within the U.S.; Eastern or Central time zones preferred
U.S. Citizenship Required – Ability to obtain Public Trust Clearance 
Pay: $55.00-$60.00 per hour
26-08455

Job Summary

The AWS Data Engineer designs, develops, and supports scalable cloud-based data pipelines and automation solutions using Python, PySpark, SQL, and AWS data services. This role focuses on building and troubleshooting production data workflows, integrating batch and streaming data sources, and optimizing data processing performance. The position also supports secure cloud environments, CI/CD practices, production operations, and emerging AI-enabled data solutions.

Job Responsibilities

• Design, develop, test, and support scalable Python and PySpark data pipelines.
• Build ETL/ELT workflows for large-scale structured and unstructured datasets.
• Develop batch and near-real-time data ingestion pipelines using Apache Spark and PySpark.
• Integrate streaming technologies such as Kafka, Flume, or similar platforms.
• Develop solutions using AWS services such as S3, Glue, Lambda, EMR, Step Functions, and AWS database technologies.
• Develop and optimize complex SQL-based data transformations and queries.
• Integrate REST APIs and external data sources into automated data workflows.
• Troubleshoot and optimize Spark jobs, SQL queries, data pipelines, and production workloads.
• Support data reconciliation, performance monitoring, incident resolution, and root-cause analysis.
• Apply source-control and CI/CD practices using GitHub, Azure DevOps, Jenkins, or similar tools.
• Support secure cloud environments using IAM, encryption, VPC controls, and role-based access.
• Participate in Agile and DevOps delivery practices and production support activities.
• Support AI and GenAI-enabled data solutions using platforms such as Amazon Bedrock, Azure OpenAI, or other LLM services.
• Collaborate with technical and business teams to develop reliable, scalable, and maintainable data solutions.

Job Requirements

• Minimum of 5 years of experience in data engineering, data automation, or a related field.
• Demonstrated hands-on experience building and supporting production Apache Spark/PySpark data pipelines.
• Strong hands-on Python and SQL skills.
• Experience building and supporting production data pipelines.
• Experience with AWS-based data engineering environments.
• Experience with multiple AWS services, including S3, Glue, Lambda, EMR, Step Functions, or AWS databases.
• Experience with ETL/ELT and data pipeline development.
• Experience with Kafka, Flume, or similar streaming and ingestion technologies.
• Experience integrating REST APIs and external data sources.
• Experience troubleshooting production data platforms, including performance tuning, incident resolution, reconciliation, and root-cause analysis.
• Experience with source control and CI/CD tools such as GitHub, Azure DevOps, Jenkins, or similar platforms.
• Bachelor’s degree in Computer Science or a related technical field.
• U.S. citizenship required.
• Ability to obtain and maintain a Federal Public Trust.
• Ability to work remotely within the U.S.
• Ability to travel occasionally or quarterly to the Gaithersburg, MD/Washington, DC area as required.

Preferred Qualifications

• Experience working in AWS and Azure multi-cloud environments.
• Experience with Amazon Bedrock, Azure OpenAI, LangChain, or other GenAI/LLM technologies.
• Experience with RAG, embeddings, vector search, Solr, or OpenSearch.
• Experience with Docker and Kubernetes.
• Experience with Terraform or other infrastructure-as-code tools.
• AWS or Azure certifications.

Additional Job Details (Commute, Attendance, etc.)

• Fully remote position within the U.S.
• Eastern or Central time zone candidates preferred.
• Occasional or quarterly travel to the Gaithersburg, MD/Washington, DC area may be required.
• Federal Public Trust eligibility is required before starting the assignment.

#ED

About Equiliem

Equiliem believes in empowering success. It’s our job to cultivate relationships that connect people and employers in a way that is inclusive, intelligent, and allows both to thrive.

Across the U.S., leading companies in healthcare, government, engineering, manufacturing, professional services, and energy rely on us for their workforce solutions. Our recruiting and HR services include contract and direct hire staffing, Payrolling/EOR, Independent Contractor Compliance, and Managed Services.

For almost 30 years, we’ve helped shape our industry. Today, we continue to research, ask questions, and continuously enhance the candidate journey and client experience.

Benefits

We offer a comprehensive benefits package to our employees, which includes:

• Medical Insurance
• Vision & Dental Insurance
• Life Insurance
• Matching 401(k)
• Commuter Benefits
• Employee Discounts & Rewards
• Payroll Payment Options

Future Communication Consent

By applying, you are providing consent for Equiliem to engage with you via phone calls, AI-generated calls, AI-generated text messages, standard text messages, and/or emails to share job opportunities. Consistent with our Privacy Policy, information obtained will not be shared with third parties for marketing or promotional purposes. Message frequency may vary for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You may reply STOP to cancel or HELP for assistance.

EEO Employer

Equiliem is an equal opportunity employer. We do not discriminate or allow discrimination based on race, color, religion, creed, sex, age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Equiliem will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at [email protected] or (732) 238-6050.