Job Description

LeverageTek is actively seeking a Data Engineer (Python / PySpark) for a 5-month contract with its Ottawa-based customer.

Key Tasks

• Convert remaining Python code packages to PySpark, including testing, validation and integration into the nightly ETL pipeline.
• Review all existing business-owned code packages to identify and implement performance optimizations for nightly pipeline execution.
• Write, validate and integrate code improvements to improve pipeline efficiency and accuracy.
• Document end-to-end ETL pipeline processes for package ingestion (time permitting).
• Collaborate with IT colleagues to support improvements to pipeline flow, automation and efficiency (time permitting).

Key Qualifications

• Intermediate to Senior hands-on experience with Apache Spark (PySpark).
• Intermediate to Senior experience with Python, including code conversion and optimization.
• Experience with GitLab or GitHub for version control and collaborative development.

Qualifications

• Experience with AWS EMR Clusters (asset).
• Experience with Apache Airflow (asset).
• Healthcare data background preferred.
• Experience working within existing ETL pipeline architectures.
• Strong ability to work independently in a fully remote environment.
• Familiarity with AWS data services (S3, Glue, Redshift, Lake Formation) considered an asset.

Compensation

Hourly Rate Negotiable. This is an existing position vacancy.

About LeverageTek Staffing Solutions

Founded in 2003, LeverageTek provides end-to-end, cross-functional staffing solutions throughout North America. We are a trusted partner to leading private and public sector organizations and experts in talent solutions that create optimal business outcomes.

Don't let our name fool you. Our roots are in technology, but we are also a proven leader in accounting and finance, sales and marketing, human resources, supply chain, and legal talent acquisition. We offer contract and permanent staffing, executive search, talent mapping, management consultancy, and contractor payroll management.

LeverageTek is an equal opportunity employer. We offer a welcoming and inclusive environment in service to one another, our customers, the candidates we represent, and the diverse communities we call home. We do all of this with kindness, empathy, and respect for each other. LeverageTek is committed to employment equity and creating a diverse and inclusive workplace. We welcome applications from all qualified individuals regardless of race, religion, gender, national origin, age, disability, and marital status.

Accessibility accommodations are available upon request.

LeverageTek's specialized recruiting professionals use their expertise and our proprietary AI to accelerate connections between top talent and leading companies, while ensuring all screening and hiring decisions remain with our expert human team.