New 
Python Engineer, PySpark/Spark Performance Tuning, Azure DevOps/Jenkins, Hive/Impala; NJ
 Genesis10 | |
 United States, New Jersey, Jersey City | |
 Feb 03, 2026 | |
|
Start: ASAP
Glider: Candidate Identification Only 3 days Onsite Required Jersey City, NJ - NJ2-101-06-01. MUST INCLUDE CURRENT LOCATION AND INTENDED ON LOCATION RESUME The Python Engineer designs and implements data ingestion and transformation jobs using Python and PySpark. They build reusable frameworks for data quality checks and integrate pipelines with CI/CD processes. The role requires performance tuning, secure coding practices, and strong debugging skills. Close collaboration with architects and DataOps teams ensures robust and compliant solutions. Required Qualifications The Python Engineer designs and implements data ingestion and transformation jobs using Python and PySpark. They build reusable frameworks for data quality checks and integrate pipelines with CI/CD processes. The role requires performance tuning, secure coding practices, and strong debugging skills. Close collaboration with architects and Data Ops teams ensures robust and compliant solutions. Desired Qualifications Strong Python (packaging, virtual environments), PySpark/Spark performance tuning. Experience with data ingestion (batch/stream), schema management, and error handling/retry logic. Test discipline: unit/integration tests, data quality assertions, reproducible pipelines. CI/CD (Azure DevOps/Jenkins), Git workflows, artifact versioning, release readiness. Experience on Cloudera/Hadoop (HDFS, Spark, Hive/Impala) and Databricks (clusters, jobs, notebooks, Delta). Observability: structured logging, metrics, tracing; debuggability in distributed contexts. Secure coding and handling of secrets, PII protection, and compliance considerations. Strong communication and collaborative work style; documentation of frameworks and usage patterns. Performance optimization: partitioning, caching, broadcast joins, memory tuning. Comfort with Agile ceremonies and iterative delivery. Min 5 yrs of experience Enable Skills-Based Hiring No Qualification Assessment Must Have Technical Python and 0 years Technical by default Microsoft Azure and 0 years Spark and 0 years | |