Lead Data Engineer (Lakehouse / Databricks)
Stafford Gray
- Lansing, MI
- Hybrid
- Senior level
- 4+ yrs experience
- Data Engineering
- Visa: not specified
The role in 30 seconds
Stafford Gray is seeking a hands-on Lead Data Engineer to architect and build a modern cloud data lakehouse for a large public-sector organization. The role is the technical lead for data engineering: designing and building pipelines that feed the enterprise data lake, turning raw data into governed data products, and coaching in-house data engineers on standards and best practices. The engineer works alongside data scientists, analysts, and an external research team, and still builds every day while setting direction for the team.
Is this role a fit for you?
The backgrounds this role is most likely to hire from, as RoleStack reads the posting.
Strong match
- Data platform / ML pipeline
- B2B infrastructure / API / developer tools
- Enterprise-scale
- Regulated environment
Workable stretch
- B2B enterprise platform / workflow software
- Internal tooling / back-office systems
- Growth-stage
Harder transition
- B2C consumer-facing application
- Embedded systems / hardware-adjacent
- Startup / greenfield
Industry: Government / Public Sector. See your personal match score
What they're really screening for
Must-haves
- PLUS Bachelor's degree in Computer Science, Data Science, Information Systems, or a related field
- CORE 4+ years of hands-on experience building extract-and-load pipelines into an enterprise data lake on a major cloud provider
- CORE 4+ years of hands-on experience transforming data lake datasets into data products with traceability to the systems of record
- PLUS 2+ years of experience enforcing data sharing or access policies on data products
- CORE 4+ years of programming in Python, PySpark, and SQL, including SQL tuning and loading data from databases, flat files, CSV, JSON, and APIs into Parquet
- PLUS Experience designing star-schema data stores for BI reporting
- PLUS Must live within 90 miles of Lansing, MI at the time of submission and be able to work a hybrid onsite schedule
- PLUS Experience building data pipelines for a public-sector organization
- CORE Experience with cloud data lakehouse architecture (Medallion bronze/silver/gold) on AWS, Azure, or GCP
- PLUS Experience setting data quality, monitoring, validation, and governance standards
Nice-to-haves
- PLUS Databricks, Spark, DBT, Glue, Redshift Spectrum, and SageMaker
- PLUS Kafka, Kubernetes, and Docker
- PLUS DevOps and infrastructure-as-code: GitHub, Jenkins, Terraform, YAML, and Airflow DAGs
- PLUS Cloud-native data governance and security tooling
- PLUS Experience with NoSQL and graph databases, and with ML pipelines
- PLUS A track record of championing data engineering standards across a developer community
Résumé keywords for this role
Terms the posting stresses. Include the ones you can honestly back up.
Tools & technologies
- AWS
- Azure
- GCP
- Python
- PySpark
- SQL
- Parquet
- CSV
- JSON
- Power BI
- Tableau
- Databricks
- Spark
- DBT
- Glue
Concepts & phrases
- Lead Data Engineer
- cloud data lakehouse
- enterprise data lake
- Medallion (bronze/silver/gold) architecture
- Parquet-based data products
- traceable back to its source system
- data sharing and access policies
- star-schema operational data stores
- data quality, monitoring, validation, and governance standards
- design reviews, code reviews, and hands-on training
- extract-and-load pipelines
- systems of record
- SQL tuning
- infrastructure-as-code
- Airflow DAGs
Words that count as the same thing: "Medallion (bronze/silver/gold) architecture" ≈ Multi-hop lakehouse architecture, Bronze/Silver/Gold data layers, Delta Lake medallion pattern · "extract-and-load pipelines" ≈ ELT pipelines, Data ingestion pipelines, ETL/ELT development · "data products" ≈ Curated datasets, Analytics-ready datasets, Governed data assets · "traceability to systems of record" ≈ Data lineage, Source-to-target mapping, Data provenance · "star-schema" ≈ Dimensional modeling, Kimball modeling, Fact and dimension tables
Check my résumé against these keywordsHiring at Stafford Gray
Open senior roles RoleStack tracks right now, posted in the last 30 days.
More open roles at Stafford Gray
Full job description from Stafford Gray
From Stafford Gray's posting. View the original
Stafford Gray is looking for a hands-on Lead Data Engineer to architect and build a modern cloud data lakehouse for a large public-sector organization. You’ll be the technical lead for data engineering. You’ll design and build the pipelines that feed the enterprise data lake, turn raw data into governed data products, and coach the in-house data engineers on standards and best practices. You’ll work alongside data scientists, analysts, and an external research team. This role is for someone who still builds every day and also sets direction for the team.
Show full description Show less
What you’ll do
• Design, build, and maintain pipelines that ingest relational, semi-structured, and unstructured data into a cloud data lake on AWS, Azure, or GCP.
• Transform lake datasets through a Medallion (bronze/silver/gold) architecture into Parquet-based data products, with every attribute traceable back to its source system.
• Enforce data sharing and access policies so each subscriber sees only the version of a data product they’re entitled to.
• Design star-schema operational data stores that feed Power BI and Tableau reporting.
• Set data quality, monitoring, validation, and governance standards.
• Lead and mentor data engineers through design reviews, code reviews, and hands-on training.
• Partner with data scientists, analysts, product teams, and business stakeholders to deliver data and ML solutions.
Required
• Bachelor’s degree in Computer Science, Data Science, Information Systems, or a related field.
• 4+ years of hands-on experience building extract-and-load pipelines into an enterprise data lake on a major cloud provider.
• 4+ years of hands-on experience transforming data lake datasets into data products with traceability to the systems of record.
• 2+ years of experience enforcing data sharing or access policies on data products.
• 4+ years of programming in Python, PySpark, and SQL, including SQL tuning and loading data from databases, flat files, CSV, JSON, and APIs into Parquet.
• Experience designing star-schema data stores for BI reporting.
• Must live within 90 miles of Lansing, MI at the time of submission and be able to work a hybrid onsite schedule.
Preferred
• Databricks, Spark, DBT, Glue, Redshift Spectrum, and SageMaker.
• Kafka, Kubernetes, and Docker.
• DevOps and infrastructure-as-code: GitHub, Jenkins, Terraform, YAML, and Airflow DAGs.
• Cloud-native data governance and security tooling.
• Experience with NoSQL and graph databases, and with ML pipelines.
• A track record of championing data engineering standards across a developer community.
FAQ
How much experience do you need?
The posting asks for 4+ years in data engineering. It is a senior-level role.
Does Stafford Gray sponsor visas for this role?
The posting does not say whether visa sponsorship is available. Ask the recruiter before you apply.
Is this role remote?
It is a hybrid role: part remote, part in the office.
What tools and technologies does the posting mention?
AWS, Azure, GCP, Python, PySpark, SQL, Parquet, CSV, JSON, Power BI, Tableau, Databricks, Spark, DBT, Glue.