Lead Data Engineer, Databricks (Remote, US)
V4C.ai
- Remote - US
- Remote
- Senior level
- 7+ yrs experience
- Data Engineering
- Visa: not specified
The role in 30 seconds
V4C.ai is a pure-play Databricks partner delivering production Lakehouse pipelines on customer engagements. Lead Data Engineers serve as the technical backbone of delivery, leading the engineering workstream from design through production handover across batch and streaming on messy enterprise source systems. The role owns workstream quality, performance and cost, and mentors engineers while working directly with customers and Databricks field teams.
Is this role a fit for you?
The backgrounds this role is most likely to hire from, as RoleStack reads the posting.
Strong match
- Data platform / ML pipeline
- B2B infrastructure / API / developer tools
- Enterprise-scale
- Growth-stage
Workable stretch
- B2B enterprise platform / workflow software
- Internal tooling / back-office systems
- Regulated environment
Harder transition
- B2C consumer-facing application
- Embedded systems / hardware-adjacent
- Startup / greenfield
Industry: Horizontal agnostic (consulting delivery across verticals; healthcare/life sciences preferred). See your personal match score
What they're really screening for
Must-haves
- CORE 7+ years of data engineering plus experience leading projects or teams
- CORE Strong proficiency in Python
- CORE Strong proficiency in SQL
- CORE Strong proficiency in Apache Spark
- CORE Strong proficiency in Delta Lake
- CORE Production experience building both batch and streaming pipelines at enterprise scale
- CORE Hands-on experience with AWS, Azure, or GCP and its data services
- PLUS Experience with data modeling
- PLUS Experience with Git
- PLUS Experience with CI/CD for data workloads
Nice-to-haves
- PLUS Healthcare or life sciences experience
- PLUS Existing Databricks Certified Data Engineer Associate or Professional certification
- PLUS Hands-on experience with Unity Catalog, Lakeflow, or Databricks Asset Bundles
- PLUS Experience with dbt, Airflow, or equivalent orchestration and transformation tooling
- PLUS Experience migrating a legacy ETL estate to a modern cloud platform
- PLUS Consulting or client-facing engineering background
Résumé keywords for this role
Terms the posting stresses. Include the ones you can honestly back up.
Tools & technologies
- Databricks
- PySpark
- SQL
- Delta Lake
- Apache Spark
- Python
- AWS
- Azure
- GCP
- Git
- CI/CD
- Unity Catalog
- Lakeflow
- Databricks Asset Bundles
- dbt
Concepts & phrases
- Lead Data Engineer
- Databricks partner
- production Lakehouse pipelines
- batch and streaming pipelines
- Medallion architecture
- data quality and validation as code
- Spark performance
- platform cost
- client communication skills
- technical trade-offs
- legacy ETL estate
- modern cloud platform
- client-facing engineering
- data engineering
- leading projects or teams
Words that count as the same thing: "PySpark" ≈ Apache Spark, Spark, Python Spark · "Delta Lake" ≈ Databricks Delta, Lakehouse tables · "Medallion architecture" ≈ Bronze/Silver/Gold architecture, multi-hop lakehouse architecture · "Lakeflow" ≈ Databricks Lakeflow, Delta Live Tables, DLT · "CI/CD" ≈ continuous integration/continuous delivery, continuous deployment
Check my résumé against these keywordsHiring at V4C.ai
Open senior roles RoleStack tracks right now, posted in the last 30 days.
More open roles at V4C.ai
Full job description from V4C.ai
From V4C.ai's posting. View the original
About V4C
V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named Databricks' 2026 Partner of the Year (C&SI Emerging).
As a Lead Data Engineer, you will lead the build of production Lakehouse pipelines on customer engagements, working directly with customers and often Databricks' own field teams. Every technical hire here gets a certification roadmap built for them and taught in-house by Databricks-approved trainers, plus a dedicated Employee Success Manager whose only responsibility is your experience at V4C.
Show full description Show less
The Role
Lead Data Engineers are the technical backbone of V4C delivery. You lead the engineering workstream on an engagement — translating an architecture into working pipelines, setting the patterns the team follows, and owning the quality of what ships. You will work across batch and streaming, on messy enterprise source systems, under real deadlines, and in front of the customer explaining what you built and why. The difference from a Data Engineer is ownership: you are accountable for a workstream and the engineers on it.
Responsibilities
• Lead engineering on customer engagements, from design through production handover
• Build and optimize batch and streaming pipelines using PySpark, SQL, and Delta Lake on Databricks
• Implement Medallion architecture across cloud ecosystems and set the layer conventions the team follows
• Own performance and cost: benchmark Spark jobs, tune SQL, right-size compute, and explain the trade-offs to the customer
• Implement data quality and validation as code, including Lakeflow expectations and automated testing in CI
• Mentor mid-level and junior engineers through code review, pairing, and design feedback
• Partner with the Data Architect on design decisions, and escalate when implementation reveals the design will not hold
• 7+ years of data engineering plus experience leading projects or teams
• Strong proficiency in Python, SQL, Apache Spark, and Delta Lake
• Production experience building both batch and streaming pipelines at enterprise scale
• Hands-on experience with AWS, Azure, or GCP and its data services
• Experience with data modeling, Git, and CI/CD for data workloads
• Demonstrated experience optimizing Spark performance and managing platform cost
• Strong client communication skills, including the ability to explain technical trade-offs to a non-technical stakeholder
Preferences
• Healthcare or life sciences experience preferred but not required
• Existing Databricks Certified Data Engineer Associate or Professional certification
• Hands-on experience with Unity Catalog, Lakeflow, or Databricks Asset Bundles
• Experience with dbt, Airflow, or equivalent orchestration and transformation tooling
• Experience migrating a legacy ETL estate to a modern cloud platform
• Consulting or client-facing engineering background
FAQ
How much experience do you need?
The posting asks for 7+ years in data engineering. It is a senior-level role.
Does V4C.ai sponsor visas for this role?
The posting does not say whether visa sponsorship is available. Ask the recruiter before you apply.
Is this role remote?
It is a remote role.
What tools and technologies does the posting mention?
Databricks, PySpark, SQL, Delta Lake, Apache Spark, Python, AWS, Azure, GCP, Git, CI/CD, Unity Catalog, Lakeflow, Databricks Asset Bundles, dbt.