Principal Data Architect
Babel Street
- Reston, VA ; Somerville, MA
- Hybrid
- Staff/Principal level
- 12+ yrs experience
- Data Architecture
- Visa: not specified
- $220k–$250k
The role in 30 seconds
Babel Street is seeking a Principal Data Architect to redesign and rebuild its petabyte-scale data infrastructure and pipelines, migrating the platform to Google Cloud. This role involves setting target architecture, writing critical code, and leading engineers to create a more cost-effective, reliable platform built for AI and agentic workloads. The architect will own pipelines, ingestion, storage, compute, search, cloud migration, AI enablement, and data quality, ensuring operability and technical leadership.
Is this role a fit for you?
The backgrounds this role is most likely to hire from, as RoleStack reads the posting.
Strong match
- B2B infrastructure / API / developer tools
- B2B enterprise platform / workflow software
- Software, SaaS & AI Platform
- Enterprise-scale
- Growth-stage
Workable stretch
- Internal tooling / back-office systems
- Data platform / ML pipeline
- Startup / greenfield
Harder transition
- B2C consumer-facing application
- Creator / builder / self-serve tool
- Marketplace / multi-sided platform
- Regulated environment (healthcare, finance, government)
Industry: Software, SaaS & AI Platform. See your personal match score
What they're really screening for
Must-haves
- CORE 12+ years building data systems, with at least 5 as the lead architect or principal engineer on a large-scale data platform.
- CORE You still write production code. Strong in Python plus at least one of Java, Scala or Go, and fluent in SQL.
- CORE You have personally led a major re-architecture or cloud migration of a data platform and can walk us through the design, the sequencing, what broke and what you would change.
- CORE Deep hands-on experience with distributed processing and streaming, for example Spark, Beam/Dataflow, Flink, Kafka, Pub/Sub or Pulsar.
- CORE Production experience with GCP data services such as BigQuery, Dataflow, Pub/Sub, GCS and Spanner. AWS experience is also expected, since you will be migrating from it.
- CORE Expert-level operation of search and index systems (Elasticsearch or OpenSearch) at scale, including cluster sizing, mappings, aggregation performance and cost tuning.
- CORE Experience with lakehouse table formats (Iceberg, Delta or Hudi) and orchestration (Airflow/Composer, Dagster or similar).
- PLUS A track record of cutting platform cost materially without hurting performance, and the numbers to show for it.
- PLUS Clear written communication. You can write a design doc that a CEO and a senior engineer both understand.
- CORE Ability to obtain a US security clearance
Nice-to-haves
- CORE Experience with vector search, embeddings pipelines and retrieval architectures for LLM and agentic applications.
- CORE Experience with entity resolution, knowledge graphs or graph databases, ideally across multilingual or messy real-world data.
- CORE Background in OSINT, national security or law enforcement, risk intelligence, and financial data
- PLUS Experience building for FedRAMP, IL5 or air-gapped environments, including running data platforms without managed cloud services.
- PLUS Experience with data observability, lineage and governance tooling (for example OpenLineage, Great Expectations, Monte Carlo, Dataplex).
- PLUS Experience combining large volumes of scraped multimedia and structured third-party data.
Résumé keywords for this role
Terms the posting stresses. Include the ones you can honestly back up.
Tools & technologies
- Python
- Java
- Scala
- Go
- SQL
- Spark
- Beam/Dataflow
- Flink
- Kafka
- Pub/Sub
- Pulsar
- GCP
- BigQuery
- GCS
- Spanner
Concepts & phrases
- Principal Data Architect
- Data Platform and Pipelines
- large-scale data platform
- major re-architecture
- cloud migration
- distributed processing and streaming
- GCP data services
- search and index systems
- lakehouse table formats
- platform cost materially
- vector search
- agentic applications
- entity resolution
- knowledge graphs
- graph databases
Words that count as the same thing: "agentic workflows" ≈ agent-based systems, AI agents · "petabyte-scale" ≈ Exabyte-scale data, Massive data storage · "lakehouse" ≈ Data lake, Data warehouse · "OSINT" ≈ Open Source Intelligence
Check my résumé against these keywordsHiring at Babel Street
Open senior roles RoleStack tracks right now, posted in the last 30 days.
More open roles at Babel Street
- Senior Software Engineer $150k–$160k
- Sr. Director of Engineering - API $220k–$240k
Full job description from Babel Street
From Babel Street's posting. View the original
Babel Street is the trusted technology partner for the world’s most advanced identity intelligence and risk operations. We deliver advanced AI and data analytics solutions providing unmatched, analysis-ready data regardless of language, proactive risk identification, 360-degree insights, high-speed automation, and seamless integration into existing systems. Babel Street empowers government and commercial organizations to transform high-stakes identity and risk operations into a strategic advantage. The actionable insights we deliver safeguard lives and protect critical assets around the world. Babel Street is headquartered in Reston, Virginia, with regional offices in Boston, MA and Cleveland, OH, and international offices in Australia, Canada, Israel, Japan, and the U.K. For more information, visit www.babelstreet.com.
Show full description Show less
Role summary
Principal Data Architect, Data Platform and Pipelines
We are hiring a hands-on architect to redesign and rebuild Babel Street's data infrastructure and pipelines. You will set the target architecture, write the critical code yourself, and lead engineers through a multi-year move of a petabyte-scale platform onto Google Cloud.
Babel Street collects, enriches and resolves open-source and commercial data across hundreds of sources and dozens of languages. Today that data flows through pipelines and stores that grew up separately: a mix of warehouse, search and object storage, plus legacy delivery systems that are hard to change and expensive to run. Your job is to replace that with one coherent platform that is cheaper, more reliable, and built for AI and agentic workloads.
This is a principal engineering role, not a people-management role. You will be judged on the systems you design and ship, the quality of your technical decisions, and how much you raise the bar for the engineers around you. You will report to the SVP of Engineering and work alongside our Head of Architecture, AI, Graph and Product teams.
What you will do in your first year
• In your first 90 days, map the current estate end to end: every source, pipeline, store, consumer and delivery path, with its volume, cost, owner and failure history. Publish a target-state data architecture and a sequenced migration plan that engineering and product leadership sign off.
• Design and build the new ingestion and processing backbone. Replace bespoke pipelines with a standard pattern for streaming and batch ingestion, enrichment, and delivery, running at millions of documents per day with clear SLOs.
• Lead the data platform workstream of our GCP migration. Turn "lift and shift" into real modernization: decide what moves as is, what gets rebuilt, and what gets retired, and own the technical design of each wave.
• Rationalize storage and search. Define when we use the warehouse (BigQuery), search and index (Elasticsearch/OpenSearch), object storage, vector and graph stores, then consolidate toward that pattern and cut the cost of our most expensive workloads.
• Build the data foundation for AI. Deliver agent-callable data services, hybrid retrieval (keyword, analytical, vector, graph), and provenance and lineage on every record so our AI products can cite their evidence.
• Build the governance and compliance platform. Centralize and serve governed and compliant data (e.g. privacy laws) as part of AI-readiness, setting parameters, exclusions, and retention for optimal performance of AI.
• Make the platform operable. Put in place data contracts, quality checks, observability and cost attribution so every pipeline has an owner, an SLO and a known unit cost.
What you will own
Pipelines and ingestion. Architecture and reference implementations for collection, ingestion, parsing, enrichment (translation, NER, entity resolution) and delivery. Event-driven and queue-based processing, idempotency, backfills and replay, schema evolution, and exactly-once or effectively-once guarantees where they matter.
Storage, compute and search. The physical data platform across warehouse, lakehouse, search and object storage. Partitioning, indexing, tiering and retention strategies at petabyte scale. Query and aggregation performance, and the hardware and cost trade-offs behind them.
Cloud migration and environments. Technical design for moving data workloads from AWS to GCP with minimal disruption to customers. Patterns that also work in FedRAMP, disconnected and air-gapped deployments, where managed cloud services may not be available.
Retrieval and AI enablement. APIs and query layers that serve both analysts and agents. Hybrid retrieval across structured, unstructured, vector and graph data, built with our AI and Graph teams to support RAG and agentic workflows.
Data quality, lineage and economics. Data contracts, automated quality checks, lineage and provenance, observability and SLOs. Unit economics for the platform: cost per source, per pipeline, per query and per customer.
Technical leadership. Architecture decision records, design reviews and coding standards for data engineering. Mentoring senior engineers, partnering with our offshore engineering teams, and being the escalation point for the hardest data incidents.
Required qualifications
• 12+ years building data systems, with at least 5 as the lead architect or principal engineer on a large-scale data platform.
• You still write production code. Strong in Python plus at least one of Java, Scala or Go, and fluent in SQL.
• You have personally led a major re-architecture or cloud migration of a data platform and can walk us through the design, the sequencing, what broke and what you would change.
• Deep hands-on experience with distributed processing and streaming, for example Spark, Beam/Dataflow, Flink, Kafka, Pub/Sub or Pulsar.
• Production experience with GCP data services such as BigQuery, Dataflow, Pub/Sub, GCS and Spanner. AWS experience is also expected, since you will be migrating from it.
• Expert-level operation of search and index systems (Elasticsearch or OpenSearch) at scale, including cluster sizing, mappings, aggregation performance and cost tuning.
• Experience with lakehouse table formats (Iceberg, Delta or Hudi) and orchestration (Airflow/Composer, Dagster or similar).
• A track record of cutting platform cost materially without hurting performance, and the numbers to show for it.
• Clear written communication. You can write a design doc that a CEO and a senior engineer both understand.
• Ability to obtain a US security clearance
Preferred qualifications
• Experience with vector search, embeddings pipelines and retrieval architectures for LLM and agentic applications.
• Experience with entity resolution, knowledge graphs or graph databases, ideally across multilingual or messy real-world data.
• Background in OSINT, national security or law enforcement, risk intelligence, and financial data
• Experience building for FedRAMP, IL5 or air-gapped environments, including running data platforms without managed cloud services.
• Experience with data observability, lineage and governance tooling (for example OpenLineage, Great Expectations, Monte Carlo, Dataplex).
• Experience combining large volumes of scraped multimedia and structured third-party data.
Education and location
Bachelor's degree in Computer Science, Engineering or a related technical field, or equivalent hands-on experience. A Master's or PhD is a plus but not required.
Benefits at Babel Street (just to name a few...)
• Health Benefits: Babel Street covers 85-100% monthly premium costs for Medical, Dental, Vision, Life & Disability insurances – for you and your family!
• Retirement Plans: Babel Street offers both a Traditional and Roth 401(K) with a very competitive match.
• Unlimited Flexible Leave: We trust our employees to manage their own time and balance their personal and work lives.
• Holidays: Babel Street provides employees with 12 paid Federal Holidays
• Tuition Reimbursement: We are committed to investing in our employees. One way we do that is with our Tuition Reimbursement Program for continuing education.
Babel Street is an equal opportunity/affirmative action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. Further, Babel Street will not discriminate against applicants for inquiring about, discussing or disclosing their pay or, in certain circumstances, the pay of their co‐worker, Pay Transparency Nondiscrimination. In addition, Babel Street's policy is to provide reasonable accommodation to qualified employees who have protected disabilities to the extent required by applicable laws, regulations and ordinances where a particular employee works. Upon request, we will provide you with more information about such accommodations.
Range for this position based on qualifications and experience
$220,000—$250,000 USD
FAQ
How much does a Principal Data Architect at Babel Street make?
The posted pay for this role is $220k–$250k a year.
How much experience do you need?
The posting asks for 12+ years in data architect. It is a staff/principal-level role.
Does Babel Street sponsor visas for this role?
The posting does not say whether visa sponsorship is available. Ask the recruiter before you apply.
Is this role remote?
It is a hybrid role: part remote, part in the office.
What tools and technologies does the posting mention?
Python, Java, Scala, Go, SQL, Spark, Beam/Dataflow, Flink, Kafka, Pub/Sub, Pulsar, GCP, BigQuery, GCS, Spanner.