Principal Systems Software Engineer - AI Inference & Hardware/Software Co-Design

Micron Technology

  • Richardson, TX
  • On-site
  • Staff/Principal level
  • 7+ yrs experience
  • Platform Engineering
  • Visa: not specified

The role in 30 seconds

Micron's storage and memory systems pathfinding team is seeking an experienced engineer with expertise across AI inference software stacks, including frameworks, runtimes, compilers, and hardware abstraction layers. The ideal candidate will lead software architecture and optimization efforts while collaborating closely with hardware teams to influence platform capabilities through hardware/software co-design.

Is this role a fit for you?

The backgrounds this role is most likely to hire from, as RoleStack reads the posting.

Strong match

  • B2B infrastructure / API / developer tools
  • Software, SaaS & AI Platform
  • Internal tooling / back-office systems
  • Enterprise-scale (high availability, compliance, large distributed systems)
  • Growth-stage (scaling existing systems, reliability concerns)

Workable stretch

  • B2C consumer-facing application
  • Marketplace / multi-sided platform
  • Startup / greenfield (build from scratch, speed over polish)

Harder transition

  • Embedded systems / hardware-adjacent
  • Regulated environment (healthcare, finance, government)

Industry: Semiconductors & Electronics. See your personal match score

What they're really screening for

Must-haves

  • PLUS Bachelor’s or Master’s degree with 10+ years of experience, or a PhD with 7+ years of experience, in Computer Science, Software Engineering, Computer Engineering, or a related field.
  • CORE Working knowledge of end-to-end AI inference software stack architectures, with hands-on development experience in multiple layers, such as frameworks, inference engines, ML compilers, compiler infrastructure, hardware runtimes, or hardware abstraction layers.
  • CORE Experience debugging issues spanning applications, libraries, operating systems, kernels, networks, and hardware/software interfaces.
  • CORE Proficient development skills in C, C++ and Python, with Linux-specific programming experience such as multithreading, IPC, ioctl, GDB, and the /proc and /sys filesystems.
  • CORE Experience programming GPUs or other accelerators using CUDA, SYCL, OpenCL, or a comparable programming model.
  • CORE Demonstrated ability to lead technical initiatives without supervisory authority.
  • CORE Familiarity with LLM serving internals, including KV-cache management, prefill and decode phases, KV-cache offloading, and disaggregated serving.
  • CORE Familiarity with inference engines and distributed serving frameworks such as SGLang, vLLM, NVIDIA Dynamo, or TensorRT-LLM.
  • CORE Strong understanding of heterogeneous compute architecture concepts, including memory hierarchies, NUMA, caching, accelerator architectures, and interconnects such as PCIe, CXL, NVLink, or UALink.
  • CORE Design and modify AI inference software stacks to enable proof-of-concept hardware capabilities.

Nice-to-haves

  • CORE Technical leadership experience in a systems-focused pathfinding or research organization.
  • CORE Technical knowledge of HBM, DRAM, and emerging memory technologies.
  • CORE Technical knowledge of NVMe SSDs, NAND, and next‑generation storage systems.

Résumé keywords for this role

Terms the posting stresses. Include the ones you can honestly back up.

Tools & technologies

  • AI
  • Runtimes
  • Compilers
  • Hardware Abstraction Layers
  • C
  • C++
  • Python
  • Rust
  • CUDA
  • SYCL
  • Linux
  • Emulators
  • LLM
  • KV-cache
  • SGLang

Concepts & phrases

  • Principal Systems Software Engineer
  • AI Inference & Hardware/Software Co-Design
  • transform how the world
  • innovating memory and storage
  • accelerate the transformation
  • storage and memory systems
  • AI inference software stacks
  • frameworks, runtimes, compilers
  • hardware abstraction layers
  • lead software architecture
  • hardware/software co-design
  • proof-of-concept hardware capabilities
  • system features
  • technical lead
  • small teams

Words that count as the same thing: "AI inference software stack" ≈ frameworks, inference engines, ML compilers · "LLM serving internals" ≈ KV-cache management, prefill and decode phases, KV-cache offloading · "heterogeneous compute architecture" ≈ memory hierarchies, NUMA, caching

Check my résumé against these keywords

Hiring at Micron Technology

Open senior roles RoleStack tracks right now, posted in the last 30 days.

More open roles at Micron Technology

Full job description from Micron Technology

From Micron Technology's posting. View the original

Our vision is to transform how the world uses information to enrich life for all.

Micron Technology is a world leader in innovating memory and storage solutions that accelerate the transformation of information into intelligence, inspiring the world to learn, communicate and advance faster than ever.

Micron's storage and memory systems pathfinding team is seeking an experienced engineer with expertise across AI inference software stacks, including frameworks, runtimes, compilers, and hardware abstraction layers. The ideal candidate will lead software architecture and optimization efforts while collaborating closely with hardware teams to influence platform capabilities through hardware/software co-design.

Show full description Show less

Responsibilities

• Design and modify AI inference software stacks to enable proof-of-concept hardware capabilities.
• Develop software to quantify and demonstrate the value of proposed system features.
• Lead software architecture and development efforts, both independently and as a technical lead for small teams.
• Research emerging AI software topics to identify memory and storage system challenges and opportunities.
• Develop and modify Linux applications and libraries using C, C++, Python, Rust, CUDA, and/or SYCL for advanced heterogeneous computing systems.
• Develop and use emulators to predict system‑level performance and functional impacts in advance of hardware availability.
• Debug system-level issues spanning applications, libraries, operating systems, kernels, system configuration, networks, and hardware/software interfaces.
• Use emerging AI tools to improve productivity.
• Collaborate across Micron organizations and with external partners.
• Mentor interns and earlier-career engineers.
• Present technical findings, insights, and innovations to internal or external customers.
Minimum Qualifications

• Bachelor’s or Master’s degree with 10+ years of experience, or a PhD with 7+ years of experience, in Computer Science, Software Engineering, Computer Engineering, or a related field.
• Working knowledge of end-to-end AI inference software stack architectures, with hands-on development experience in multiple layers, such as frameworks, inference engines, ML compilers, compiler infrastructure, hardware runtimes, or hardware abstraction layers.
• Experience debugging issues spanning applications, libraries, operating systems, kernels, networks, and hardware/software interfaces.
• Proficient development skills in C, C++ and Python, with Linux-specific programming experience such as multithreading, IPC, ioctl, GDB, and the /proc and /sys filesystems.
• Experience programming GPUs or other accelerators using CUDA, SYCL, OpenCL, or a comparable programming model.
• Demonstrated ability to lead technical initiatives without supervisory authority.
• Familiarity with LLM serving internals, including KV-cache management, prefill and decode phases, KV-cache offloading, and disaggregated serving.
• Familiarity with inference engines and distributed serving frameworks such as SGLang, vLLM, NVIDIA Dynamo, or TensorRT-LLM.
• Strong understanding of heterogeneous compute architecture concepts, including memory hierarchies, NUMA, caching, accelerator architectures, and interconnects such as PCIe, CXL, NVLink, or UALink.
Preferred Qualifications

• Technical leadership experience in a systems-focused pathfinding or research organization.
• Technical knowledge of HBM, DRAM, and emerging memory technologies.
• Technical knowledge of NVMe SSDs, NAND, and next‑generation storage systems.

As a world leader in the semiconductor industry, Micron is dedicated to your personal wellbeing and professional growth. Micron benefits are designed to help you stay well, provide peace of mind and help you prepare for the future. We offer a choice of medical, dental and vision plans in all locations enabling team members to select the plans that best meet their family healthcare needs and budget. Micron also provides benefit programs that help protect your income if you are unable to work due to illness or injury, and paid family leave. Additionally, Micron benefits include a robust paid time-off program and paid holidays. For additional information regarding the Benefit programs available, please see the Benefits Guide posted on micron.com/careers/benefits.

Micron is proud to be an equal opportunity workplace and is an affirmative action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, age, national origin, citizenship status, disability, protected veteran status, gender identity or any other factor protected by applicable federal, state, or local laws.

To learn about your right to work click here.

To learn more about Micron, please visit micron.com/careers

For US Sites Only: To request assistance with the application process and/or for reasonable accommodations, please contact Micron’s People Organization at hrsupport_na@micron.com or 1-800-336-8918 (select option #3)

Micron Prohibits the use of child labor and complies with all applicable laws, rules, regulations, and other international and industry labor standards.

Micron does not charge candidates any recruitment fees or unlawfully collect any other payment from candidates as consideration for their employment with Micron.

AI alert: Candidates are encouraged to use AI tools to enhance their resume and/or application materials. However, all information provided must be accurate and reflect the candidate's true skills and experiences. Misuse of AI to fabricate or misrepresent qualifications will result in immediate disqualification.

Fraud alert: Micron advises job seekers to be cautious of unsolicited job offers and to verify the authenticity of any communication claiming to be from Micron by checking the official Micron careers website in the About Micron Technology, Inc.

FAQ

How much experience do you need?

The posting asks for 7+ years in computer science, software engineering, computer engineering, or a related field. It is a staff/principal-level role.

Does Micron Technology sponsor visas for this role?

The posting does not say whether visa sponsorship is available. Ask the recruiter before you apply.

Is this role remote?

It is an on-site role.

What tools and technologies does the posting mention?

AI, Runtimes, Compilers, Hardware Abstraction Layers, C, C++, Python, Rust, CUDA, SYCL, Linux, Emulators, LLM, KV-cache, SGLang.