• FAQ
  • Docs
  • Jobs
  • Contact
  • FAQ
  • Docs
  • Jobs
  • Contact
Get Started

Copyright © 2026 SpeedyApply. All rights reserved.

GitHubLinkedInDiscordDiscordTikTok

Resources

ContactFAQBlogAffiliate

Legal

Privacy PolicyTerms of Service

Chrome Web StoreEdge Add-onsAdd-ons for Firefox
  • FAQ
  • Docs
  • Jobs
  • Contact
  • FAQ
  • Docs
  • Jobs
  • Contact
Get Started

AIML - LLM Inference Quality Engineer

Apple • Seattle, WA

A
Apple

AIML - LLM Inference Quality Engineer

AI / MLNew GradSeattle, WA3 days ago

Summary

Apple is building the next generation of AI data center infrastructure to power large language model (LLM) services at scale. We are looking for a senior engineer to own the correctness, consistency, and stability of LLM inference in production. You will design and build the automation frameworks, evaluation methodology, and qualification strategy that give us confidence in every model, prompt, serving stack, and infrastructure change before it reaches our users. Your work is the safety net that lets Apple ship frontier AI quickly, without sacrificing quality, safety, or trust.

Description

In this role, you will lead the qualification of an LLM inference, output-correctness, automation framework for large-scale AI data center deployment and platform validation. You will architect qualification systems for generative AI, including agentic workflows and tool-using models, running at data center scale, validating that outputs remain correct, consistent, and stable as models, weights, prompts, serving runtimes, quantization schemes, and accelerator hardware change underneath them. This is a hands-on AI systems engineering role at the intersection of machine learning, distributed systems, and quality engineering. You will write production-quality software in Python, C++, and/or Swift, apply statistical and ML-evaluation methods to validate both data and model behavior, and build AI-assisted QA tooling that makes testing itself smarter and faster. You will partner closely with ML engineers, inference/performance engineers, distributed systems engineers, and platform teams, and you will be a trusted Quality voice on what “correct” and “stable” mean for LLM inference at scale, including the inherently non-deterministic, probabilistic nature of generative output. You should be comfortable operating in ambiguity, defining quality bars where none exist yet, and delivering rigorous results on tight schedules.

Responsibilities

  • Design and develop qualification frameworks for LLM inference systems deployed at data center scale, validating output correctness, consistency, and stability across model, weight, prompt, runtime, and infrastructure changes.
  • Build the LLM inference output-correctness automation framework used for large-scale deployment and platform validation.
  • Architect the end-to-end test and qualification strategy for distributed ML serving systems — spanning unit, integration, regression, and end-to-end layers.
  • Design harnesses that enable validation of ML/LLM services independently of upstream model and service dependencies (mocking, trace replay, and synthetic traffic generation).
  • Gate releases in CI: every change to a model, weight, prompt, tool definition, or runtime runs against the relevant evals, producing a before/after comparison a reviewer can read in one glance.
  • Create prompt-level and scenario-based regression suites, and track quality across model, prompt, and infrastructure versions over time.
  • Validate tool-use, function-calling, and multi-turn agentic workflows for correct invocation, argument formation, task completion, and multi-turn coherence; flag output that is wrong, empty, or silently degraded.
  • Develop safety, refusal, and policy-compliance coverage, verifying guardrail and alignment behavior holds across releases.
  • Probe AI-specific risk: prompt injection, data leakage across users and permissions, hallucinated or ungrounded output, and cost/latency regressions.
  • Build adversarial and robustness tests, long-context, ambiguous and underrepresented inputs, to surface failure modes before users encounter them.
  • Qualify inference-optimization techniques (quantization, KV-cache, speculative decoding, batching) for output parity against a full-precision reference.
  • Validate output equivalence across serving backends and accelerators, ensuring hardware and runtime swaps preserve generation quality.
  • Perform functional, performance, fault-injection, and reliability testing, benchmarking quality-vs-latency/throughput trade-offs to surface the correctness cost of performance optimizations.
  • Develop and use AI-powered/agentic QA tooling that improves test-case generation, defect detection, flaky-test analysis, and failure triage automating any check performed manually more than once.
  • Define and implement metrics, logging pipelines, distributed tracing, and observability tooling to monitor output correctness, model quality, data quality, drift, latency, and system health in production.
  • Apply log analysis using Splunk and Apple internal debugging tools to root-cause failures in complex LLM serving systems.
  • Perform regression, performance, and integration testing across platforms, including iOS and macOS, for on-device and server-backed AI features.
  • Develop production AI infrastructure software in Python, C++, and/or Swift, applying modularity, scalability, testability, and performance principles.

Minimum Qualifications

  • 1 year of experience with hardware and/or software development lifecycle processes.
  • 1 year of experience in one or more compiled languages (e.g., C, C++, Objective-C/Swift).
  • Proficiency in one or more scripting languages (e.g., Python, Go, or JavaScript).
  • Experience using AI/LLMs to improve testing itself — test-case generation, defect detection, or reducing manual QA effort.
  • Familiarity with LLM serving and inference optimization (batching, KV-cache, quantization, speculative decoding, distributed/multi-accelerator serving) and GPU-accelerated software.
  • Experience with CI/CD, Kubernetes, containers, and cloud-native systems, and integrating automated evals into release pipelines.
  • Experience with machine learning and LLMs, their common practical applications, and commonly used frameworks (e.g., PyTorch, Hugging Face Transformers, or comparable inference/serving stacks).
  • Experience with observability and distributed tracing for ML systems (e.g., Splunk, Prometheus, Grafana, LangSmith, or similar) and monitoring for model drift.
  • B.S., M.S., or Ph.D. in Computer Engineering, Electrical Engineering, Computer Science, or equivalent experience.

Preferred Qualifications

  • 5+ years using one or more of the following scripting languages: Python, Go, or JavaScript
  • 5+ years of experience working with and building operating systems
  • 2+ years of experience in LLM testing/development.

54 results

  • A
    AppleSoftware Engineering Intern - App Intelligence Experience
    Cambridge, United Kingdom
    Software EngineeringInternship
    2 days ago
  • A
    AppleIntern - Software Development Engineer
    Munich, Germany
    Software EngineeringInternship
    2 days ago
  • A
    AppleSoftware Engineer - AI & Data Platforms
    Hyderabad, India
    Software EngineeringNew Grad
    2 days ago
  • A
    AppleAR/VR Software Engineer - Spatial System UI - Vision Products Software
    Sunnyvale, CA
    Software EngineeringNew Grad
    2 days ago
  • A
    AppleAR/VR Software Engineer - Spatial System UI - Vision Products Software
    Boulder, CO
    Software EngineeringNew Grad
    2 days ago
  • A
    AppleGraduate Systems Software Engineer - File Provider Sync - CoreOS
    Paris, France
    Software EngineeringNew Grad
    3 days ago
  • A
    AppleAIML - LLM Inference Quality Engineer
    Seattle, WA
    AI / MLNew Grad
    3 days ago
  • A
    AppleSoftware Engineering Intern - Full-Stack & AI Integration - Shanghai
    Shanghai, China
    Software EngineeringInternship
    4 days ago
  • A
    AppleSoftware Engineering Intern - Apple Services - Dublin - Ireland
    Dublin, Ireland
    Software EngineeringInternship
    4 days ago
  • A
    AppleSoftware Engineering Intern - File Previewing & Editing - Quick Look and Preview
    Paris, France
    Software EngineeringInternship
    5 days ago
  • A
    AppleJunior Fullstack Engineer - ETS
    Hyderabad, India
    Software EngineeringNew Grad
    5 days ago
  • A
    AppleDefensive AI Security Intern
    Paris, France
    AI / MLInternship
    6 days ago
  • A
    AppleInternship - Pro Apps Machine Learning Ecosystem Engineer
    London, United Kingdom
    AI / MLInternship
    10 days ago
  • A
    AppleCamera Software - Intern
    Cambridge, United Kingdom
    Software EngineeringInternship
    10 days ago
  • A
    AppleInternship - Software Development Intern - Core OS
    Prague, Czechia
    Software EngineeringInternship
    10 days ago
  • A
    AppleMachine Learning Engineering Intern - Personal Agentic Search
    Cambridge, United Kingdom
    AI / MLInternship
    12 days ago
  • A
    AppleVirtualisation Software Engineering Internship - CoreOS
    London, United Kingdom
    Software EngineeringInternship
    12 days ago
  • A
    AppleIntern - Software Automation Engineer
    Cork, Ireland
    Software EngineeringInternship
    16 days ago
  • A
    AppleSoftware Engineer Intern - Full-Stack & AI Agents - Shanghai
    Shanghai, China
    Software EngineeringInternship
    18 days ago
  • A
    AppleSoftware Engineer - VE
    Cupertino, CA
    Software EngineeringNew Grad
    19 days ago
1 / 3Next

Logos provided by Logo.dev