← Back to results

Sr. Inference Engineer (LLM Serving)

Own the serving layer of GoodData Native Inference, deploying and optimizing LLMs on GPUs for production use.

Location
Prague, Czech Republic
Compensation
Not disclosed
Level
senior
Type
full time · Hybrid

Posted by employer 21 hours ago

First seen on Joblaze 8 hours ago

Last verified on the company career page 8 hours ago

Apply at GoodData.AI → Save job Scanned from gooddata.ai

What you'll build

  • Deploy, operate and upgrade open-weight LLMs
  • Tune batching, KV and prefix caching
  • Evaluate new open-weight models
  • Grow our inference gateway
  • Adapt models with LoRA adapters and fine-tuning

Must have

  • 5+ years of software engineering
  • 2+ years running ML or LLM models in production
  • Hands-on production experience with modern LLM serving engine
  • Experience with Docker, Kubernetes and GPU instances
  • Clear communication in English

Nice to have

  • Speculative decoding
  • Contributions to vLLM, SGLang, PyTorch
  • CUDA or Triton kernels
  • LLM routing and guardrail models
  • Go or Rust; Terraform and Helm
  • Observability for LLM systems

AI in the day-to-day

We run our AI on open-weight models on our own GPUs, ensuring customer data remains secure.

Requirements

Experience
5+ years

Not disclosed in this posting: compensation, visa sponsorship.

Joblaze summary

In this role, the Sr. Inference Engineer at GoodData.AI is responsible for managing the serving layer of their Native Inference platform, ensuring that AI models operate efficiently in production. Key skills include hands-on experience with modern LLM serving engines and a solid understanding of inference bottlenecks, as well as proficiency in Docker and Kubernetes. This position is ideal for someone with over five years of software engineering experience, particularly in machine learning, who can communicate effectively with both technical and product teams.

Joblaze insights

  • Listed today — first seen on Joblaze November 20, 2026. Last confirmed on GoodData.AI's careers page November 20, 2026.

Quick facts

Is the Sr. Inference Engineer (LLM Serving) role remote?
It's hybrid — GoodData.AI expects some on-site time in Prague, Czech Republic.
How much experience is required?
At least 5 years of relevant experience for this Sr. Inference Engineer (LLM Serving) role.
Where is the role based?
GoodData.AI is hiring for this position in Prague, Czech Republic.
What's the tech stack?
Joblaze extracted these technologies from the posting: AWS, Kubernetes, LangFuse, NVIDIA GPUs, PyTorch, Python.
What seniority level is this role?
GoodData.AI targets senior candidates for this position.
Is this full-time or contract?
Full-time for this Sr. Inference Engineer (LLM Serving) role at GoodData.AI.

From the original posting

Our platform turns governed business data into answers, reports and agents that companies put in front of their own customers. To make that AI trustworthy, private and affordable, we run it on our own inference: open-weight models on our own GPUs, so customer data never has to leave our boundary.

The role

You will own the serving layer of GoodData Native Inference. That means the models, the engine and the GPUs behind every AI answer our customers get. You will join the team that builds GoodData AI and become the person the whole company turns to for anything about running models in production.

What you will do

  • Deploy, operate and upgrade open-weight LLMs and smaller task models (guardrails, PII detection, classifiers, judges) on our GPUs, in production and across regions.

  • Make inference fast and cheap: tune batching, KV and prefix caching, quantization and parallelism, and track time to first token, tokens per second per GPU, p95 latency and cost per answer.

  • Choose the models: put new open-weight models through our evaluation harness on real analytics workloads, and recommend what goes to production.

  • Grow our inference gateway: routing by intent and complexity across models, a semantic cache, and validation that escalates weak answers.

  • Adapt models to our use cases with LoRA adapters and fine-tuning, such as generating queries for our semantic layer, root-cause analysis and narrative reports.

What you bring

  • 5+ years of software engineering, including 2+ years running ML or LLM models in production.

  • Hands-on production experience with at least one modern LLM serving engine: vLLM, SGLang or TensorRT-LLM.

  • A working understanding of inference bottlenecks (KV cache, attention, memory bandwidth, batching, parallelism), and the habit of reasoning about latency, throughput and cost in numbers.

  • Experience with Docker, Kubernetes and GPU instances in a public cloud.

  • Discipline in benchmarks and evaluation: you do not ship a model you have not measured.

  • Clear communication in English, with engineers and with product people.

Nice to have

  • Speculative decoding, multi-LoRA serving, or PEFT fine-tuning.

  • Contributions to vLLM, SGLang, PyTorch or similar open-source projects.

  • CUDA or Triton kernels, and the NVIDIA GPU stack (drivers, NCCL).

  • LLM routing, guardrail models, embeddings and rerankers.

  • Go or Rust; Terraform and Helm.

  • Observability for LLM systems (OpenTelemetry, Langfuse, Prometheus, Grafana).

  • Experience with analytics, semantic layers or text-to-SQL.

Our stack

Python, Kubernetes on AWS, NVIDIA GPUs, SIE (an open-source inference engine on SGLang), vLLM, PyTorch, gRPC, Langfuse, and our own evaluation harness.

 

#LI-Hybrid

Standard company text repeated across GoodData.AI's postings is omitted here.

Similar positions

Hippocratic AI
LLM Inference Systems Engineer
Hippocratic AI · Menlo Park, CA
GoodData.AI
(Sr.) AI Python Developer
GoodData.AI · Prague
Inferact
Member of Technical Staff, Inference
Inferact · San Francisco
Inferact