Senior Software Engineer
Job description
About the role & what you’ll do
Run a fleet of coding agents to build inference infrastructure in Rust and Python to power modern AI applications.
You'll work with high-performance services for running AI models, metering tokens, processing images, audio and video, used by real teams in our open source K8s inference cluster and our serverless API product.
This role is hands-on and end-to-end: from designing clean APIs to making them fast, observable and easy to operate.
Tech stack
Python: Python 3.12+, FastAPI, Pydantic, asyncio, uv, pytest, ruff, mypy
Ecosystem: Kafka, VDBs (Qdrant, Redis, TopK), OpenTelemetry (Loki, Grafana, Tempo, Mimir, Prometheus), Infinity, Locust
Infrastructure: GCP (CloudRun, GKE), Kubernetes, Docker, Terraform, Github Actions, mise, Monorepo
Data: Ray, Polars, Pandas
Rust: Hands-on experience debugging, troubleshooting, and optimizing Rust code.
AWS (EKS): Experience with AWS, particularly Elastic Kubernetes Service (EKS).
GCP: Experience with Google Cloud Platform
Soft skills
Strong ownership: able to take a problem end-to-end with minimal supervision.
Clear, concise communication in a distributed international team.
Reliable execution and ability to deliver within agreed timelines.