Generative AI on Kubernetes: Operationalizing Large Language Models
About this ebook
With actionable insights with real-world examples, readers will learn to tackle the opportunities and complexities of managing GenAI applications in production environments. Whether you're experimenting with large-scale language models or facing the nuances of AI deployment at scale, you'll uncover expertise you need to operationalize this exciting technology effectively.
- Learn how to deploy LLMs more efficiently with optimized inference runtimes
- Get hands-on with GPU scheduling, including hardware detection and multinode scaling
- Monitor and understand LLM-specific metrics like Time to First Token and token throughput
- Know when to fine-tune a model or when retrieval augmentation is the better choice
- Discover how to evaluate models with standardized benchmarks before committing GPU resources
- Learn to run agentic applications with secure tool integration, identity management, and persistent state
Discover more
About the author
Dr. Roland Huss is a seasoned software engineer with over 25 years of experience in the field. Currently working at Red Hat, he is the architect of OpenShift Serverless and a former member of the Knative TOC. Roland is a passionate Java and Golang coder and a sought-after speaker at tech conferences. An advocate of open source, he is an active contributor and enjoys growing chili peppers in his free time.
Daniele Zonca is a Senior Principal Software Engineer and Architect for model serving of Red Hat OpenShift AI, Red Hat's flagship AI product combining multiple stacks.
