First Boot, First Inference

In this episode of The IT Guy Show, Eric sits down with Damen Knight, CIQ Sr. Principal Automation Engineer, to talk about something every AI team knows but nobody budgets for: the hidden cost of configuring Linux for GPU workloads.
Together they dig into why general-purpose Linux distributions were never really built for AI, what actually happens when you spend 30 to 60 minutes per node doing manual CUDA setup, and why a "pre-installed" stack and a "validated" stack are two very different things. From driver conflicts and framework dependency failures to the CIQ Linux Kernel (CLK) and NVIDIA authorization, this episode covers the infrastructure decisions that determine whether your AI program ships fast or gets stuck in configuration hell.
Whether you're an ML engineer tired of fighting CUDA every time a kernel update drops, a sysadmin managing a growing GPU fleet, or a tech leader wondering why production AI is harder than it should be, this one is for you.

Welcome to The IT Guy Show, your go-to destination for all things tech with Eric, your friendly neighborhood IT guy! Join Eric as he shares his wealth of knowledge, insights, and experiences gained from years of working in the IT industry. From troubleshooting common tech issues to exploring the latest trends and innovations, each episode is packed with practical tips, skilled advice, and engaging discussions.

Connect with me! https://linktr.ee/itguyeric
CIQ Press Release: https://ciq.com/blog/rlc-pro-ai-is-here/
RLC Pro AI: https://ciq.com/products/rocky-linux/pro/ai/
NVIDIA CUDA Toolkit: https://developer.nvidia.com/cuda-toolkit

Chapters:
00:00 Stream start
00:50 Introduction
02:43 Sponsor: CIQ
04:08 AI at CIQ
06:02 State of AI
19:20 General-purpose limitations
31:37 RLC Pro AI
43:53 What's next in AI?
52:47 Wrap up

Graphics and bumpers by Free Hive Agency
Photo by Igor Omilaev on Unsplash