Grumpy old Coders

David and Thomas dive into the inner workings of the Transformer architecture behind modern Large Language Models. They break down foundational concepts including vectors, matrices, function approximations, artificial neural networks, and backpropagation. Supported by a live Python code demonstration, they walk through tokenization, word embeddings, sinusoidal positional encodings, and Query-Key-Value attention dynamics. The episode wraps up with a discussion on scaling context windows, KV caching, and the philosophical debate around AI consciousness versus human self-awareness. Outline: 00:21 – Episode intro, podcast banter, & topic overview 02:37 – Mathematical foundations: Vectors, matrices, & functions 06:18 – Artificial Neural Networks: Layers, weights, & backpropagation 11:42 – ANN practical example: Image classification mechanics 14:51 – Tokenization & vocabulary constraints 18:40 – Word embeddings & GloVe vector spaces 21:00 – Positional encoding & sinusoidal matrices 27:00 – Self-attention components: Queries (Q), Keys (K), and Values (V) 31:37 – Live Python code walkthrough & script execution 37:25 – Attention matrix calculations & Softmax normalization 45:56 – Multi-head attention & multi-stage learning dynamics 49:44 – Large Language Model context window scaling 55:13 – KV caching & inference optimization 58:28 – AI consciousness debate 1:14:24 – Guardrails, sampling temperature, & LLM watermarking 1:20:51 – Closing thoughts and sign-off

David and Thomas recap their multi-country motorcycle journey through Germany, Austria, Slovenia, and Italy. They share key moments from the road, including navigating winding mountain serpentines. Along the way, they reflect on awkward luggage setups and unpredictable alpine weather. The episode wraps up with a discussion on the tech job market, the realities of AI "vibe coding" versus software engineering fundamentals, and preliminary planning for their next road trip. Outline: 00:20 – Episode intro, camera/mic troubleshooting, & tour premise 01:00 – Bike comparison: Suzuki SV650 vs. BMW R 1250 GS 03:11 – Lake Chiemsee stop, rain, and duck watching 06:08 – Crossing into Austria via Kufstein & taking biker photos 08:20 – Triassic Park landmark & tour snacking habits 10:32 – Riding through Slovenia: Serpentines 15:00 – Wörthersee stopover 20:04 – Tom's leg injury 24:25 – High-altitude dining & Slovenian road marking confusion 29:13 – Luggage setup challenges 35:00 – Nolan helmet visor breakdown 40:14 – Trattoria al Norgé: Homemade pasta & Italian hospitality 45:37 – Narrow mountain pass & wet bitumen hazards 52:30 – Staying in Innervillgraten & Innsbruck hospital visit 1:01:54 – Returning to Germany 1:11:00 – Tech industry state, LLMs, & "vibe coding" vs. engineering 1:19:10 – Future trip planning 1:22:50 – Closing thoughts and sign-off

David and Thomas dive into the critical role of context management and agentic memory with large language models. They explore the core mechanics of AI agents, breaking down the differences between low-level tools, Model Context Protocol (MCP) servers, and high-level skills, before tackling the token cost challenges of stateless LLMs. The hosts analyze human-inspired cognitive strategies like context "compaction" and background "dreaming". The episode closes with thoughts on codebase visualization using Graphify and Obsidian, and a humorous nod to Inception-style agent recursion. Outline: 00:21 – Heatwave talk & audio troubleshooting 07:16 – Chatbots vs. AI agents 12:20 – Defining tools, MCP, and skills 20:51 – Context boundaries & compaction 33:12 – Cognitive concepts and "dreaming" 40:14 – Exploring several tools 1:11:15 – Closing

After a year-long break, David and Thomas return to discuss CHAOSS (Community Health Analytics in Open Source Software), a Linux Foundation framework for evaluating open source project health. They explore metrics like compliance/security, governance, and community activity. The hosts emphasize that while metrics provide valuable data, community trust and authentic collaboration matter most. They also share motorcycle adventures—David's new Suzuki SV650 and their Black Forest tour. The episode closes with plans for more frequent releases and inviting guest speakers. Outline: 00:00 – Reunion & motorcycle talk 08:20 – What is CHAOSS? 15:26 – Metric groups (compliance, governance, community) 22:27 – Bus factor & sustainability 25:00 – GrimoireLab monitoring tool 28:14 – When to adopt CHAOSS 34:26 – Future plans & closing

In this episode, David and Thomas delve into the concept of Product-Led Growth (PLG). They discuss the differences between sales-led and product-led approaches, emphasizing the importance of user experience and collaboration across teams. The conversation highlights the benefits of PLG, particularly for startups, and the necessity of understanding customer needs and feedback to ensure product success. In this conversation, Thomas and David explore the intricacies of product-led growth (PLG) and the journey users take from being strangers to becoming champions of a product. Chapters: 00:00 Introduction to Product-Led Growth 02:55 Understanding Go-to-Market Strategies 10:05 The Role of User Experience in PLG 15:09 Benefits and Challenges of PLG 21:16 Collaboration Across Teams for Success 36:36 The Genesis of Open Source Products 37:58 Understanding Customer Acquisition Costs 38:55 The Product-Led Growth Framework 40:37 Transitioning from Strangers to Explorers 43:41 The Evaluation Stage: From Stranger to Explorer 46:51 Activation: The Moment of Value Realization 51:20 From Explorer to Beginner: The Activation Journey 53:11 The Importance of Documentation and Training 56:47 Onboarding Experiences and User Guidance 01:01:29 Driving Adoption and Cultivating Regular Users 01:06:41 The Role of Champions in Product Advocacy 01:08:00 The Role of Champions in Product Growth 01:10:11 Understanding the Flywheel Effect in SaaS 01:14:02 Brand Loyalty and User Experience 01:15:56 The Competitive Landscape of Cloud Services 01:19:02 Open Source vs. Proprietary Software Keywords: Product-Led Growth, Go-to-Market Strategy, User Experience, Sales, Marketing, SaaS, Customer Success, PLG Benefits, Collaboration, Startup, open source, product-led growth, customer acquisition, evaluation stage, activation, onboarding, user experience, documentation, champions, SaaS, Product Growth, SaaS, Champions, Flywheel Effect, Brand Loyalty, Cloud Services, Open Source, PLG, Software Licensing

In this episode, Thomas and David discuss the evolving landscape of e-mobility, focusing on electric vehicles (EVs) and their implications for consumers and the environment. They explore various types of electric vehicles, including hybrids and fully electric models, and delve into battery technology, efficiency, and the cost of ownership. The conversation also touches on the charging infrastructure, everyday usage experiences, and the longevity of EV batteries. Additionally, they examine the competitive landscape, particularly the rise of Chinese manufacturers in the EV market. Chapters: 00:00 Types of Electric Vehicles 06:09 Efficiency of Electric Vehicles 12:28 Comparing Electric and Traditional Vehicles 19:30 Pricing and Competition in the Market 25:17 Range and Charging Infrastructure 29:12 Benefits and Considerations of E-Mobility 34:32 Understanding Charging Options 36:26 Availability and Pricing of Charging Stations 44:02 Different Prices for Charging 46:38 Charging at Home and Battery Lifespan 53:18 Lifespan of Electric Car Batteries 57:30 Replacing the Battery 01:01:07 Efficiency and Maintenance of Electric Cars 01:03:23 Financial Considerations and Environmental Impact 01:13:28 Chinese Car Manufacturers in the Global Market Keywords: e-mobility, electric vehicles, battery technology, charging infrastructure, environmental impact, cost considerations, market trends, efficiency, hybrid vehicles, Chinese competition

In this episode, David and Thomas discuss their use of a large language model to build a chatbot based on documentation. They explain the concept of prompt engineering and how it can be used to guide the model's responses. They also introduce the concept of retrieval augmented generation (RAG). They also discuss the use of knowledge base articles and documentation to influence the behavior of LLMs. Additionally, they touch on the use of graph databases and embeddings to identify redundancies in documentation and improve search results. The conversation concludes with a discussion on the challenges and limitations of working with LLMs.

Merry Christmas (or just a happy holiday season)! Thomas and David discuss how technology is part of a wider definition of culture and how techies can contribute to technical education of kids.

Thomas and David talk about Technical Enablement, how David came to work on Technical Enablement projects, how it can help compensating the pains of scaling quickly, Learning Experience Design and more.

Thomas and David talk about how to produce a podcast. You will learn which equipment is needed, which software can be used and how the prep work and post-production looks like. Have fun!

Thomas and David talk about complexity classes, book recommendations, the meaning of the number 37, far too many books, and other life complexities.

The March episode is about 'Agile Software Development'. We explore the history, discuss why 'agile' makes sense, and give a brief introduction into several methodologies. We are also getting a bit grumpy about the fact that 'agile' is sometimes mixed up with 'quick and dirty'.

Thomas and David chat about cryptocurrencies, blockchain and NFT-s. Do you want to know why we are all living in a big bubble? Then listen to this episode ;-) .

Thomas and David talk about the year 2021. We had a lot of fun recording this year's episodes of the 'Grumpy old Coders', but there were indeed also deeper topics. We wish you a happy winter time and a great start to 2022!

In this episode our guest, Elena Kolevska, shares her experiences of being a women in technology and software development. We will learn how Elena came to the tech industry, why she considered coding as her profession, what organizations like 'Women in Tech' do for our work life, and more ... .

The 12th episode is called 'Just thingies'. Thomas and David are talking about a piece of art which got some human-machine interaction added with the help of the Arduino open-source electronic prototyping platform. Want to learn more about how to build such little thingies and how David (as a software guy) tended to ignore physics entirely, then have fun listening to this episode!

The 11th episode of the Grumpy old Coders podcast covers a more serious topic. Thomas and David discuss about the dark side of working in ever-changing and demanding work environment.

Data what? Thomas and David are discussing a bit about Data Engineering and Data Science. Not exactly being experts, they are trying to make sense out of it. Interested in what Data Engineering and Data Science actually is? Ever asked yourself what the differences between AI, ML and Deep Learning are? Then this episode is for you!

This is the ninth episode of the tech podcast ‘Grumpy old Coders’. David started to work a bit with Kotlin. So Thomas and David take a look at two of David’s toy projects which kicks off a vital discussion about coding with Kotlin and should give you a good first impression about this programming language. We are referring to some source code examples in this episode. They can be found under 'https://github.com/nosqlgeek/kEventLib' and 'https://github.com/nosqlgeek/kRTree'.

This is the eighth episode of the tech podcast ‘Grumpy old Coders’. We have two special guests this time. Christoph and Martin are the ‘The Linux in Laws’ and that said they love Open Source Software. We are talking about communism, OSS projects, the OSS business models, licenses, and more. Full disclosure: This episode is very entertaining.