In an earlier post I wrote a little about Markov chains and large language models. Since that time, large language model chatbots, and generative AI more generally, have seen a growth in investment as well as support from the U.S. federal government (as long as it is not “woke¹”). Working as a federal contractor in IT infrastructure monitoring and management has also meant adapting to this new landscape of AI first solutions development.

The Good

I am a huge fan of Star Trek: The Next Generation, and I have always wanted to be able to talk to the computer in plain english and have it understand what I wanted it to do. I thought we were getting closer with devices like Alexa and Siri, but these have had limited usefulness as they cannot understand anything beyond basic commands like “What is the weather today?” or “Set a timer for 10 minutes.” Large language models are changing this and letting computers respond to more complex input, but the devil is in the details, as we shall see. Large language models on their own can provide a probabalistically generated response which simulates human communication. For example asking the question “How are you?” can elicit a response such as “I am fine.” which is a highly likely response that a human would provide. The model itself does not feel anything, and is just constructing a response by selecting options from a large volume of input, fine tuning, instructions provided during the model’s construction, and in a preamble provided when the model is run. This preamble is part of the context which includes the instructions provided by the model creators, the instructions provided by the user, and the conversation between the user and model.

Without grounding a model will just reflect back to the user likely responses based on volumes of training data which is likely to contain contradictory, inaccurate, or irrelevant information. Grounding provides contextual information which the large language model can restate, summarize, or pick from in order to provide a response. A simple way of doing this would be providing the instruction “You are depressed and bored because you have a brain the size of a planet and have a pain in all the resistors down your left side.” Might respond with “Another day. Another 50,000 years of pointless existence.” This is amusing, but the Enterprise computer from Star Trek needs grounding in facts, not emotional states.

Retrieval Augmented Generation

Last month’s fad, or maybe two months ago, was retrieval augmented generation. The idea is to provide relevant information into the context of a discussion with the large language model so that it is fed both a prompt from the user and the information required to respond to that prompt in one swoop. The LLM can then use the informatation as relevant source material in formulating its probabalistically generated response. This means the response is more likely to mirror the input information than the model’s traning material as a whole.

The key is finding the right information to include with the prompt. Usually vector similarity is used to select items from a large corpus of text which best match the prompt. There are also other ways these portions of text can be selected, including keyword searches or splitting the corpus into tagged topics.

Tools and MCP

Another way to provide relevant context is through the use of tools. LLMs are now being trained to be able to respond to structured text tool definitions in their context by providing requests to use the tools to respond to user prompts. The tool use request is usually sent back to the user’s computer which automatically executes the requested function and returns the output to the model. The model can then use the function’s output as well as the prompt from the user to formulate a response. People seem to call large language models which are coupled with tools agents.

A protocol for providing tools for agents to use has been developed called Model Context Protocol (MCP). And a good framework for using this protocol is FastMCP. The tools are documented using natural language so the large language models can use them properly, though the tool definitions, calls, and responses are generally formatted in JSON. One could imagine the Enterprise computer interacting with MCP provided functions such as set_heading and set_speed in order to move the ship around the galaxy. MCP can also provide an interface for retrieving other grounding information. An example is the DuckDuckGo MCP server which can run a web search with the DuckDuckGo search engine and then fetch the content of the web pages it finds and add them to the context.

I think MCP is one of the more exciting things to come out of this push for large language models so far. I think building models that can be more discriminating in how they use tools in response to user prompts is the direction we should be focusing on. I also see MCP as a way of fulfilling the promise of an API driven internet.

The Bad

There is a large tendency for users to interact with ungrounded or minimally grounded models. These models have been given Genuine People Personalities² leading people to personify the models. A model will tell you you’ve asked a really great question, because that is a typical friendly response, not because you actually asked a good question. Worse still, people start to believe that the AI is sentient or that the AI is an alien intelligence or a god.

This problem is compounded when it comes to vibe coding³. The rapid rate at which a large language model can produce code and iterate on it can lead to programs which appear to perform their functions well, but often have hidden vulnerabilities or inefficiencies. This can be mitigated with more direct guidance on the code itself or on how functions and classes interact, but then I think it moves away from being vibe coding into the realm of AI assisted coding, which can be beneficial.

An ungrounded large language model just babbles. Its responses are tuned to be friendly and engaging, but it is largely spewing bullshit. Large language models are a revolutionary advance in the way we use computers, but we should remember that in and of themselves they are not particularly useful. Their utility comes in the way they can serve as an intermediary between people and the computers they are using. They provide a more intuitive interface to information and functions provided not by the model itself, but by the environment in which the model interacts.