AI Is Not a Product Feature
I recently saw on HN an essay titled AI Horseless Carriages by Pete Koomen. In this essay, the author explains why integrating generative AI into existing products doesn’t work.
Since the democratization of LLMs, the first temptation has been to integrate them into our everyday software. We add an LLM to our email applications to help us write content, or to our document management software to summarize documents. We add the LLM as an additional layer on top of our existing tools. An “AI” button in our apps for “augmented production” without thinking further about the features. We see this in Gmail (per Koomen’s example), in the Office 365 suite, in content editing software, on social networks, or in messaging apps (WhatsApp). AI everywhere to “enrich” content and delegate writing to LLMs. Once the “wow” effect of the first few minutes of use wears off, you quickly realize the mediocrity of the results generated by these LLMs.
This integration approach is not the best because AI should not be considered as an option of a “product.” It is not the air conditioning or the metallic paint of a car. We need, I believe, to adapt our tools, our methods, our habits to AI and not the other way around.
The problem is not AI. The problem is our inability to imagine software designed for AI, because generative AI represents a real transformation.
I had already written about this aspect here and I think it is fundamental to properly understand what generative AI and LLMs do in order not to be disappointed by the results. Some people think AI generates nonsense. This assessment results from the intentions we ascribe to LLMs (see my note “generative AI of nonsense”). We must be wary of sales pitches and promises made left and right. If you want to dig deeper into the subject, go read Ethan Mollick and particularly the concept of “Jagged Frontier” that he developed here and there.
Today, I have the impression that many people use LLMs a bit like the man sitting on the hood of his car below. We add a new layer on top of something existing, hoping it works “better” or expecting insane productivity gains. We are in a kind of “wishful thinking.” This obviously creates disappointments.
When I need to explain what generative AI does not do, I use the image of a nail you want to drive in with a saw. It is technically possible but it is not made for that. We often say that LLMs should not be used as a search engine because that is not what they are made for. We are in the same line of thinking here.
Also, we often read that LLMs can automate repetitive tasks. In fact, if you want to automate repetitive tasks, the LLM may not be the best solution. Task sequencing has existed for years, and we did not wait for LLMs to use tools like Zapier, Make, or N8N. It is therefore somewhat reductive to say that generative AI can automate repetitive tasks (I also keep wondering how to create an automation in a conversational interface, by the way, but that is another topic :)) and moreover, the result risks being disappointing because generative AI is a tool that generates content, not automations.
If you have read this far, you must be thinking “okay, great, but what do we do?”
In my view, the LLM is a contextual generation tool: the user needs to customize their use of the LLM to extract value from it and move beyond generic use cases. And this is where Pete Koomen’s essay is brilliant, because it concretely and pragmatically illustrates how to customize the LLM by recalling the distinction between “system prompt” and “user prompt.”
The “system prompt” is the generic framework. The “user prompt” is the user input, their question, their request.
In the context of a conversational agent like ChatGPT or Claude Sonnet, the “system prompt” applies to all conversations. Today, the AI tools offered as overlays on existing software are configured generically, in a standard way. A bit like if your car radio had only one FM frequency. It will suit you for a while, but if you are a classical music enthusiast, you will quickly turn off your radio if the only available channel is a 24-hour news station.
The solution is therefore to customize the “system prompt.” If you read the “system prompt” of Claude Sonnet, you will understand that the instructions given to the system are very general, and that this generic level cannot satisfy specific uses such as those we integrate into our professional practices, for example.
To benefit from generative AI, you therefore need to 1) agree on what it is capable of doing and 2) customize the AI so that it “fits” your use case.