The following is a collection of thoughts on recent febrile discussion of “prompt engineering” as a way to communicate intention to models.
My initial intuition here was that the rather excitable proponents of prompt engineering were making a category mistake (in thinking that all intentions could be reduced to language). On deeper consideration I don’t believe that intuition was correct – but nevertheless that “prompt” engineering, or replacement of formal with natural languages in technical contexts, is even less plausible.
By clawing the whole of human cognition into language it cheapens language’s transcendent aesthetic qualities
The problem
The whole question here is of engineering “prompts” to AI to get it to do what you want.
As mentioned my knowledge of linguists theory is pretty thin and this is a good excuse for me to learn a bit about it. I have a bit of an antipathy to philosophy of language – like I say, there is always someone trying to reduce the whole of thought to linguistics, and I find these kinds of totalizing systems disagree with my digestion.
The axe I have to grind here is: I am in love with language, but I believe there is ontology and phenomenology outside language which it cannot replace.
So my intuition was that converting formal instructions to natural language “prompts” was a way to allow linguistics to invade this space too, and it must be predicated on this strong idea that language = thought = action.
The tl;dr is that I don’t think my intuition is right here – but actually that the “prompt” instruction idea gets into even worse trouble, no matter what your metaphysical role of language is…
I found this book chapter Leibnitz’s Linguistic Realism (which is nominally about Leibnitz, but you can ignore those bits) to be really helpful here. It clarifies the terminology which can ironically be pretty confusing here, and introduces some of the neo-realist ideas of Meillassoux which sound pretty cool. The complete collection New Realism and Contemporary Philosophy is very much worth reading.
Linguistic Realism and Idealism
Broadly, understanding of the metaphysical role of language can be split into two traditions:
- The Aristotalean/Humean realist idea whereby thought and logic has an independent existence a priori to language, and that language merely provides an instrumental tool for their expression. The correlation of this with the things in themselves is a moot point – but the transcendental faculty that connects these with cognition may be reason (Kant). Natural language can be tricky and people can screw up (cheat of words, Hume) – but:
The challenges that language imposed on thought did not follow from some intrinsic linguistic nature of our ideas in virtue of which language would constitute the unsurpassable horizon of our minds; rather, the reasons for the so-called cheat of words lay in an improper understanding of the mechanism of signification.
- The linguistic turn originating from von Humboldt, Saussure, to the late Wittgenstein, Heidegger & Rorty. This position may be called linguist idealism and holds that there is no thought beyond language, and that language is the transcendental correlative that allows us “talk about” the things-in-themselves (if they even exist):
1. “that thought is essentially dependent on and bounded by language”; 2. “that concepts or meanings are constituted—not by referents, Platonic forms, mental ‘ideas,’ or whatnot, but—by word-usages”; 3. “that mankind exhibits deep linguistic and conceptual-intellectual diversities, especially between historical periods and cultures,” and consequently; 4. that “the investigation of the varying characters of people’s modes of thought and conceptualization should primarily take the form of an investigation of the varying characters of their languages.”
The late Wittgenstein very influentially settled the second view as dependent upon language games, whereby there are no fixed properties to language, but meaning is determined purely by use and the interaction of two people who eventually find an understanding that works for them by playing the language game.
Truth and Consequences
What does this mean for AI prompting? Well, let’s take either position:
- That there exist forms of thought independent of language, and certainly independent of natural language (in which we will write the AI prompts).
The skill is expressing these thoughts – or commands – in an intelligible way. Now, we can give our computer program commands (which are intended to manipulate bits of the things-in-themselves or language-independent-concepts) in two ways.
We can use a formalised programming language, which is a way to eliminate the ambiguity and vagueness of natural language. [Creating of a perfected language for expression of formal logical ideas is highly influenced by this realist position, and underlies a lot of the efforts of the logical positivists Carnap etc].
Or we can use natural language. The implication is clear here: natural language will be deeply inferior to a formalised language for expression of the pre-linguistic thoughts, because of the massively increased capacity for errors in expression (the cheat of words). The only advantage will be that it is easier for someone with less rigorous thought – as Hume puts it “puts the mind into error for reasons ranging from the inherent ‘imperfection of words’ to deliberate ‘abuse of words’ giving rise to the ‘learned Gibberish’ of Scholasticism”).
But that’s the last thing we want!
- That all thought is in fact expressed linguistically, and communication consists of navigation of a series of language games.
In which case, moving from an artificial to a natural language might not be so bad – they’re all just language games right?
But the problem with this is that the reliability – indeed the commensurability – of meanings between language systems completely falls away. The only reason we understand each other at all is because of a deep and broad web of context that we learn throughout our lives and engaging in these language games.
In natural language, this web is thoroughgoingly human. Misunderstanding the context and playing the game poorly will lead to fundamental errors in understanding. And what is going to be worst at understanding this context and playing the language game? A computer. An AI cannot play.
This view drags the whole of human cognition into the realm of language, and anything that doesn’t have human cognition (even if it were rational and aware of this context … it doesn’t have a skin) is going to be at a disadvantage.
So give up on trying to make computers compete – just use a human (there are enough of us around). The only real justification is economic, but justifying very poor quality on economic terms often turns out to be a false economy anyway.
So why use a programming/formal language in the case of (2) if it’s still only a language game? Well, it’s a heck of a lot simpler. Context still matters and you still have to play the game (tabs vs spaces in python!) – but the range and seriousness of mistakes you will likely make will be much more limited, just because it’s so much less expressive.
Language for Me
I’m more of a “realist” than an “idealist” here, thought I think both are important. A note on my priors here, because I’m very much a follower of Nietzsche in that I believe all philosophy is “unconscious biography” and reveals more about the writers “digestion” than anything else:
- I constitutionally recoil from totalising systems (I like wild plurality, exceptions, freak occurrences … this is Nature to me). That makes me very antipathetic to the linguists who (quelle surprise!) decide that everything turns out to be linguistics (#2). It reminds me of the insufferable people who claimed you didn’t need to study biology if you understood chemistry, or physics if you understood maths. A cynic might decide this has more to do with academic clout than anything else! By the same token, the implication from #1 is that natural languages are a bit … rubbish, and we should replace them with fake maths (hello Carnap!). Again, this seems ridiculously reductive and was an epic fail in practice.
- Part of this is I really mean it when I say “I am in love with language”. Language is an aesthetic, sensuous thing for me best expressed by the best literature (and I apologise right now for the violence I am doing that asethetic in my rather clunky writing). So I do tend to think that language has a life of its own, is transcendental in a way. But by clawing in the whole of human cognition into language it cheapens that somewhat. It is terrifically important to our cognition, but as a largely aesthetic phenomenon that may have an independent (read: realist) existence outside of language games. It won’t surprise anyone to see me mention Nabokov here.
- Of course there is something beyond language. Close your mouth and walk into the December sea, and tell me that’s a linguistic phenomenon. The power and beauty of Nature obliterate the petty anthropocentrism of strong linguistic idealism.
The Two Cultures
Sharing these thoughts with my colleagues, I was very interested to find an almost complete acceptance of the realist conception of language. This is almost totally at odds with the situation in the humanities and Continental Philosophy, where the idealistic strand has been – in more or less strong form – pretty dominant over the 20th century as I understand it.
They might not go as far as “thought is essentially dependent on and bounded by language” – which seems very strong to me – but even those who leaned heavily away from a mediated understanding of the world (like early Heidegger – who was all about being in the world, i.e. not being separated from it) seemed to come around to language as being the thing we really inhabited (later Heidegger later, the inconsistent).