I think the Wikipedia definition of thought is quite good.
However, I have a feeling whatever definition I came up with, you’d just claim LLMs fit into it because their output is sometimes somewhat coherent.
You can claim that technically LLMs “think” because the output text sometimes contains conclusions, and sometimes they’re even rational, even though the LLMs still struggle with counting Rs in “strawberry”.
I find that disingenuous because it implies that the LLM is in any way aware of anything, that it can passively form ideas.
Most importantly, it implies that you can trust it for even basic reasoning. That you can trust the plagiarism machine that tells you that you should put glue on your pizza, eat rocks and walk to the car wash instead of driving, or that you will be able to trust it at some point in the future.
Whatever definition of thinking we use, it should include a simple rule - that the allegedly thinking entity should demonstrate that intelligence by being able to reliably answer simple queries correctly. Humans, by and large, can do that. LLMs fail at it miserably. If the LLMs were truly thinking, that should be shocking. Understanding the underlying technology - and that it is not truly reasoning - makes it obvious and expected.
Even OpenAI admitted hallucinations are an unfixable mathematical inevitability - something you handwaved as a matter of time to fix. No, the fact that humans can have hallucinations is not comparable.
Cognitive process in which the mind considers, creates, or manipulates ideas, representations, or information.
Yes, you are correct - that is exactly what LLM do.
You make it sound as if 99% of the time the output of an LLM is gibberish, and only 1% it is coherent, but in my personal experience it is the exact opposite ratio. However, I can’t account for your experiences, as I am very aware that both expectation and perception are the basis for reality, and yours are obviously tuned negatively. You see what you expect to see, and if others have drastically different experiences that is something you cannot fathom as you have only your own to draw from anecdotally.
An LLM certainly could passively form ideas, if it was allowed to passively execute and output it’s thoughts to a console. That is not typically how they are used, though, and have been trained to output one response for one input. The capability is there, though.
I trust AI far more than I do a random person. They have access to far more information, and are more likely to be correct about any particular question asked. If you need a specialist, a human will usually be superior to the AI if you seek one out, but the next best thing is an AI - not a random person.
If you’re going to claim that hallucinations are an inevitably, please cite your source. My sincere understanding is that hallucinations occur when an LLM lacks context, and rather than seeking it, it takes a shortcut by using a random value for the missing information. This is due to a combination of a limited context window, because of memory constraints, and performance considerations as the operators want faster responses over thorough, comprehensive thinking as it aligns with cost savings concerns from the business perspective.
The advent of MoE (Mixture of Experts) has cut down on hallucinations considerably, by reducing the footprint of the thinking model to only that which is relevant to the prompt, which frees up memory that can be allocated to context. Similar incremental improvements will inevitably make hallucinations a funny story from the past, which I believe is the inevitability.
If you have some mathematical formula that proves hallucinations are unsolvable, I’d be happy to read it and reflect upon it. However, I strongly suspect that any formula you produce is only relevant to the technology of LLMs as they operate today, and that such problems are indeed solvable, and in no way inevitable.
Your derisive pet names for the technology make it clear you’ve already made up your mind, though, so don’t feel obligated to continue to argue the case that no one will ever solve what you believe is unsolvable.
I trust AI far more than I do a random person. They have access to far more information, and are more likely to be correct about any particular question asked.
That is a terrifying stance. And, frankly, embarrassing.
OpenAI, the creator of ChatGPT, acknowledged in its own research that large language models will always produce hallucinations due to fundamental mathematical constraints that cannot be solved through better engineering, marking a significant admission from one of the AI industry’s leading companies.
[…] The research proposed “explicit confidence targets” as a solution, but acknowledged that fundamental mathematical constraints meant complete elimination of hallucinations remained impossible.
I, of course, disagree. Humans on the whole are ignorant and the average person is insufferably ill-informed. You are proving to be the exception. In general however, people suck at answering questions on subjects in which they’re not experts.
I have read that study, but I believe it to be flawed and I’m not alone. Please read the counter argument that hallucination is a structural phenomenon of estimation itself.
This reframes hallucination as structural misalignment between loss minimization and human-acceptable outputs, and hence estimation errors induced by miscalibration.
In other words, it is a problem that can be solved.
Our work demonstrates that targeted, high-quality SFT data teaching meta-cognitive skills can effectively reduce hallucination without preference/RL pipelines, providing mechanistic insights and a practical path toward more reliable AI systems.
Well, I suppose we can at least agree to disagree.
I have seen so much incoherent but confident nonsense produced by LLMs (mainly by frontier models trying to do even basic software development) that I would not be able to say in good conscience that thought was involved. Junior developers would have done better. The experience definitely fits the behavior of a word predictor, though.
Having seen what LLMs claim about software development, my stance is that absolutely no one should trust at face value what these models output. They’re Dunning-Kruger machines.
As for producing new ideas, these models are as creative as a random number generator. Coincidentally, that’s what is responsible for faking their creativity (the “temperature” parameter).
I guess that’s all I feel like saying in this particular thread.
At the company where I’ve been the lead developer for fifteen years, sentiment is split down the middle - half think as I do, half think as you do. In (nearly) every instance where one of the opposing developers shows me nonsense, it’s been easy to identify the cause: a lazy prompt with insufficient context. Garbage in, garbage out.
Having seen the results of the US elections, I don’t think anyone should trust humans. Yet here we are.
As for temperature, yes, I’m aware of the parameter. The human equivalent would be genetic mutation, although we can’t alter ours on the fly.
Thank you for the civil discussion. Until next we butt heads in the threads 👋
I think the Wikipedia definition of thought is quite good.
However, I have a feeling whatever definition I came up with, you’d just claim LLMs fit into it because their output is sometimes somewhat coherent.
You can claim that technically LLMs “think” because the output text sometimes contains conclusions, and sometimes they’re even rational, even though the LLMs still struggle with counting Rs in “strawberry”.
I find that disingenuous because it implies that the LLM is in any way aware of anything, that it can passively form ideas.
Most importantly, it implies that you can trust it for even basic reasoning. That you can trust the plagiarism machine that tells you that you should put glue on your pizza, eat rocks and walk to the car wash instead of driving, or that you will be able to trust it at some point in the future.
Whatever definition of thinking we use, it should include a simple rule - that the allegedly thinking entity should demonstrate that intelligence by being able to reliably answer simple queries correctly. Humans, by and large, can do that. LLMs fail at it miserably. If the LLMs were truly thinking, that should be shocking. Understanding the underlying technology - and that it is not truly reasoning - makes it obvious and expected.
Even OpenAI admitted hallucinations are an unfixable mathematical inevitability - something you handwaved as a matter of time to fix. No, the fact that humans can have hallucinations is not comparable.
Wikipedias definition:
Yes, you are correct - that is exactly what LLM do.
You make it sound as if 99% of the time the output of an LLM is gibberish, and only 1% it is coherent, but in my personal experience it is the exact opposite ratio. However, I can’t account for your experiences, as I am very aware that both expectation and perception are the basis for reality, and yours are obviously tuned negatively. You see what you expect to see, and if others have drastically different experiences that is something you cannot fathom as you have only your own to draw from anecdotally.
An LLM certainly could passively form ideas, if it was allowed to passively execute and output it’s thoughts to a console. That is not typically how they are used, though, and have been trained to output one response for one input. The capability is there, though.
I trust AI far more than I do a random person. They have access to far more information, and are more likely to be correct about any particular question asked. If you need a specialist, a human will usually be superior to the AI if you seek one out, but the next best thing is an AI - not a random person.
If you’re going to claim that hallucinations are an inevitably, please cite your source. My sincere understanding is that hallucinations occur when an LLM lacks context, and rather than seeking it, it takes a shortcut by using a random value for the missing information. This is due to a combination of a limited context window, because of memory constraints, and performance considerations as the operators want faster responses over thorough, comprehensive thinking as it aligns with cost savings concerns from the business perspective.
The advent of MoE (Mixture of Experts) has cut down on hallucinations considerably, by reducing the footprint of the thinking model to only that which is relevant to the prompt, which frees up memory that can be allocated to context. Similar incremental improvements will inevitably make hallucinations a funny story from the past, which I believe is the inevitability.
If you have some mathematical formula that proves hallucinations are unsolvable, I’d be happy to read it and reflect upon it. However, I strongly suspect that any formula you produce is only relevant to the technology of LLMs as they operate today, and that such problems are indeed solvable, and in no way inevitable.
Your derisive pet names for the technology make it clear you’ve already made up your mind, though, so don’t feel obligated to continue to argue the case that no one will ever solve what you believe is unsolvable.
That is a terrifying stance. And, frankly, embarrassing.
“OpenAI admits AI hallucinations are mathematically inevitable, not just engineering flaws”: https://www.computerworld.com/article/4059383/openai-admits-ai-hallucinations-are-mathematically-inevitable-not-just-engineering-flaws.html
I, of course, disagree. Humans on the whole are ignorant and the average person is insufferably ill-informed. You are proving to be the exception. In general however, people suck at answering questions on subjects in which they’re not experts.
I have read that study, but I believe it to be flawed and I’m not alone. Please read the counter argument that hallucination is a structural phenomenon of estimation itself.
In other words, it is a problem that can be solved.
https://arxiv.org/abs/2509.21473?hl=en-US
Edit: Here’s an approach to solving it, in fact:
Inducing Epistemological Humility in Large Language Models https://arxiv.org/abs/2603.17504?hl=en-US
Well, I suppose we can at least agree to disagree.
I have seen so much incoherent but confident nonsense produced by LLMs (mainly by frontier models trying to do even basic software development) that I would not be able to say in good conscience that thought was involved. Junior developers would have done better. The experience definitely fits the behavior of a word predictor, though.
Having seen what LLMs claim about software development, my stance is that absolutely no one should trust at face value what these models output. They’re Dunning-Kruger machines.
As for producing new ideas, these models are as creative as a random number generator. Coincidentally, that’s what is responsible for faking their creativity (the “temperature” parameter).
I guess that’s all I feel like saying in this particular thread.
That we can.
At the company where I’ve been the lead developer for fifteen years, sentiment is split down the middle - half think as I do, half think as you do. In (nearly) every instance where one of the opposing developers shows me nonsense, it’s been easy to identify the cause: a lazy prompt with insufficient context. Garbage in, garbage out.
Having seen the results of the US elections, I don’t think anyone should trust humans. Yet here we are.
As for temperature, yes, I’m aware of the parameter. The human equivalent would be genetic mutation, although we can’t alter ours on the fly.
Thank you for the civil discussion. Until next we butt heads in the threads 👋