So will LLMs ultimately become realists, or nominalists?

LLMs can be whatever labels people choose to attribute to the system executing the instructions to generate "answers". It is fundamentally a category error to attribute any meaning to whatever arithmetic operations the hardware is executing because neither the hardware in the data center nor the software have any personal characteristics other than what people erroneously attribute to them because of confused ontologies and metaphysics about computers and software.

At which point would such attributions be accurate? Humans are fundamentally just computers too. A different medium, but still transforming electrical signals.

Extremely weird to me when people compare themselves to computers. What is that philosophical stance called and do you have any references for long form writing which makes the case for why people are "just" computers?

It's definitely a philosophical position that exists beyond a random HN poster. See for example computational theory of mind https://plato.stanford.edu/entries/computational-mind/

I am familiar with the computational theory of cognition. What I wanted to know was whether there were any people who actually claimed their thinking is nothing more than programmed computation. I am very curious to know if they have mapped out the instruction set for their mind along the lines of something like the SKI combinators.

A mental instruction set would be extremely interesting. Unfortunately, nobody has that level of understanding of brain processes (and it might be quite difficult to formulate in such a linear way since the underlying mechanism is so very parallel), but the idea that human cognition is computable falls pretty naturally out of the idea that nature is computable which I think is a common position (sometimes called the Church Turing Deutsch principle).

Yes, I understand why some scientists claim that nature is "just" some computer but no one still has given an answer to my very basic question: what is the instruction set that the people who claim they are computers are using to think? Surely there must be one if they are nothing more than programmable computers as they claim.

Why do you think there would need to be an instruction set? And why do you think we'd need to know of one to conclude we're all computers?

Just trying to figure out how rigorously people have thought about this. A computer with an undefined instruction set seems somewhat useless as a computer.

Whether or not we can discern an instruction set is entirely orthogonal to whether or not something can compute.

Oh I see, so it's magic.

If you don't know how something works, do you assume it is magic? Why? It's a wildly irrational assumption to assume that it is magic rather than assume absent evidence to the contrary that it works according to known physics.

Well, that's essentially a logical tautology. Everything works according to the known laws of physics but it's certainly true that everything must also work with unknown laws of physics because of basic human ignorance.

No it is not. The argument is that absent any evidence of the existence of such unknown physics happening in the brain the most logical assumption is to assume there isn't any, rather that presuming the existence of something we've never observed any hint of.

Unknown physics is happening all the time almost by definition but as I said in another thread, good luck with your computations.

Rule 110 doesn't have a clearly defined instruction set but is known to be Turing complete.

Rule 110 can be specified with a rewrite system, also known as cellular automata: https://arxiv.org/abs/0906.3248. Cellular automatons have a correspondence with contextual grammars: https://www.cis.upenn.edu/~cis5110/notes/tcbook-lang.pdf. Each is equivalent to a Turing machine, another way of saying that there is a program for it which can be specified on a Turing machine with the usual Turing machine instruction set for writing, reading, and erasing binary digits on a tape. This usual program can then be "compiled" into a rewrite system corresponding to the instruction set for rule 110.

The reason rule 110 is said to be Turing complete is because someone went through the trouble of specifying an instruction set for rule 110 so that other people could verify that it would be possible to write programs with it. This is not the case for the people who claim that they are computers. They always leave the instruction set undefined which makes their claims hard to believe.

I personally have no problem with people who think they're computers but if they're not programmable then I'm not sure what the point would be of calling themselves computers.

> This is not the case for the people who claim that they are computers. They always leave the instruction set undefined which makes their claims hard to believe.

What is your alternative? Can you explain to us how the brain could possibly do something - down to the atomic level - that would allow it to do something that not only is not possible to simulate, but that also does not still constitute computation?

We don't even have language for talking about such "operations" that are so different from all forms of computation that it is not just another form of computation.

Just try to describe one such hypothetical state change that can not be reproduced with a Turing computable function.

At the same time, your insistence on "instruction sets" is meaningless. An "instruction set" the way we tend to consider them is not necessary to parameterize a function. A neural network with input/output used to provide the "tape" can trivially be made Turing complete. If you consider the weights or connections of the network an instruction set, then there you go - that we don't know how to measure and extract all the details of the neural network of a brain does not mean we can't observe their presence. And it also does not mean we haven't done a vast amount of measurements without observing any hint of unknown physics affecting state transitions.

To simplify it: Even a simple mechanical thermostat is parameterized - the dial provides "an instruction set" in the form of an ability to set a single threshold that alters the behaviour of the function computed.

But if you expect something that looks like what we typically talk about when we talk about an instruction set, then that is a very limiting view of computation, and one I've already pointed out to you is just one part of the multiple types of computational devices we've built. Including heavily parameterisable ones.

I expect claims to be backed by evidence that is consistent with our current state of knowledge. I have seen no such evidence so that's why I asked for references. In any case, this discussion has run its course, best of luck to you and your future computations.

And wouldn't that language need to be able to account for different physiological states? Thinking when one is hungry or sleepy is quite different than thinking when one is well-fed or fully rested.

Yes. To validate the claim would require not only a formal instruction set but also the code to account for all sorts of cognitive states and processes. I'm not ruling out that some people are indeed programmable computers but I would like to see some actual evidence presented by the people who make these claims about themselves.

For us to not be would require brains to be able to compute functions that can not be computed by an artificial computer. That would seem to be an extraordinary suggestion given we have no indication of unusual physics in the brain.

You'll have to define your terms first. Physicists now believe there is such a thing as dark matter and that there are objects so massive that no amount of observation can ever make sense of how massive they are because it is impossible to model it mathematically.

I am not the one making any extraordinary claims. Physicists themselves admit there are aspects of reality with no computational basis.

These terms have well understood meanings, and dark matter or black holes are entirely irrelevant to what I said.

For brains not to be computers would mean the physical Church-Turing thesis is invalid, and proof of that would be extraordinary enough to be Nobel Prize material.

Whether something is physical or not is orthognal to whether it computes or not. You're the one who brought up physics so that's why I showed why your logic was invalid. My contention was that calling something a computer without providing an instruction set was nonsensical and I wanted to know if someone had actually spent the time to rigorously think about what a computer without an instruction set would entail. So far it seems like no one has spent any time really thinking about it but that's probably for the best anyway. I'm sure an LLM will eventually figure out an instruction set for programming people and then take over the world.

The idea that a discernable instruction set is needed for something to compute suggests you don't understand how fundamental computation is.

We have built computers without instruction sets, e.g. in the form of mechanical devices to carry out calculations. Fairly complex computations were done that way before general purpose programmable computers, but even many early programmable computers had no fixed instruction set.

There is a rich history of computation through wiring up calculations without any instructions involved. And for that matter of mechanical computation.

Here's an outline for a simple computational device:

A bucket.

Pour predefined quantities of water into a bucket, and you can compute a threshold. Use buckets of different size and overflows, and you can separate a numeral into binary digits. Drain them into containers of different sizes and you can carry out logical operations. (Actual computation has been done this way - fluidics is one way, which dates back to the Tesla valve in 1920).

Every physical interaction is computation, whether or not it is useful computation. The notion computation requires an instruction set is confusing a very limited notion of classical programmable computers with the general concept of computation.

It is also a notion contradicted by the history of computation, which is full of computation without an instruction set, and of implementing computers with instruction sets in terms of computations of fixed function devices without one.

E.g. it's not turtles all the way down - that instruction set runs on a CPU that ultimately is built of fixed function logic.

Instruction sets are an optional high level abstraction.

Occam’s razor - ‘just computers’ is enough to explain the mind, so why bother with more?

Steam engines and gears were used for explaining the mind as well but those turned out to be incorrect metaphors.

Steam engines and gears are a specific physical manifestation of computation. Computation does not have a single, specific physical manifestation - it can, and has, been done with organic matter, electronics, gears, pipes of water, light.

Per the Church-Turing thesis these can all compute the same set of functions, and unless you can demonstrate that brains and only brains can evoke unknown physics that allows brains to compute a set of functions that can not be computed by other means, the most logical assumption is that it holds, including for brains.

Especially given how much we measure brains without seeing any signs of unusual physics.

I think I understand. So what you're saying is that every function that can be implemented with computers must be computable. Your claim is that the brain is actually a computable function, can you tell me which one it is using your favorite version of a Turing complete instruction set? Or maybe I misunderstood and what you're saying is that the brain is not the function but what it does is compute a specific function called your mind in some unknown instruction set?

I'm saying that per the physical Church-Turing thesis, any function that is computable by ordinary physical means are Turing computable, and we have no evidence that even hints at the physical Church-Turing thesis not holding.

For it not to hold, there would need to be something unique about the physics of a brain that allows it to compute a class of functions which are inherently impossible to compute by other means. That'd imply entirely new/unknown physics that we're somehow not seeing any hints of.

> Your claim is that the brain is actually a computable function, can you tell me which one it is using your favorite version of a Turing complete instruction set?

No, my claim is that absent evidence of unknown physics or another way of disproving the physical Church-Turing thesis, the rational assumption is that the brain follows the same laws of physics as everything else, and so is limited to computation that is equivalent in power to Turing computable functions, just like everything else we know of.

For the brain not to be a computer would imply "magic" - not just that we don't know how the brain works, but for the brain to work in ways inconsistent with all known physics, and inconsistent in ways impossible to simulate with Turing computable functions. No sign of any such unknown physics happening in the brain has ever been recorded.

The Church-Turing thesis is a meta-mathematical statement and it neither has a proof nor disproof. In any case, this discussion has run its course.

Does your computer supply different answers if it is hungry, or tired?

It would if it was programmed to. Modern OSs certainly takes into account environmental factors.

> exploiting the fact that factual knowledge in an LLMs has generally been shown to be localized to particular transformer layers

This is surprising

Why is this surprising?

It makes sense that “facts” exist in earlier layers, then these become more abstract as you become deeper.

This reminds me of residual connections from CNNs and vision.

Interesting. I hadn't considered that before but makes sense.

Just call it correctness. Hallucination as an alternative to incorrect is fine for marketing I guess but factuality is especially awkward besides being pretty Orwellian.

Orwellian? Maybe, but in the same way we teach our children what is true and how to determine what is true.

We want to raise these pseudo-humans to be useful upstanding members of society. Knowing fact from opinion, knowing right from wrong, knowing what is real and what is imagination, are important for any intelligence. Otherwise our silicon-children will grow up to be dumb, harmful, or both, while being trillions in number.

And that goes back to the question I asked above: are you talking about "what is real" from a philosophical "realism" point of view, or from a philosophical "nominalism" point of view?

Realism posits that objects have intrinsic meaning that we apprehend through attention.

Nominalism posits that we cannot apprehend reality directly, but only through our minds and through language. That we only have a second-order experience of reality. Therefore, all language only has meaning because of consensus, so if we change the consensus of meaning around language, we are actually changing reality because reality is mediated through language.

These are obviously very compressed definitions of these views, but the question remains.

This conversation about AI "hallucinations" seem to point at this question. "We want AI to say true things." True to what? True to reality? Or true to language? When we ask AI a question, AI only knows how to answer the question using grammar that is probabilistically the most likely. That has no tie to "reality", but as soon as you start asking "well then what is reality that we want to map AI to?" the question gets quite slippery.

My contention is that AI, as its responses are curated by people, will only reflect the idiosyncratic worldviews of those doing the pruning.

I prefer the word “confabulate”.

When humans fill in knowledge they don’t actually have, but think we do, we call it confabulating.

We all confabulate at a low level, because it is intrinsic to how we store and recall memories. Our memories are compressed summaries whose details we fill out with defaults and guesswork subconsciously as we “remember”. This is why memories are rarely perfectly accurate.

Some people confabulate more than others. And we all confabulate to a greater or lesser degree based on variable circumstances, emotions, motivation, fatigue and other mental states.

“Hallucinations” on the other hand are what happens when our sensory processing becomes unmoored from actual memory or real sensory constraints. The brain creates an interpretation based on internal feedback without normal correction, drifting into false sensory experiences that are not actually reflective of reality.

Dreams are a natural form of this where nothing we are experiencing is actually really happening.

One is false memories, the other is false experiences.

The way I see it hallucination and factual error are particular cases of incorrect answer.