this post was submitted on 17 Nov 2025
28 points (100.0% liked)
SneerClub
1203 readers
15 users here now
Hurling ordure at the TREACLES, especially those closely related to LessWrong.
AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)
This is sneer club, not debate club. Unless it's amusing debate.
[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]
See our twin at Reddit
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Even if true, why couldn't the electrochemical processes be simulated too? I don't think it's necessary to strictly and completely reproduce a biological brain to produce thought in a computer, but even if it is, it's "just" a matter of scale. If you can increase the fidelity of the simulation with effectively infinite computing power, what would it be missing? It'd have to be something that can't be predicted, can't even have its unpredictability described with an equation (I don't know what any coin flip will turn up as, but I do know how to write a program that produces indistinguishable results from a real coin for a simulation), so it's just changing all the time and follows no rules whatsoever, but also you can't just write a program that does its own "random crap that can't be predicted" simulation because the real one is somehow also so precise that it's the only thing that makes consciousness work and a mechanical one isn't good enough?
I want to take this argument of efficiency in a different direction. First, two key observations: the system doing the simulation will never be as efficient as the system being modeled. Second a conscious system is aware of it's own efficiency. This means even if you simulate a whole human body to create consciousness it will not have the same quality. It will either be aware of all the extra resources required to create "self" or fed a simulation of self that hides it's own nature and thus cannot be self aware.
Philosophically, right, if you allow me infinite resources, right, to do a thing I don't actually know how to define,
It's not infinite! If you take my cherry picked estimate of the computational power of the human brain, you'll see we're just one more round of scaling to have matched the human brain, and then we're sure to have AGI ~~and make our shareholders immense profits~~! Just one more scaling, bro!
money please!
I think you're reading some arguments I'm not making. The author seems to be of the opinion that even with infinite resources it's outright impossible to have a computer that thinks or experiences consciousness, which is obviously a philosophical, not practical, argument, and I don't agree. I'm not saying we should actually try it, or that it's doable with our current or foreseeable resources.
That being said, I am defining it. I'm saying that even if we assume that it's utterly impossible to have consciousness any other way, it's some incredibly unique combination of the things that make us human and literally any deviation whatsoever makes it all fall apart, there still seems to be a possible path to a computer with consciousness via simulation of that particular and special process. That's my thing you don't think I can define: a physics simulation of sufficient fidelity to simulate a thing we already know demonstrates consciousness. "There aren't enough resources in the universe" would block that path, sure, but that's not very interesting. Lots of other things could block that path, it's an insane path to an incredibly difficult goal. But saying "artificial consciousness is impossible, period" means something is blocking it, and the idea that it's some law of physics that is both crucial to consciousness and can't be simulated is interesting. I'm struggling to imagine how that would be possible, and if it's a failure of my imagination I'd like to know. The universe doesn't have to be computable or deterministic to make a simulation that imitates it so well that observations of the real and simulated physics yield indistinguishable results, and at that point I don't see anything inherently preventing consciousness.
How do you define consciousness?
Well, I don't really need to for what I'm saying, which is that I don't see any reason a computer is fundamentally incapable of doing whatever it is that humans do to consider ourselves conscious. Practically incapable, maybe, but not by the nature of what it is. Define it how you like, I don't see why a computer couldn't pull the same trick in the distant future.
Personally, though, I define it as something that exhibits awareness of a separation between itself and the rest of the world, with a large fuzzy spectrum between "has it" and "doesn't have it". Rocks exhibit no awareness whatsoever, they're just things with no independence. Plants interact with the world but don't really seem like they're doing much more than reacting, which doesn't demonstrate much, if any. Animals do a lot more, and the argument that at least some are conscious is quite plausible. An LLM kinda, sorta, momentarily glances off of this sometimes, enough that you can at least ask the question, but the fact that outside of training it is literally just an unchanging matrix of numbers and that its only "view" into the world is what you choose to give it means that it can't be aware of itself versus the world, it's at best aware of itself versus its training set and/or context window, a tiny keyhole to a photograph of something in the world, means it makes a barely discernible blip on the scale, on the level of plants, and even that might be generous. An artificial consciousness would, in my opinion, need to be independent, self-modifying, and show some of the same traits and behaviors that we see in animals up to and including ourselves with regard to interacting with the rest of the world as a self-contained being.
yud-length posts like these that boil down to “nuh-uh” are why we have the “no debate unless it’s amusing debate” rule, and I can see by the downvotes from local and the reactions that you’ve failed to be amusing
if I were you I’d reconsider this thread
If anything I'm posting conjures that bozo's name for any reason, I'll cut my losses, my apologies.
ah, I see. Ok. Well, based on all that, you haven’t actually engaged with anything from the post, nor have you said anything non-trivial. Was hoping that you’d at least say something wrong instead.
As a summary,
(I'm going to say "you" in this response even though you're stating some of these as arguments from the author and not yourself, so feel free to take this as a response to the author and not you personally if you're playing devil's advocate and don't actually think some of these things.)
But it does take place in the real world. Where do you think the computers are going to be? Computers can and do exist in and interact with the real world, they always have, so that box is already checked. You can imagine the computations as happening in a sort of mathematical void outside of the universe, but that's mostly only useful for reasoning about a system. After you do all that, you move electrons around in a box and see the effects with your own human senses.
Well, yeah, current LLMs are tiny and stupid. Something bigger, and probably not an LLM at all, might not be.
It doesn't have to actually fit reality perfectly, and it doesn't have to be able to predict reality like a grand unified theory would. It just needs to behave similarly enough to produce the same effects that brains do. It hasn't been shown to be possible, but there's also no reason to think we can never get close enough to reproduce it.
Yes it does. If they're indistinguishable, there is no difference.
I don't have any experience writing physics simulators myself, but they definitely exist. Even as a toy example, the iOS app Dice by PCalc does its die rolls by simulating a tossed die in 3D space instead of a random number generator. (Naturally, the parameters of the throw are generated, the simulation is just for fun, but again, it's a distinction without a difference. If the results have the same properties, the mechanism doesn't matter.) If I give you a billion random numbers, do you think you could tell if I used the app or a real die? Even if you could, would using one versus the other be the difference between a physics simulation being accurate or inaccurate enough to produce consciousness?
Of course. This is addressing an argument made by the post that computers are inherently incapable of intelligence or consciousness, even assuming sufficient computation power, storage space, and knowledge of physics and neurology. And I don't even think that you need to simulate a brain to produce mechanical consciousness, I think there would be other, more efficient means well before we get to that point, but sufficiently detailed simulation is something we have no reason to think is impossible.
Why not? And even if so, what's stopping you from bringing in the externalities as well?
What are the rules of the filing system? If they're complex enough, and executed sufficiently quickly that I can converse with it in my lifetime, let me be the judge of whether I think it's intelligent.
So, I actually agree broadly with you in the abstract principle but I've increasingly come around to it being computationally intractable for various reasons. But even if functionalism is correct...
We don't have the neurology knowledge to do a neural-level simulation, and it would be extremely computationally expensive to actually simulate all the neural features properly in full detail, well beyond the biggest super computers we have now and "moore's law" (scare quotes deliberate) has been slowing down such that I don't think we'll get there.
A simulation from the physics level up is even more out of reach in terms of computational power required.
As you say:
We really really don't have the neuroscience/cognitive science to find a more efficient way. And it is possible all of the neural features really are that important to overall cognition, so you won't be able to do it that much more "efficiently" in the first place...
Lesswrong actually had someone argue that the brain is within an order or magnitude or two of the thermodynamic limit on computational efficiency: https://www.lesswrong.com/posts/xwBuoE9p8GE7RAuhd/brain-efficiency-much-more-than-you-wanted-to-know
I think that this is your best path forward. Go simulate some rigid-body physics. Simulate genetics with genetic algorithms. Simulate chemistry with Petri nets. Simulate quantum computing. Simulate randomness with random-number generators. You'll learn a lot about the limitations that arise at each step as we idealize the real world into equations that are simple enough to compute. Fundamentally, you're proposing that Boltzmann brains are plausible, and the standard physics retort (quoting Carroll 2017, Why Boltzmann brains are bad) is that they "are cognitively unstable: they cannot simultaneously be true and justifiably believed."
A lesser path would be to keep going with consciousness and neuroscience. In that case, go read Hofstadter 2007, 'I' is a strange loop to understand what it could possibly mean for a pattern to be substrate-independent.
No, you're likely to suffer the ELIZA Effect. Previously, on Awful, I've explained what's going on in terms of memes. If you want to read a sci-fi story instead, I'd recommend Watts' Blindsight. You are overrating the phenomenon of intelligence.
I'm clearly failing to communicate my thoughts, and doing it in the wrong forum, but I appreciate the links, I'm excited to learn new things from them.
I'll gladly endorse most of what the author is saying.
This isn't really a debate club, and I'm not really trying to change your mind. I will just end on a note that:
Neither the author nor me really suggest that it is impossible for machines to think (indeed humans are biological machines), only that it is likely—nothing so stark as inherently—that Turing Machines cannot. "Computable" in the essay means something specific.
Simulation != Simulacrum.
And because I can't resist, I'll just clarify that when I said:
It means that the test does (or can possibly) exist that, it's just not achievable by humans. [Although I will also note that for methods that don't rely on measuring the physical world (pseudo random-number generators) the tests designed by humans a more than adequate to discriminate the generated list from the real thing.]
Sure, doesn't have to be a debate of course. My read was a pretty explicit belief that there is likely something to biology that is fundamentally unreachable to a computing machine, which I was skeptical of.
this is an extended "nuh-uh"
I think the point being made is perfectly reasonable. It's entirely possible to think that there's probably nothing stopping things more like a computer than like a brain having actual intelligence/consciousness, even if there's no particular pointer from where we are to that state. We are an existence proof that matter can do this, and there's little reason to think it's the only way it can.
No, it's not reasonable. This is what you're saying:
This is logically valid but meaningless. There's nothing to be done with this. There's no reason to be had here.
Then we have the banal take of "if we had a magic box with infinite capabilities, it could do X!", where, in this case, X happens to be "have consciousness". Ok! Great. You have fun playing in the sandpit, thinking about your magic box. I'm gonna smoke cigarettes and play slot machines for an hour.
It's only when we start bringing the discussion down to simulations on a Turing machine that this stuff gets interesting. But that's not what y'all are trying to talk about, because you haven't read the goddamn essay.
thread winner
I saw a chance to RP a deadbeat parent and I took it
the existence of a brain doesn't show you can simulate it in a computer. the universe is not necessarily feasible to simulate down to the atoms, as the essay points out. did you read the essay?