SneerClub

Not to be too much of a fujoshi in public but Shakespeare is writing about a boy not a girl!!!!!!!

This was a great essay.

Man. Using Sonnet 18 here is just utterly brilliant. There’s a lot that could be said but my angle is: you can’t directly compare someone to a summer’s day; it has to be done through poetry and metaphor. Shit, cat. Shaka when the walls fell.

Sorry for the following somewhat disproportionate aggression

Searle said

I feel somewhat embarrassed to give even this answer to the systems theory because the theory seems to me so unplausible to start with. The idea is that while a person doesn’t understand Chinese, somehow the conjunction of that person and bits of paper might understand Chinese. It is not easy for me to imagine how someone who was not in the grip of an ideology would find the idea at all plausible

As I was reading this I was screaming silently: YOU invented the chinese room. It was ENTIRELY YOUR IDEA to come up with a ridiculous, unphysical, implausible thought experiment where a single human somehow does the task of millenia in the span of minutes.
And now you object that it seems implausible???

Millenia is very optimistic by the way. If you tried to simulate chatgpt with paper and a pen, it would take much, much longer than that.
AND THE PIECE OF JUNK STILL WOULDN'T EVEN GET THE CHINESE CHARACTERS RIGHT.

Author echoes my thoughts by calmly stating:

Searle simply puts the cart before the horse. Let the high speed men with paper, pencil, and rubber commence using their rulebook to carry on a conversation, whether in Chinese or any other language, and then we can discuss the metaphysical implications."

Motherfucker, Sartre has set the cart on fire and shot the horse, and you are contemplating whether to dance on the remains!
Ok, ok, maybe it metaphysically makes sense. But you're exhaustively drawing a connection between the metaphysical and the practical! Now it can't make sense!

Interrogate our intuitions with one centillion shrimp.
incoherent screeching

I'm going to be a little indirect and poetic here.

In Turing’s view, if a computer were to pass the Turing Test, the calculations it carried out in doing so would still constitute thought even if carried out by a clerk on a sheet of paper with no knowledge of how a teletype machine would translate them into text, or even by a distributed mass of clerks working in isolation from each other so that nothing resembling a thinking entity even exists.

Yes. In Smullyan's view, the acoustic patterns in the air would still constitute birdsong even if whistled by a human with no beak, or even by a vibrating electromagnetically-driven membrane which is located far from the data that it is playing back, so that nothing resembling a bird even exists. Or, in Aristoteles' view, the syntactic relationship between sentences would still constitute syllogism even if attributed to a long-dead philosopher, or even verified by a distributed mass of mechanical provers so that no single prover ever localizes the entirety of the modus ponens. In all cases, the pattern is the representation; the arrangement which generates the pattern is merely a substrate.

Consider the notion that thought is a biological process. It’s true that, if all of the atoms and cells comprising the organism can be mathematically modeled, a Turing Machine would then be able to simulate them. But it doesn’t follow from this that the Turing Machine would then generate thought. Consider the analogy of digestion. Sure, a Turing Machine could model every single molecule of a steak and calculate the precise ways in which it would move through and be broken down by a human digestive system. But all this could ever accomplish would be running a simulation of eating the steak. If you put an actual ribeye in front of a computer there is no amount of computational power that would allow the computer to actually eat and digest it.

Putting an actual ribeye in front of a human, there is no amount of computational power that would allow the human to actually eat and digest it, either. The act of eating can't be provoked merely by thought; there must be some sort of mechanical linkage between thoughts and the relevant parts of the body. Turing & Champernowne invented a program that plays chess and also were known (apocryphally, apparently) to play "run-around-the-house chess" or "Turing chess" which involved standing up and jogging for a lap in-between chess moves. The ability to play Turing chess is cognitively embodied but the ability to play chess is merely the ability to represent and manipulate certain patterns.

At the end of the day what defines art is the existence of intention behind it — the fact that some consciousness experienced thoughts that it subsequently tried to communicate. Without that there’s simply lines on paper, splotches of color, and noise. At the risk of tautology, meaning exists because people mean things.

Art is about the expression of memes within a medium; it is cultural propagation. Memes are not thoughts, though; the fact that some consciousness experienced and communicated memes is not a product of thought but a product of memetic evolution. The only other thing that art can carry is what carries it: the patterns which emerge from the encoding of the memes upon the medium.

So one point I have to disagree with.

More to the point, we know that thought is possible with far less processing power than a Microsoft Azure datacenter by dint of the fact that people can do it. Exact estimates on the storage capacity of a human brain vary, and aren’t the most useful measurement anyway, but they’re certainly not on the level of sheer computational firepower that venture capitalist money can throw at trying to nuke a problem from space. The problem simply doesn’t appear to be one of raw power, but rather one of basic capability.

There are a lot of ways to try to quantify the human brain's computational power, including storage (as this article focuses on, but I think its the wrong measure, operations, numbers of neural weights, etc.). Obviously it isn't literally a computer and neuroscience still has a long way to go, so the estimates you can get are spread over like 5 orders of magnitude (I've seen arguments from 10^13 flops and to 10^18 or even higher, and flops is of course the wrong way to look at the brain). Datacenter computational power have caught up to the lowers ones, yes, but not the higher ones. The bigger supercomputing clusters, like El Capitan for example, is in the 10^18th range. My own guess would be at the higher end, like 10^18, with the caveat/clarification that evolution has optimized the brain for what it does really really well, so that the compute is being used really really efficiently. Like one talk I went to in grad school that stuck with me... the eyeball's microsaccades are basically acting as a frequency filter on visual input. So literally before the visual signal has even got to the brain the information has already been processed in a clever and efficient way that isn't captured in any naive flop estimate! AI boosters picked estimates on human brain power that would put it in range of just one more scaling as part of their marketing. Likewise for number of neurons/synapses. The human brain has 80 billion neurons with an estimated 100 trillion synapses. GPT 4.5, which is believed to have peaked on number of weights (i.e. they gave up on straight scaling up because it is too pricey), is estimated (because of course they keep it secret) like 10 trillion parameters. Parameters are vaguely analogs to synapses, but synapses are so much more complicated and nuanced. But even accepting that premise, the biggest model was still like 1/10th the size to match a human brain (and they may have lacked the data to even train it right).

So yeah, minor factual issue, overall points are good, I just thought I would point it out, because this factual issue is one distorted by the AI boosters to make it look like they are getting close to human.

That’s because there’s absolutely reams of writing out there about Sonnet 18—it could draw from thousands of student essays and cheap study guides, which allowed it to remain at least vaguely coherent. But when forced away from a topic for which it has ample data to plagiarize, the illusion disintegrates.

Indeed, Any intelligence present is that of the pilfered commons, and that of the reader.

I had the same thought about the few times LLMs appear to be successful in translation, (where proper translation requires understanding), it's not exactly doing nothing, but a lot of the work is done by the reader striving to make sense of what he reads, and because humans are clever they can somtimes glimpse the meaning, through the filter of AI mapping a set of words unto another, given enough context. (Until they really can't, or the subtelties of language completely reverse the meaning when not handled with the proper care).

One needn’t go as far as souls anyway. Jefferson’s hypothesis—that there is some electrochemical basis to thought—is sufficient to solve the problem. Were it true, the reason computers seem fundamentally blocked from progress on the Turing Test would amount to the fact that they are wholly mechanical objects, while “thought” is as much a biological function as “digestion” or “copulation.”

Even if true, why couldn't the electrochemical processes be simulated too? I don't think it's necessary to strictly and completely reproduce a biological brain to produce thought in a computer, but even if it is, it's "just" a matter of scale. If you can increase the fidelity of the simulation with effectively infinite computing power, what would it be missing? It'd have to be something that can't be predicted, can't even have its unpredictability described with an equation (I don't know what any coin flip will turn up as, but I do know how to write a program that produces indistinguishable results from a real coin for a simulation), so it's just changing all the time and follows no rules whatsoever, but also you can't just write a program that does its own "random crap that can't be predicted" simulation because the real one is somehow also so precise that it's the only thing that makes consciousness work and a mechanical one isn't good enough?