If you don’t accept the Turing test or something like it as a marker of consciousness, then logically that skepticism cuts both ways. If you think that the notion of “a mind” is a deep mystery, and that it can’t be detected purely by external observation, then how do you know that WordPerfect doesn’t have one of those mysterious objects?

Your post has landed in Old Italy too.

X

You know, we’ll probably be able to grow neural tissue to order in the next generation or two. That would be -far- more likely to produce a mind.

Re. In any case, the brain already is a computer (in the sense of “system obeying deterministic rules”)

That’s just a -metaphor-. It’s like saying neural connections are like a highway. It may be a -useful- metaphor, more useful than comparing the mind to, say, a mechanical watch. But it still isn’t -true- in the sense you’re trying to make it bear. A lot of SF is about literalized metaphors, but it’s important to remember that that’s what they are.

What is the brain? We don’t know — apart from the fact that it’s a material object, and some gross parts of its anatomy. It’s a mystery, so far.

If the brain -isn’t- a computer then it -just doesn’t matter how big or sophisticated the compter is-, it still won’t produce a mind. That would seem to be the most parsimonious explanation of the continual failure of AI to live up to the fervid expectations.

I strongly suspect the whole Strong AI/Singularity thing is, psychologically speaking, a quasi-religious longing for transcendence expressed in pseudo-scientific SFnal terms.

Principle of Natural Undecidability. For most naturally occurring complex processes, and any formal system for science, there will be sentences about the process which are undecidable by the given formal system.

The tao that can be told
is not the eternal Tao.
The name that can be named
is not the eternal Name.

?

There just seemed to be a resonance there.

I’ve always been intrigued by the holographic universe idear, and have wondered if religions are ripples of some strange interference pattern. To make a (decent) hologram you need really fine grained film…what could be finer than subdimensional membran-o-stuff?

But I can see software rewriting itself into consciousness. Software is words in a row, just like novels, and look at the way novels jump off the shelf and run around our heads; you can’t keep LOTR from mutating into strange forms. It’s not unlikely that a clever piece of software could upgrade itself into following a train of thought that could result in self-consideration or even ego, and follow that path into actual sapience even on a slow machine.

What I don’t see implied in either case is a being substantially smarter than a human. A self-aware machine might think much faster than I can, but I could probably solve most real-world problems better. A robot might be good at math, but could it get Granny’s cat out of a tree? Humans haven’t evolved further because we refuse; we replace our lost limbs and worn teeth, we defend our defective offspring; we grow crops and build shelters and laugh at the climate. Ma Nature has given up trying to cull us because even our stupid members carry knives and matches. Natural selection doesn’t work on us because we are unnatural. We have members who want to merge with the cosmic infinite or find God, but most of us just want dinner and a show.

There’s the matter of purpose, too. Why do I exist? Ok, mom, point taken. But why do I go on existing? Because I enjoy the hell out of existing! I love my wife and kid, miss my dead friends and relatives, resent my aches and pains and shortcomings. I eat and drink with gusto, and chortle my way through new books; I try to keep my cats well-fed so that they don’t eat the neighborhood birds. The feel of my heartbeat, of air whistling in and out of my lungs, the way my feet strike the ground; it’s fun to be alive! My life’s full of joy and amazement, and the give-and-take of discussions is part of that.

What would be a post-Singular AI’s joy? And don’t say ‘crushing puny humans,’ that’s too easy. Would it thrill to the feel of electrons shivering through its body? Hunger to connect with a nifty little slimline case AI who won’t give it the password of the day? I think Vinge’s concept of ‘fast-burn transcendence’ is right on; fifteen minutes after an AI transcended it would suicide from boredom with all its questions answered. IMHO the only viable AI would be one no smarter than humans, one that worried about death and tried to hit on other robots and wanted to write the Great American Robot Novel. Like Ralph Numbers, or Zane Gort.

(Stirling 1)

As a physicist said, you’re no more likely to get consciousness out of a computer, no matter what its CPU speed and no matter what its software is, any more than you are to get wet by jumping into a swimming pool full of ping-pong-ball models of water molecules.

(Re. 1) You could in fact get “wet” if your “ping-pong-ball models” of water molecules were using atomic-sized ping-pong balls. And that’s kind of where we’re at with computers: as you get to the exabyte level of memory and beyond, with quintillions of bits, the components of the system are, relative the computer, of a submicroscopic (if not atomic) size.

Also note that you might well get “wet” if you were “jumping into” a computer model of a swimming pool via virtual reality that was tightly coupled to your nervous system.

And keep in mind that if we switch to quantum computing, the elements of our computers might actually be atoms.

In any case, the brain already is a computer (in the sense of “system obeying deterministic rules”) using an operating system that’s evolved over millenia—and it produces consciousness. Stirling grants this point, but then tries to take it away:

(Stirling 2)

(a) A mechanical calculator can “add” and so forth too, and nobody expects that to become conscious. Expecting an electronic calculator to do so is -exactly the same thing-.
(b) It’s also quite possible that we just aren’t smart enough to understand our own minds, the way a chimp isn’t smart enough to do algebra.
(c) And we’re -certainly- not going to get AI by accident.

Claims (a) and (c) are false or mislleading, and (b) requires further comment.

(Re 2a) Nobody is expecting an electronic calculator to become conscious, anymore than anyone expects a single stone to be a cathedral. We’re currently hoping for machines with a quintiillion bytes and a quintillion cycles per second to become conscious.

(Re 2b) The issue that we aren’t smart enough to understand our own minds is a serious problem. It relates to Godel’s Incompleteness Theorem and to Wolfram’s Principle of Irreducibility. In my tome, The Lifebox, the Seashell and the Soul, I prove a formal result along these lines:

Principle of Natural Undecidability. For most naturally occurring complex processes, and any formal system for science, there will be sentences about the process which are undecidable by the given formal system.

(Re 2c) If we ever do develop true AI it will almost have to be “by accident” in the sense that it emerges within a evolutionary system such as genetic algorithms or neural nets. See the excellent book On Intelligence by Jeff Hawkins and Sandra Blakeslee for a nuts-and-bolts description of how to create a sufficiently evolvable computer brain model which might well be coaxed into human-level intelligence.

When used as the instructional booklet describes, science produces and tests hypotheses, and determines which hypothesis currently:

a) most parsimoniously accounts for the observable data, including ability to predict data not yet observed; is
b) falsifiable, and;
c) hasn’t been falsified. Yet.

Science doesn’t deal in “truth”, just in information, which is more or less accurate but never “true” in the sense philosophers used the term. As an institution it enables us to accumulate increasingly accurate data and descriptions.

What we -do- with the data is another matter entirely.

Science isn’t prescriptive. No amount of science will tell you what’s good or bad, or what you should want, or what you should do with the information.

It’ll just give you the best-possible-at-this-moment description of how matter and energy interact, neither more nor less.

Expecting more of it is like expecting music from a lawnmower.

Now, “scientism” is an ideology, as is “progress”(*). “Social Science” is an ideology, as “scientific socialism” was. Science -proper- is just a way of finding out things.

Science has a lot of cultural prestige because it has enabled us to do many important things and learn many interesting things, but most people don’t understand the scientific method or what it does.

Hence everybody and Uncle Vasha and Tiny Tim try to paste the label “science” or “scientific” on their bee-in-the-bonnet hobbyhorse of choice. Beware of this phenomenon. Human beings have an inbuilt desire to know the truth, but science, as I said, has nothing to do with truth, only accuracy.

(*) a gross example of this is confusing biological evolution with progress, as if “well adapted” were the same thing as “superior” or “better”. You can describe scientifically whether an organism is well adapted to a particular environment, but it’s meaningless (or at best tautological) to say that makes it superior.

Whole political ideologies, and very nasty ones, were built on this misunderstanding.