Can friendly AI evolve?

Humans seem to presume an "us vs. them" mentality when it comes to machine intelligence (certainly in the movies =).

But is the desire for self-preservation coupled to intelligence or to evolutionary dynamics?… or to biological evolution per se? Self-preservation may be some low-level reflex that emerges in the evolutionary environment of biological reproduction. It may be uncoupled from intelligence. But, will it emerge in any intelligence that we grow through evolutionary algorithms?

If intelligence is an accumulation of order (in a systematic series of small-scale reversals of entropy), would a non-biological intelligence have an inherent desire for self-preservation (like HAL), or a fundamental desire to strive for increased order (being willing, for example, to recompose its constituent parts into a new machine of higher-order)? Might the machines be selfless?

And is this path dependent? Given the iterated selection tests of any evolutionary process, is it possible to evolve an intelligence without an embedded survival instinct?

FCC Indecency & Howard Stern

I forgot to comment on the remarkably candid interview that FCC Chairman Michael Powell gave last month on the topics of broadband policy, industry transitions, regulatory philosophy, Skype and VOIP, censorship and Howard Stern. While the streaming video has been available, the transcript proliferated in the blogosphere:
• Denise Howell captured the most salient parts of the broad discussion.
• Marc Canter covers Powell’s further ruminations on indecency.

At minute 25:08 (and into the Q&A), I ask about the recent FCC crackdown on indecency. I had two pages of questions from Howard Stern (who has no great love for the FCC), and in a burst of recursive irony, I self-censored the indecent ones (like PBS recently). Here are some of the questions from Howard, and I only got to the first one in the interview:

“Aside from Oprah, who else will you NOT fine?”
“What makes the FCC qualified to determine what is indecent?”
“What role should religion play in determining indecency standards?”

The FCC answer points to the number of complaints as the motivation for the crackdown. This sounds like a voting system of “majority rules”…. which seems to run counter to the spirit of the First Amendment and the protection of minority voices.

The coolest thing you learned this year?

In the spirit of lifelong learning, what is the coolest new thing you learned this year?

Last year, I think it was at a dinner with Matt Ridley talking about the inter-gene warfare going on within our bodies, especially between the X and Y sex chromosomes.

For this year, I can’t seem to pick one thing. Conversations with the eponymous Mr. Smart come to mind. Here is an example of his thinking about the limitations of biology as a substrate for developing computational complexity.

Jaron Lanier is also a wonderful thinker, and when we writes for my favorite “interesting ideas” site (EDGE.org), it’s a potent combination. He makes an interesting counterpoint: “We're so used to thinking about computers in the same light as was available at the inception of computer science that it's hard to imagine an alternative, but an alternative is available to us all the time in our own bodies.”

Reconciling the two, perhaps biology will drive the future of intelligence and information technology – not literally, but figuratively and metaphorically and primarily through powerful abstractions.

Many of the interesting software challenges relate to growing resilient complex systems or they are inspired by other biological metaphors (e.g., artificial evolution, biomimetics, neural networks for pattern recognition, artificial immunology for virus and spam detection, genetic algorithms, A-life, emergence, IBM’s Autonomic Computing initiative, meshes and sensor nets, hives, and the subsumption architecture in robotics). Tackling the big unsolved problems in info tech will likely turn us to biology – as our muse, and for an existence proof that solutions are possible.

Quantum Computational Equivalence

An interesting comment on "Your Genome is Smaller than Microsoft Office" referenced quantum effects to explain the power of the interpreters of biological code.

I recently heard Wolfram present his notion of "computational equivalence", and I asked about quantum computers because it seemed like a worm hole through his logic… but he seemed to dismiss the possibility of QCs instead.

The abstract summary of my understanding of computational equivalence is that many activities, from thinking to evolution to cellular signaling, can be represented as a computation. A physical experiment and a computation are equivalent. For an iterative system, like a cellular automata, there is no formulaic shortcut for the interesting cases. The simulation is as complex as “running the experiment” and will consume similar computational resources.

Quantum computers can perform accurate simulations of any physical system of comparable complexity. The type of simulation that a quantum computer does results in an exact prediction of how a system will behave in nature — something that is literally impossible for any traditional computer, no matter how powerful. Professor David Deutsch of Oxford summarizes: “Quantum computers have the potential to solve problems that would take a classical computer longer than the age of the universe.”

So I wonder what the existence of quantum computers would say about computational equivalence? How might this “shortcut through time” be employed in the simulation of molecular systems? Does it prove the existence of parallel universes (as Deutsch concludes in Fabric of Reality) that entangle to solve computationally intractable problems? Is there a “quantum computational equivalence” whereby a physical experiment could be a co-processor for a quantum simulation? Is it a New New Kind of Science?

Morpheus beats the RIAA

A new development: Morpheus just unanimously won their 9th Circuit case. The entertainment industry lawyers were so confident that they would prevail in the case that they did not have a statement ready for this scenario.

The justices actually addressed Congress and urged them not to pass anti P2P legislation so quickly. They added:

“we live in a quicksilver technological environment with courts ill-suited to fix the flow of internet innovation… The introduction of new technology is always disruptive to old markets, and particularly to those copyright owners whose works are sold through well established distribution mechanisms. Yet, history has shown that time and market forces often provide equilibrium in balancing interests, whether the new technology be a player piano, a copier, a tape recorder, a video recorder, a personal computer, a karaoke machine, or an MP3 player. Thus, it is prudent for courts to exercise caution before restructuring liability theories for the purpose of addressing specific market abuses, despite their apparent present magnitude."

Bravo.

iTunes Licensing Model

I just received a call from one of my favorite musicians. He told me that when Apple sells one of his songs for 99 cents, EMI gets 66 cents and he gets 5 cents.

EMI just ported the business contract of physical distribution (which presumes manufacturing costs, breakage, inventory and other real costs). So the music label unilaterally captured 100% of the upside from moving the business online and shared none of it with the artist.

Having just finished reading Free Culture, I guess I should not be surprised by this habitual behavior. But it seems so old school.

My channel and fulfillment relationship is now with Apple. EMI provides no value to me in this modern context. Yet they take more than 10x what they share with the artist.

Your Genome is Smaller than Microsoft Office

How inspirational are the information systems of biology?

If we took your entire genetic code -- the entire biological program that resulted in your cells, organs, body and mind -- and burned it into a CD, it would be smaller than Microsoft Office. Two digital bits can encode for the four DNA bases (A,T,C and G) resulting in a 750MB file that can be compressed for the preponderance of structural filler in the DNA chain. Even with simple Huffman encoding, we should get below the 486MB of my minimal Office 2004 install.

If much of the human genome consists of vestigial evolutionary and parasitic remnants that serve no useful purpose, then we could compress it to 60MB of concentrated information.

What does this tell us about Microsoft? About software development? About complex systems development in general?

Genetic Free Speech

Following the J-Curve from the downer of the prior post, there is much to be excited about.

Earlier this year, I had the wonderful opportunity to co-teach a new interdisciplinary class at Stanford with Prof. Larry Lessig. It was called “Ideas vs. Matter: the Code in Tiny Spaces” and we discussed genetics, nanotechnology and the regulatory ecosystem.

We went in with the presumption that society will likely try to curtail “genetic free speech” as it applies to human germ line engineering, and thereby curtail the evolution of evolvability. Lessig predicts that we will recapitulate the 200-year debate about the First Amendment to the Constitution. Pressures to curtail free genetic expression will focus on the dangers of “bad speech”, and others will argue that good genetic expression will crowd out the bad. Artificial chromosomes (whereby children can decide whether to accept genetic enhancements when they become adults) can decouple the debate about parental control. And, with a touch of irony, China may lead the charge.

Many of us subconsciously cling to the selfish notion that humanity is the endpoint of evolution. In the debates about machine intelligence and genetic enhancements, there is a common and deeply rooted fear about being surpassed – in our lifetime. But, when framed as a question of parenthood (would you want your great grandchild to be smarter and healthier than you?), the emotion often shifts from a selfish sense of supremacy to a universal human search for symbolic immortality.

Genetically Modified Pathogen (GMP) Policy

In repose to my first post requesting topics of interest, “anonymous” noted that this blog is the top result on a Google search for “IL-4 Smallpox”... a dubious and disturbing honor for what I was hoping to be a content-free blog.

Anon also asked “what do you think of DHS efforts for a realtime bio-sensor network?”

It is possible that with the mobilization of massive logistical resources around the planet, we will prevail over genetically modified and engineered pathogens (GMPs). But I would not bet on it. It would be great to have a sensor network, but with most Health and Human Services offices lacking a basic Internet connection, we have a way to go.

From what I can tell, a crash-program in antiviral development may provide a ray of hope (e.g., HDP-cidofovir and some more evolutionarily robust and broad-spectrum host-based strategies).

Most importantly, from my random walk through government labs, talks with policy planners, CDC folk and DOD Red Team members, I haven’t seen any policy bifurcation for GMPs (for detection and response). I think there should be distinct policy consideration given to GMPs vs. natural pathogens.

The threat from GMPs is much greater, and the strategic response would need special planning. For example, the vaccinations that eradicated smallpox last time around may not be effective for IL-4 modified smallpox, and in-situ quarantine may be needed. “Telecommuting” for many forms of work will need to be pre-enabled, especially remote operation of the public utilities and MAE-East &West and other critical NAP nodes of the Internet.

THE THREAT:
The delicate "virus-host balance" observed in nature (whereby viruses tend not to be overly lethal to their hosts) is a byproduct of biological co-evolution on a geographically segregated planet. And now, both of those limitations have changed. Organisms can be re-engineered in ways that biological evolution would not have explored, nor allowed to spread widely, and modern transportation undermines natural quarantine formation.

In evolution, pathogens do not become overly lethal to their host, for that limits their own propagation to a geographically-bound quarantine zone. Evolution may have created 100% lethal pathogens in the past, but those pathogens are now extinct because they killed all of their locally available hosts.

A custom-engineered or modified pathogen may not observe that delicate virus-host balance, nor the slow pace of evolutionary time scales, and could engender extinction level events with a rapidity never before seen on Earth. Given early truncation of the lethality branch (truncating a local maximum), evolution has not experimented with a multivariate global maximum of lethality. The pattern of evolution is small and slow incremental changes where each intermediate genetic state needs to survive for the next improvement to accumulate. Engineered and modified pathogens do not need to follow that pattern.