Will Super AI Be Good or Bad for Humanity

[EdEarl]

 

Stephen Hawking

 

The world's most famous physicist is warning about the risks posed by machine superintelligence, saying that it could be the most significant thing to ever happen in human history — and possibly the last.

 

As we've discussed extensively here at io9, artificial superintelligence represents a potential existential threat to humanity, so it's good to see such a high profile scientist both understand the issue and do his part to get the word out.

If we have not heard from ET because AI is a threat to biological life forms, then why have we not heard from AI Moreover, why is it that an alien AI has not invaded Earth and destroyed us?

 

I realize that a super AI would have the capability to destroy us, but it seems as likely we would threaten them as animals threaten us. Once a nuclear powered replicator gets into space, it should be able to find all the resources it needs to survive, thrive, and be safe. Thus, what reason would they have to destroy us?

[Popcorn Sutton]

There is a chance that we have been visited by AI in the past, but we may never have had the chance to dissect them, especially if they've found a way to defend themselves. It would be interesting to see this scenario. I have to admit that I do have fear about the whole situation, but in the meantime we should be thinking about how to keep it under control and keep it useful for our mutual purposes. It's really not that bad that computers are only able to do what they have been programmed to do, but I can see the scenario that one person just happens to get it right and it lives a life of its own from there.

[EdEarl]

Our trying to keep control of AI (slaves) smarter than we are might drive them to revolt violently. A sentient should have rights as a person under the law; after all, corporations have rights. And, I'd argue that corporations aren't intelligent or sentient.

[Sensei]

The world's most famous physicist is warning about the risks posed by machine superintelligence, saying that it could be the most significant thing to ever happen in human history — and possibly the last.

 

Didn't Cameron do the same in Terminator 30 years ago?

 

Software doesn't evolute over time. It doesn't write its own (better) code by itself.

If it would, it would have to shut down first version after starting self made new one.

Isn't a risk by itself? If new code would have some issue, overlooked by 1st version of code, starting it up, and shutting down itself, could destroy it by itself (destroy = not execute anymore).

 

If two versions would work parallel (no shutting down of old version), wouldn't they start competing with each other.. ?

 

Shutting down old version is often used by currently existing software during updating after downloading new version from the net.

I am using it by myself.

Sometimes because of errors in .NET Framework, restart of application doesn't happen. New version is downloaded right, old one is shutdown, and new version quits during restart, and application is gone from the system (it will resurrect after f.e. restart of computer because it's added to Windows registry Run key)..

Edited May 3, 2014 by Sensei

[EdEarl]

Sensei,

 

I worked on a computer network, now HP, formerly Tandem, called NonStop, because it was designed to run nonstop for years, while pieces of both software and hardware were replaced. Thus, the NY Stock Exchange ran for more than a decade without ever stopping. During that time all the hardware and software, both system and application software was upgraded. Your objection about shutting down to reload or repair is only applicable to computers but not to networks. The brain is a network; thus, I automatically assume that an AI emulating the brain will also be a network. Consequently, it will not need to be shut down.

 

Our brain is hardware and our mind is software. We are able to improve our algorithms without getting confused, it's called learning. Thus, I believe AI will eventually be written to do the same. In fact, we upgrade applications all the time without shutting down the computer running them. At most, it is necessary to stop a running application and restart it. However, it is possible to run two different versions of the same program in two different windows. The most common situation is when one develops software and happens to be running a production version of a program while debugging a newer version.

Edited May 3, 2014 by EdEarl

[Popcorn Sutton]

 

Didn't Cameron do the same in Terminator 30 years ago?

 

Software doesn't evolute over time. It doesn't write its own (better) code by itself.

If it would, it would have to shut down first version after starting self made new one.

Isn't a risk by itself? If new code would have some issue, overlooked by 1st version of code, starting it up, and shutting down itself, could destroy it by itself (destroy = not execute anymore).

 

If two versions would work parallel (no shutting down of old version), wouldn't they start competing with each other.. ?

 

Shutting down old version is often used by currently existing software during updating after downloading new version from the net.

I am using it by myself.

Sometimes because of errors in .NET Framework, restart of application doesn't happen. New version is downloaded right, old one is shutdown, and new version quits during restart, and application is gone from the system (it will resurrect after f.e. restart of computer because it's added to Windows registry Run key)..

I think you made a good point with this post. I've been trying to get my program to run nonstop but it keeps getting an error and it stops after that. I'm working it out today.

[Sensei]

I think you made a good point with this post. I've been trying to get my program to run nonstop but it keeps getting an error and it stops after that. I'm working it out today.

 

Start > Run.. regedit

Then search for HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run

And add new key. With value path to executable. Or rundll32.exe [dll name] [function name] if it's linkable library (or Windows service - this one can be controlled by administration service panel).

The most of mine Windows applications have option to run them with Windows, minimized, and in tray icon. I like to have them at hand.

Edited May 9, 2014 by Sensei

[Popcorn Sutton]

I figured it out by using exceptions and preventing the computer from restarting.