Quote (dude_927 @ Dec 24 2014 02:06pm)
everything, you are talking to me right now from likely a very great distance on a big old calculator, and it doesn't want to kill you lol

Lol, what I mean is, we make calculators and computers that perform algorithmic computations. We are the ones who wrote the computations, but if we could compute the outputs as efficiently and accurately as the computer, we wouldn't have built computers in the first place, cause they'd just be redundant.
So this elucidates what I mean when I say that understanding the algorithm doesn't imply knowing the output.
Quote (dude_927 @ Dec 24 2014 02:06pm)
I would actually speculate that a "higher intelligence" would be more peaceful and empathetic in all likelihood just due to mutual gain from cohabitation
This is a possibility, but it seems to me vastly unlikely than an aggressive AI. Keep in mind that I'm not saying the AI is "angry" or has any emotional sway at all for killing. Merely that it has some goal, even an arbitrary one (say, accurately counting the grains of sand on Earth) and if something gets in the way of the goal (humans) or prevents it from accomplishing that goal with maximum efficiency, it won't think twice about removing the obstacle (killing us).
Quote (dude_927 @ Dec 24 2014 02:06pm)
(also circuitry is pretty delicate and theres alot of drunk rednecks just waiting for some terminators or a zombie to come around)
It's clear that a superintelligence would be able to anticipate attacks on it and defend itself accordingly. If it gained a decisive strategic advantage (which it almost necessarily would if it were sufficiently more intelligent than us), there would be literally nothing humans could do to stop it from maximizing its utility function. This is why the utility function we use is so important.
Quote (dude_927 @ Dec 24 2014 02:06pm)
yes but don't you think it's just as likely the ai would inform someone it is out of materials, its an ai, it doesnt have to go logic loop insane over a simple command, or if it did a shotgun couldn't fix it? again i'm not even disagreeing, just pointing out that this is all wild speculation
If it's out of materials, it would try to get more materials. It would be unconcerned with anything outside of what its programmed values are. No, a shotgun would most certainly be ineffective because the AI would recognize being destroyed as being a hindrance on it accomplishing whatever goal it's programmed to do and would defend itself accordingly, or remove the threat, or replicate itself, or what have you.