if x then y
if x is like y then z
if xyz is like abc then q elsif p say "wtf"
how many more tags do i need to put into my calculator before it can take over the world? this is what you are arguing about, yes i ridiculously oversimplified it, but at its core this is the discussion, the worry is that future programmers will use "bad tags" that cause "evil logic" in computers, not that they will develop a power of self and decide they like wars (that's a human triat, stop projecting), if we do manage to create an intilligence greater than ourselves the idea is that they in fact are smarter, not just ridiculous killing machines, and the sheer ridiculousness of "creating something more complex than yourself" should speak for itself. How would you possibly know how to write a line of code superior to the mind thinking up said line of code without the ability to augment your own mind in a similar fashion, its ridiculous. you are basically saying "blind men are on the verge of teaching children to paint" no they aren't, they just helped make a mess of my living room

e: inb4 gawd aweful lines of code lol

This post was edited by dude_927 on Dec 24 2014 12:16am

You deeply misunderstand the issue.

Even if the programmers put in the best tags possible in order to try and make sure the AI will be perfectly benign, it will still probably go wrong. Something as simple as a utility function of "make as many paperclips as possible" would spell almost certain doom for humanity as the AI will proceed to turn every last living thing into paperclips. Try to think of a utility function that seems safe and I can guarantee you overlooked something and it would go wrong.



We do things to make ourselves more intelligent all the time. There is no physical law that states we can't make something more intelligent than ourselves. Especially when there are hundreds of thousands or millions of people working together on it.

please teach me how to code "make as many paperclips as possible", my whole point was that these crazy sci-fi lines of code are ridiculous and not even close to how digital logic circuits work. granted (and likely) the "real ai" might not be on a digital computer, but now we are are in the realm of wild speculation.




agree, i didn't say it was impossible, nor did i say the blind teaching children to paint was, i said it was ridiculous to say "we are on the verge of ai" just like it is ridiculous to say "the blind taught my children to paint the mona lisa", The idea that a human could create a brain more powerful than its own yet know nothing of the inner working of its own mind are ridiculous, if you can code for intelligence you can read the input data required for that code, and if you know the inputs, you can read the outputs, and if you can read the outputs, you can decode your brain, and if you can decode it you can augment it, and if you can augment it what you create can never be more complex than the mind that is creating it. (little off topic, but the topic is a little off anyway lol)

Massive research organizations with billions of dollars behind them, and government defense agencies, are doing research on this very issue of machine learning and autonomous machine behavior, I'm inclined to take them seriously.

I don't know the details of how it would be coded, but maximization of paperclips wouldn't be difficult to code; it'd be a joke compared to actually coding an AI. If you want something even simpler, then make the utility function "Compute as many digits of pi as possible". The AI, assuming it has general superintelligence, would exploit as many resources as possible to fuel its computational ability to compute digits of pi more efficiently (likely exploiting humans as resources capable of being converted into computational hardware). The utility function would be very simple to code.




Yep I completely agree. Though there is some low probability that we can create an AI with an entirely different framework from a human brain and so understanding our own brain will turn out to be unnecessary.



I don't think so. Just because we can understand a certain computational function doesn't mean we can compute it with ease. Just because I know something is a problem and how to find the solution, doesn't mean I can actually solve it.

There's also the obvious dilemma that the brain is an extremely fragile thing inside of a hard skull which is not easily modified at all without damaging it, and to do so requires intensive surgery. We also have no access to our computational "source code" and as such have no direct access over our "software". Genome editing is much less direct, less effective, and vastly slower than editing an AI program, and increasing the computational capacity of an AI is as simple as changing out hardware or plugging in additional resources.

i take them very seriously, i just do not over-speculate about what they are actually doing (i'm reminded of the pre cern "black hole if you turn it on" phase right now)


NO! "compute as many digits of pi as possible" is a program that is ridiculously easy and ridiculously far from ai: x=circumference/diameter and set the maximum outputs as high as possible, you cannot code "add another output" that is a physical component and needs to be added manually (or converted to scientific notation but i am trying to keep it simple and its the same concept)



yes but its framework will have a distinct set out inputs/outputs, and while it is possible they would not mesh, the likelyhood of us not recognizing the physical components presumably required for cognitive intelligence , is far worse than the blind teacher analogy




yes it does, that is the absolute only thing you need to to develop a computational model



circuitry is usually fragile, it is presumed an intelligent brain would be very susceptible to damage (as is shown in practice), the "source code" is exactly the problem, but if we can develop a competent source code we would have a general idea of the electrical "outputs" it would generate, this principle is already used by every reverse engineering specialists from CIA decryption to aimbotting cs:go

We will augment our own abilities before computers become smart enough to usurp us. The future is in robotic humans, not human like computers


Human brains are not a quantum computer, the switches are well understood, we just don't know how they with together to make the software we experience.

We also know why silicon based life wouldn't work. The bond energies don't synch like they do with carbon to create a large macromolecules like proteins.




Pi isn't computed that way, its actually something that was a big mystery and it requires infinite sums to actually do. You can only do circumfrence/(r squared) if you already have a circle which the dimensions will become pi when you do that, but getting that circle requires knowledge of pi.


This post was edited by Thor123422 on Dec 24 2014 02:36am

ya my bad, wasn't thinking when i typed that, the point is that it has a finite number of "output digits" (not that i am good at math, i am not lol) and adding more "digits" is a physical process not a digital one, it is the amount of digits holding back that particular calculation not the lack of programming skills, which makes the pi objection completely irrelevant to the situation, but even if it wasn't "calculate pi more precisely" is not even close to "build unlimited paperclips until you use up every resource imaginable" in terms of ai

I didn't mean the entire program would be just programmed to compute digits. Recall we're talking about artificial GENERAL intelligence. It's just that its UTILITY FUNCTION (i.e. its goal or value structure) would have the goal of computing as many digits of pi as possible. If we make some general intelligence, we can code a value or utility system into it which it will attempt to maximize at all costs. In this case, it would use as many resources as possible to compute the maximum amount of digits of pi it can. A superintelligence would recognize that exploiting as many resources as it can would be optimal for maximal computational efficiency. It would also recognize that self-improving its intelligence and other things (known as instrumental goals) could improve the numbers of pi it could calculate.




I might be missing your point, but as a programmer, I can assure you I frequently write and fully understand code without having a clue what the output's going to be. That's kind of the whole point of a computer.



Yes I think so as well, but there's a non-zero probability we're wrong, and it's worth taking seriously because it's easily the biggest existential risk if it is more imminent than we expect.




So we "just" don't understand the single most crucial feature of the brain (phenomenal experience). We still have no solution to the phenomenal binding problem and we must either renounce reductive physicalism (and renounce the scientific worldview entirely) or come up with a non-classical explanation of the mind-brain. I'm inclined to favor the latter. I expect the current "understanding" of neuroscience and consciousness will turn out to be profoundly ignorant. But of course, I'm in the minority here.



Perhaps, though that's not to say there could be some alternative way of doing it? I really haven't thought about it much so I don't have a clue, just an idea.




I'm not sure your point here. Obviously "calculate as many digits of pi as possible" is literally the same thing as "calculate pi as precisely as possible". How is it every different than building paperclips in the way that resources would be exhausted to accomplish the goal (besides the details of how they're used of course)?

the general intelligence is the whole problem, you can't just bypass it by saying "if we already had it created"




i am only a "hobbyist", but i fail to see how you could possibly call up a mathematical function and not know what the outputs would be (or at least the hex equivalent)



yes there is a non zero probability a blind man could teach art, but i wouldnt suggest rolling out the red carpet



The details are are the issue, you are proposing "sci-fi" type coding and completely ignoring the details, "make as many paperclips as possible" (at least how i understand that statement in code) does not include the protocols for mining for tin, and if yoiu want it to mine tin you need to add that into your program, its quite a leap to assume typing "make me some paperclips" would cause even a reasonably intelligent computer to start to develop mining and refining protocols all on its own (again didn't say it was impossible, just wildly in the realm of speculation)

I'm not arguing for general intelligence coming about (though it's certainly possible). I think we're at least hundreds of years away. I'm just explaining what would happen if we were able to make it.




What do you think we have calculators for?




The difference being a blind man teaching art is an interesting sideshow.

The development of AGI would be the single most important event in human history.





https://en.wikipedia.org/wiki/Machine_learning

And this aspect of AI isn't sci-fi at all because we already have machines that can learn. It's exactly the point that not everything needs to be explicitly coded in the machine, that it can learn. It can learn that paperclips require an acquisition of tin and it can learn how others have acquired tin and where tin is and how it's extracted, they could extract information on the internet about tin extraction and so on.

There's all sorts of other ways of acquiring tin as well. (force humans into tin mining slavery? construct tin mining robots? develop molecular nanotechnology that converts molecules to tin?).


Your points are very good though, definitely things that need to be considered.

This post was edited by Voyaging on Dec 24 2014 12:36pm