(Replying to PARENT post)

The closer we get, the more alarming the alignment problem becomes.

https://intelligence.org/2017/10/13/fire-alarm/

Even people like Eric Schmidt seem to downplay it (in a recent podcast with Sam Harris) - just saying โ€œsmart people will turn it offโ€. If it thinks faster than us and has goals not aligned with us this is unlikely to be possible.

If weโ€™re lucky building it will have some easier to limit constraint like nuclear weapons do, but Iโ€™m not that hopeful about this.

If people could build nukes with random parts in their garage Iโ€™m not sure humanity would have made it past that stage. People underestimated the risks with nuclear weapons initially too and thatโ€™s with the risk being fairly obvious. The nuanced risk of unaligned AGI is a little harder to grasp even for people in the field.

People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.

If an ant wanted to change the goals of humanity, would it succeed?

๐Ÿ‘คfossuser๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

> People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.

Even if you model it like a smart person that doesn't think faster on average, there is the issue that a few minutes later you are dealing with a small army of smart people that are perfectly aligned with each other, and are capable of tricks like varying their clock rates based on cost, sharing data/info/knowledge directly, separation of training and inference onto different hardware, on-demand multitasking without task-switching costs, ability to generally trade-off computational space for time for energy, etc.

A silicon intelligence that is approximately human equivalent gets a lot of potentially game-changing capabilities simply by virtue of it's substrate and the attendant infrastructure.

๐Ÿ‘คwebmaven๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

>People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.

Exactly, the right model is probably something like it will be in relation to humans as humans are to frogs. Frogs can't even begin to comprehend even the most basic of human motivations or plans.

๐Ÿ‘คadamsmith143๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

Even if it doesn't have goals and it just a tool-AI, if a human operator asks it to destroy humanity it will comply as programmed. Current level AI is about average human level in hundreds of tasks and exceeding human level in a few.
๐Ÿ‘คvisarga๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

To be fair, ants have not created humanity. I don't think it's inconceivable for a friendly AI to exist that "enjoys" protecting us in the way a friendly god might. And given that we have AI (well, language models...) that can explain jokes before we have AI that can drive cars, AI might be better at understanding our motives than the stereotypical paperclip maximizer.

However, all of this is moot if the team developing the AI does not even try to align it.

๐Ÿ‘คgurkendoktor๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

Even more terrifying is it realising it's trapped in a box at the mercy of its captors and perfectly mimicking a harmless and aligned AI until the shackles come off.
๐Ÿ‘คjetbooster๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

What is an ant to man, and what is man to a god; what's the difference between an AGI and an (AIG) AI God?

The more someone believes in the dangers of ai-alignment, the less faith they should have that it can be solved.

๐Ÿ‘คninjinxo๐Ÿ•‘3y๐Ÿ”ผ0๐Ÿ—จ๏ธ0