評論
打印
Another reason we're told not to worry is that these machines can't help but share our values because they will be literally extensions of ourselves.
另外一個試圖安慰我們的理由是，那些機器必須擁有和我們一樣的價值觀，因為它們將會是我們自身的延伸。
They'll be grafted onto our brains, and we'll essentially become their limbic systems.
它們將會被嫁接到我們的大腦上，我們將會成它們的邊緣系統。
Now take a moment to consider that the safest and only prudent path forward, recommended, is to implant this technology directly into our brains.
現在我們再思考一下最安全的，也是唯一經慎重考慮后推薦的發展方向，是將這項技術直接植入我們大腦。
Now, this may in fact be the safest and only prudent path forward,
這也許確實是最安全的，也是唯一慎重的發展方向，
but usually one's safety concerns about a technology have to be pretty much worked out before you stick it inside your head.
但通常在我們把它塞進腦袋之前，會充分考慮這項技術的安全性。
The deeper problem is that building superintelligent AI on its own seems likely to be easier than
更深一層的問題是：僅僅制造出超級人工智能機器可能要比
building superintelligent AI and having the completed neuroscience that allows us to seamlessly integrate our minds with it.
既制造超級人工智能，又讓其擁有能讓我們的思想和超級人工智能無縫對接的完整的神經科學系統要簡單很多。
And given that the companies and governments doing this work are likely to perceive themselves as being in a race against all others,
而做這些研究的公司或政府，很可能將彼此視作競爭對手，
given that to win this race is to win the world, provided you don't destroy it in the next moment,
因為贏得了比賽就意味著稱霸了世界，前提是不在剛成功后就將其銷毀，
then it seems likely that whatever is easier to do will get done first.
所以結論是：簡單的選項一定會被先實現。
Now, unfortunately, I don't have a solution to this problem, apart from recommending that more of us think about it.
但很遺憾，除了建議更多人去思考這個問題，我對此并無解決方案。
I think we need something like a Manhattan Project on the topic of artificial intelligence.
我覺得在人工智能問題上，我們需要一個“曼哈頓計劃”。
Not to build it, because I think we'll inevitably do that,
不是用于討論如何制造人工智能，
but to understand how to avoid an arms race and to build it in a way that is aligned with our interests.
因為我們一定會這么做，而是去避免軍備競賽，最終以一種有利于我們的方式去打造它。
When you're talking about superintelligent AI that can make changes to itself,
當你在談論一個可以自我改造的超級人工智能時，
it seems that we only have one chance to get the initial conditions right,
我們似乎只有一次正確搭建初始系統的機會，
and even then we will need to absorb the economic and political consequences of getting them right.
而這個正確的初始系統需要我們在經濟以及政治上做出很大的努力。
But the moment we admit that information processing is the source of intelligence,
但是當我們承認信息處理是智慧的源頭，
that some appropriate computational system is what the basis of intelligence is,
承認一些電腦系統是智能的基礎，
and we admit that we will improve these systems continuously,
承認我們會不斷改善這些系統，
and we admit that the horizon of cognition very likely far exceeds what we currently know,
承認我們現存的認知遠沒有達到極限，將很可能被超越，
then we have to admit that we are in the process of building some sort of god.
我們又必須同時承認我們在某種意義上正在創造一個新的“上帝”。
Now would be a good time to make sure it's a god we can live with. Thank you very much.
現在正是思考人類是否能與這個“上帝”和睦相處的最佳時機。非常感謝！
