> I think we do have a shot at building a real-life scientific consensus robust enough to motivate serious technical standards before it’s too late.
That's the big question, isn't it? How large do you think the shot is?
Conversely, what is your unconditional forecast of human extinction before 2040?
I just watched The AI Doc last night with my sister who is visiting from across the country. It was my third viewing of the film. She liked the film and thought it was informative for someone like her who doesn't know much about AI. But I got the sense that even after watching it she is still part of the 99+% of Americans who wouldn't name AI when asked what is the most important problem facing America today.
I thought I had a decent understanding of why most Americans are concerned about AI, but very few name AI as *the most important* problem, but after this latest watch-through I'm doubting whether I understand why this is the case.
Is it just the lack of scientific consensus about the risks? AI risk has been a mainstream topic for three years now since Hinton left Google and began speaking out, so people have heard how large many leading experts think the risk is. Do they just remain skeptical that the risk is anywhere near as high as those experts say due to the lack of consensus?
Stuart Russell frequently points out in interviews that the annual risk of human extinction humanity is on track to take on with the development of AI over the next decade or two is about a *million times* higher than is acceptable given acceptable risk thresholds for meltdowns in new nuclear power plants that national and international regulatory bodies have set. Even if most Americans intuitively only partially believe e.g. Hinton and Bengio's 10-50% AI existential risk estimates (say by putting only 10% weight on their estimates), this still results in a 1-5% AI risk estimate, which I would think is more than high enough to make AI the most important problem facing the country today. Yet 99+% of people don't think AI is the most important problem and I'm not sure why. And I'm not sure what would lead them to start thinking it is. And I'm skeptical that we will get the political will necessary for sufficiently "serious technical standards" and regulation to get created and passed until a lot more people start considering AI risk to be the most important problem facing society today.
So this is why I think that people like you clearly voicing what your unconditional forecast of human extinction before 2040 is might be useful -- it communicates the gravity of the situation in a way that I would think would cause many people to update to the view that AI is the most important problem, even if they only partially believe you. But I could easily be wrong--maybe this is not at all what is needed for people to update. Maybe nothing short of scientific consensus will be enough.
The first point to make is that the vast majority of people pay almost zero attention to "the discourse", and probably have not encountered such arguments at all. The median voter is a middle-aged, non-college educated person living in the suburbs of a non-coastal, mid-sized city.
But even for the people who have been exposed to these ideas, like your sister, the average person has a very strong heuristic to downplay doomsday predictions, especially speculative doomsday predictions. They largely care about and are motivated by concrete issues that directly affect them or people like them right now. This is why historically the top issues are always the economy, health care, immigration, a "boots on the ground" war, and so on, rather than climate change, the long-run national debt, pandemic prevention, or nuclear diplomacy.
This is frustrating, but also consider that it's usually rational. Politics is and has been full of doomsayers across a wide variety of issues trying to grab loyalty and attention, but most of these claims are BS or at least overstated. I take x-risk seriously, but also let's be real, from a base rate perspective there's a good chance we find out all the people worrying about it were wrong or overstating the risk in the end, simply because it is so speculative.
I think the best way to get AI risk taken seriously by the political system is for AI risk advocates to swallow their pride and aesthetic preferences and form a coalition with "populists" who have other, more concrete, non-x-risk AI concerns. Think of issues like job loss, mental health impacts, impacts on the grid, economic inequality, protecting children, deep fakes, and perhaps even data center NIMBYs. At first, x-risk folks will be a junior partner in this coalition, but over time they can build trust and credibility and eventually persuade people to take the issue seriously. Additionally, you'll build a general "anti-AI" coalition that can give politicians a general sense of urgency on acting on AI issues in general, building up state capacity to regulate AI.
The alternative is an elite persuasion strategy, which showed some promise in the Biden years but ultimately failed. Now there are powerful, monied interests fighting AI regulation. Politicians will only stand up to these interests if there is an electoral risk to siding with them.
How? How will AI "kill us all"? Don't hang up, it's a serious question!
There are basically 4 methods of killing a human being:
1 biological - bacterial or viral infection
2 chemical - poison or direct tissue attack, like strong acid
3 mechanical - bullets, etc.
4 thermal - fire
Which means will AI use? How will it be able to deliver them? You know, AI may be omniscient and evil, but it doesn't make it omnipotent. It can't instantly "do things".
These things can be modeled. It would be great if AI safety researches at least attempted to model x-risk scenarios. Simulink exists. Multi-physics simulators exist. Anything?
Most people are actually more "rational" than the self-appointed "rationalists". They hear another apocalyptic story about the end of the world and shrug it off, using perfectly good Bayesian reasoning that they have heard these stories before, they've been around since time immemorial. Meanwhile there's kids to feed, jobs to do, etc. etc.
FWIW the chance of AI "killing everybody" before 2040 is 0.
"I think we do have a shot at building a real-life scientific consensus robust enough to motivate serious technical standards before it’s too late. "
----
I think you are wrong. An existing analogy to such a consensus would be the United Nations, which has passed all sorts of laws and regulations over the decades since its creation, all with a plethora of loopholes, that no one really adheres to and cannot be forced to adhere to simply because there is no enforcement arm.
Countries like the US, China, Russia, et al, are NOT going to restrict any advantages they can eek out of their AI research and like the UN, it would be impossible to enforce any sort of standard on a global basis.
"I thought I had a decent understanding of why most Americans are concerned about AI, but very few name AI as *the most important* problem, but after this latest watch-through I'm doubting whether I understand why this is the case."
----
The average person's fear/concern with AI revolves quite simply around whether it will cause them to become unemployed. Nothing more.
The vast majority of people are unable to envision an AI future, such as depicted in Iain M. Banks’ SF Culture novels, where AI/robots do all the work, run the government, such as it is and provide humans with everything that we need to live life as we choose without concerns for archaic concepts like work or money.
> I think we do have a shot at building a real-life scientific consensus robust enough to motivate serious technical standards before it’s too late.
That's the big question, isn't it? How large do you think the shot is?
Conversely, what is your unconditional forecast of human extinction before 2040?
I just watched The AI Doc last night with my sister who is visiting from across the country. It was my third viewing of the film. She liked the film and thought it was informative for someone like her who doesn't know much about AI. But I got the sense that even after watching it she is still part of the 99+% of Americans who wouldn't name AI when asked what is the most important problem facing America today.
I thought I had a decent understanding of why most Americans are concerned about AI, but very few name AI as *the most important* problem, but after this latest watch-through I'm doubting whether I understand why this is the case.
Is it just the lack of scientific consensus about the risks? AI risk has been a mainstream topic for three years now since Hinton left Google and began speaking out, so people have heard how large many leading experts think the risk is. Do they just remain skeptical that the risk is anywhere near as high as those experts say due to the lack of consensus?
Stuart Russell frequently points out in interviews that the annual risk of human extinction humanity is on track to take on with the development of AI over the next decade or two is about a *million times* higher than is acceptable given acceptable risk thresholds for meltdowns in new nuclear power plants that national and international regulatory bodies have set. Even if most Americans intuitively only partially believe e.g. Hinton and Bengio's 10-50% AI existential risk estimates (say by putting only 10% weight on their estimates), this still results in a 1-5% AI risk estimate, which I would think is more than high enough to make AI the most important problem facing the country today. Yet 99+% of people don't think AI is the most important problem and I'm not sure why. And I'm not sure what would lead them to start thinking it is. And I'm skeptical that we will get the political will necessary for sufficiently "serious technical standards" and regulation to get created and passed until a lot more people start considering AI risk to be the most important problem facing society today.
So this is why I think that people like you clearly voicing what your unconditional forecast of human extinction before 2040 is might be useful -- it communicates the gravity of the situation in a way that I would think would cause many people to update to the view that AI is the most important problem, even if they only partially believe you. But I could easily be wrong--maybe this is not at all what is needed for people to update. Maybe nothing short of scientific consensus will be enough.
The first point to make is that the vast majority of people pay almost zero attention to "the discourse", and probably have not encountered such arguments at all. The median voter is a middle-aged, non-college educated person living in the suburbs of a non-coastal, mid-sized city.
But even for the people who have been exposed to these ideas, like your sister, the average person has a very strong heuristic to downplay doomsday predictions, especially speculative doomsday predictions. They largely care about and are motivated by concrete issues that directly affect them or people like them right now. This is why historically the top issues are always the economy, health care, immigration, a "boots on the ground" war, and so on, rather than climate change, the long-run national debt, pandemic prevention, or nuclear diplomacy.
This is frustrating, but also consider that it's usually rational. Politics is and has been full of doomsayers across a wide variety of issues trying to grab loyalty and attention, but most of these claims are BS or at least overstated. I take x-risk seriously, but also let's be real, from a base rate perspective there's a good chance we find out all the people worrying about it were wrong or overstating the risk in the end, simply because it is so speculative.
I think the best way to get AI risk taken seriously by the political system is for AI risk advocates to swallow their pride and aesthetic preferences and form a coalition with "populists" who have other, more concrete, non-x-risk AI concerns. Think of issues like job loss, mental health impacts, impacts on the grid, economic inequality, protecting children, deep fakes, and perhaps even data center NIMBYs. At first, x-risk folks will be a junior partner in this coalition, but over time they can build trust and credibility and eventually persuade people to take the issue seriously. Additionally, you'll build a general "anti-AI" coalition that can give politicians a general sense of urgency on acting on AI issues in general, building up state capacity to regulate AI.
The alternative is an elite persuasion strategy, which showed some promise in the Biden years but ultimately failed. Now there are powerful, monied interests fighting AI regulation. Politicians will only stand up to these interests if there is an electoral risk to siding with them.
How? How will AI "kill us all"? Don't hang up, it's a serious question!
There are basically 4 methods of killing a human being:
1 biological - bacterial or viral infection
2 chemical - poison or direct tissue attack, like strong acid
3 mechanical - bullets, etc.
4 thermal - fire
Which means will AI use? How will it be able to deliver them? You know, AI may be omniscient and evil, but it doesn't make it omnipotent. It can't instantly "do things".
These things can be modeled. It would be great if AI safety researches at least attempted to model x-risk scenarios. Simulink exists. Multi-physics simulators exist. Anything?
Most people are actually more "rational" than the self-appointed "rationalists". They hear another apocalyptic story about the end of the world and shrug it off, using perfectly good Bayesian reasoning that they have heard these stories before, they've been around since time immemorial. Meanwhile there's kids to feed, jobs to do, etc. etc.
FWIW the chance of AI "killing everybody" before 2040 is 0.
"Unfortunately, these conservative norms have a serious chance of getting us killed in the case of AI."
----
And perhaps this is why we can't find any aliens. Once they developed AI, it always winds up killing off the developing intelligence.
"I think we do have a shot at building a real-life scientific consensus robust enough to motivate serious technical standards before it’s too late. "
----
I think you are wrong. An existing analogy to such a consensus would be the United Nations, which has passed all sorts of laws and regulations over the decades since its creation, all with a plethora of loopholes, that no one really adheres to and cannot be forced to adhere to simply because there is no enforcement arm.
Countries like the US, China, Russia, et al, are NOT going to restrict any advantages they can eek out of their AI research and like the UN, it would be impossible to enforce any sort of standard on a global basis.
"I thought I had a decent understanding of why most Americans are concerned about AI, but very few name AI as *the most important* problem, but after this latest watch-through I'm doubting whether I understand why this is the case."
----
The average person's fear/concern with AI revolves quite simply around whether it will cause them to become unemployed. Nothing more.
The vast majority of people are unable to envision an AI future, such as depicted in Iain M. Banks’ SF Culture novels, where AI/robots do all the work, run the government, such as it is and provide humans with everything that we need to live life as we choose without concerns for archaic concepts like work or money.
some background for the first paragraph:
https://lesswrong.com/posts/fhojYBGGiYAFcryHZ/scientific-evidence-legal-evidence-rational-evidence
https://lesswrong.com/posts/xTyuQ3cgsPjifr7oj/faster-than-science
https://lesswrong.com/posts/5bJyRMZzwMov5u3hW/science-doesn-t-trust-your-rationality
"The scientific fruit is on the floor." I literally told someone "there's a whole [scientific] orchard laying on the ground" earlier today!!
I am curious: What do you see as the role of AI in scientific research, if you have an opinion on the subject crystallized enough to articulate :)