We do have precursors to this that make data auditable but not public, like FedRAMP. Or financial audits. Both were built for a specific problem, both are imperfect, but both are also vastly more effective and close-to-truth than something like the Gartner magic quadrant questionnaire or another quasi-evaluative-quasi-marketing process, which, from your description, seems like it's closer to where independent evals are today.
This would constitute a takings under the Fifth Amendment of proprietary American IP on the order of many, many trillions of dollars for the express purposes of handing it to a declared foreign adversary. That seems...impractical?
I certainly agree it would be impractical! But it’s sometimes helpful to articulate the simple extreme point on a policy spectrum that would actually really solve a particular problem, so we can step back from the weeds of incremental and more practical policies and develop a better shared picture of the North Star those policies should be trying to approximate within their constraints.
I would dispute that the “express purpose” of transparency would be to hand IP to an adversary; Plan A involves a deal with China and we prioritize them being able to verify the deal, but I think the research transparency component would be really helpful even in a purely domestic regulatory regime, and the framing there would be that the loss of competitive edge is a hit worth taking for greater collective understanding of safety.
That makes sense as a way of framing the proposal. It can certainly be helpful to articulate a radical position, but that also lays bare the type of sacrifices the position would need to be willing to stomach.
I agree that the upside of this approach is the benefit of "greater collective understanding of safety". But even the most AI-pilled would have to admit that that's a vague and speculative benefit - one whose gains would accrue over time but whose costs would have to be paid upfront. America is on a completely opposite trajectory right now with regards to policy on maintenance of competitive edge over foreign adversaries, as demonstrated by things like export controls of semiconductor machinery. China is going to continue being regarded as an adversary domestically and any proposed treaty framework should acknowledge that reality.
Thinking about the type of world where something like this activity would be feasible, the scenarios I'm coming up with involve a domestic political crisis on the same scale as the first deployment of nuclear weapons. Notably, that regulatory regime only had the political will to be established after the vivid public demonstration of harm, not before. If (when?) AI causes a crisis and resulting harm on that scale, I could foresee this type of activity being a realistic response. That scenario runs into First-Try problems, but at least offers a potential feasible path.
>America is on a completely opposite trajectory right now with regards to policy on maintenance of competitive edge over foreign adversaries, as demonstrated by things like export controls of semiconductor machinery
I actually think greater research transparency is very synergistic with stronger hardware export controls. I don't think the right way to look at it is a one-dimensional spectrum of "How strict or lax are we being in our efforts to stop China from catching up?," where if we decide we want to be strict we go equally strict on every separate tool we have for stopping China from catching up. Personally, I favor going much stricter on compute and semiconductor manufacturing export controls and much laxer on AI research, because the latter has this important benefit.
> the scenarios I'm coming up with involve a domestic political crisis on the same scale as the first deployment of nuclear weapons
Yeah, the AI 2040 scenario definitely imagines a huge domestic crisis, driven by jobs and further Mythos moments.
We do have precursors to this that make data auditable but not public, like FedRAMP. Or financial audits. Both were built for a specific problem, both are imperfect, but both are also vastly more effective and close-to-truth than something like the Gartner magic quadrant questionnaire or another quasi-evaluative-quasi-marketing process, which, from your description, seems like it's closer to where independent evals are today.
This would constitute a takings under the Fifth Amendment of proprietary American IP on the order of many, many trillions of dollars for the express purposes of handing it to a declared foreign adversary. That seems...impractical?
I certainly agree it would be impractical! But it’s sometimes helpful to articulate the simple extreme point on a policy spectrum that would actually really solve a particular problem, so we can step back from the weeds of incremental and more practical policies and develop a better shared picture of the North Star those policies should be trying to approximate within their constraints.
I would dispute that the “express purpose” of transparency would be to hand IP to an adversary; Plan A involves a deal with China and we prioritize them being able to verify the deal, but I think the research transparency component would be really helpful even in a purely domestic regulatory regime, and the framing there would be that the loss of competitive edge is a hit worth taking for greater collective understanding of safety.
That makes sense as a way of framing the proposal. It can certainly be helpful to articulate a radical position, but that also lays bare the type of sacrifices the position would need to be willing to stomach.
I agree that the upside of this approach is the benefit of "greater collective understanding of safety". But even the most AI-pilled would have to admit that that's a vague and speculative benefit - one whose gains would accrue over time but whose costs would have to be paid upfront. America is on a completely opposite trajectory right now with regards to policy on maintenance of competitive edge over foreign adversaries, as demonstrated by things like export controls of semiconductor machinery. China is going to continue being regarded as an adversary domestically and any proposed treaty framework should acknowledge that reality.
Thinking about the type of world where something like this activity would be feasible, the scenarios I'm coming up with involve a domestic political crisis on the same scale as the first deployment of nuclear weapons. Notably, that regulatory regime only had the political will to be established after the vivid public demonstration of harm, not before. If (when?) AI causes a crisis and resulting harm on that scale, I could foresee this type of activity being a realistic response. That scenario runs into First-Try problems, but at least offers a potential feasible path.
>America is on a completely opposite trajectory right now with regards to policy on maintenance of competitive edge over foreign adversaries, as demonstrated by things like export controls of semiconductor machinery
I actually think greater research transparency is very synergistic with stronger hardware export controls. I don't think the right way to look at it is a one-dimensional spectrum of "How strict or lax are we being in our efforts to stop China from catching up?," where if we decide we want to be strict we go equally strict on every separate tool we have for stopping China from catching up. Personally, I favor going much stricter on compute and semiconductor manufacturing export controls and much laxer on AI research, because the latter has this important benefit.
> the scenarios I'm coming up with involve a domestic political crisis on the same scale as the first deployment of nuclear weapons
Yeah, the AI 2040 scenario definitely imagines a huge domestic crisis, driven by jobs and further Mythos moments.