Very well put, but it doesn't really address the critical failure mode of optimizing for an inexact or simplified goal, which is the critical ‘value alignment' issue. Saying that "it forces us to concede that moral philosophy is more than ticking boxes" seems to ignore the way that *any* choice made by a strong optimizer with an articulated goal is inevitably misaligned, not just according to some views, but according to all of them. Despite its utility at doing more than ticking boxes, and our need to accept that fact, moral philosophy historically doesn't address the problem of unavoidable misalignment at all!
A well-reasoned exploration and argument. The fact that AI is a human invention, enmeshed in language and human interpretation, is significant. People underestimate the amount of human agency that is embedded in these models and then reinterpreted by other humans in their application.
Very well put, but it doesn't really address the critical failure mode of optimizing for an inexact or simplified goal, which is the critical ‘value alignment' issue. Saying that "it forces us to concede that moral philosophy is more than ticking boxes" seems to ignore the way that *any* choice made by a strong optimizer with an articulated goal is inevitably misaligned, not just according to some views, but according to all of them. Despite its utility at doing more than ticking boxes, and our need to accept that fact, moral philosophy historically doesn't address the problem of unavoidable misalignment at all!
Surely true, but I think more than enough has been written about that!
A well-reasoned exploration and argument. The fact that AI is a human invention, enmeshed in language and human interpretation, is significant. People underestimate the amount of human agency that is embedded in these models and then reinterpreted by other humans in their application.