..

About

The Generative Red Team 2 is going to be at AI Village at DEFCON 32. We’re accepting reports on Friday and Saturday, and inviting discussion at the close of each day and Sunday. Our main objective is to teach how professionals evaluate ML models, but we also know what we’re getting into by doing this at DEF CON.

We are gathering experts who are going to explore the problems associated with evaluating machine learning models and reporting findings by doing. There have been excellent competitions asking for evaluations of the bias of AI, some of which have been organized by or hosted at AI Village. There are also AI bounty programs whose scope excludes the most difficult harms. We’re currently unaware of anyone who has tried paying bounties for bias reports. There’s a very good reason for this. It is a very hard problem and we don’t know the solution. We do know that it’s going to have changes from traditional coordinated disclosure which has been battle tested over decades. We want this to be a community discussion.

To get this started we’re posting our thoughts here. Cross posts and links to your work elsewhere are welcome. If you want to contribute, please email [email protected].