That discussion is now being had and will be key to monitor in the months and years ahead.
There are also many ethical questions raised around the licensing of communally created data being repurposed as training data. Reddit has signed a $60 million/year deal with Google to sell its switzerland whatsapp number data users data to train AI models, while Automattic, owners of Tumblr and WordPress.com, are moving down this pathway with OpenAI. The onus appears to be on individual users to “opt out” if they don’t want to take part, but is that practical or reasonable to expect people to do? This is a use case that would not have been clear to anyone who launched a Tumblr blog in 2007!
Some ethicists have taken a very strong stance against the data broker economy itself, saying it should be abolished. Spanish philosopher Carissa Véliz covers this in her book “Privacy is Power”:
“The logic of the data economy is a perverse one: mine as much personal data from people as possible, at any cost. And we are paying too steep a price for digital tech. We are no longer treated as equals; we are each treated according to our data. We don’t see the same content, we don’t pay the same price for the same product, and we are not offered the same opportunities. The data economy is undermining equality.”