Their findings, shared exclusively with MIT Technology Review, present a worrying development: AI’s knowledge practices danger concentrating energy overwhelmingly within the fingers of some dominant expertise firms.
Within the early 2010s, knowledge units got here from a wide range of sources, says Shayne Longpre, a researcher at MIT who’s a part of the mission.
It got here not simply from encyclopedias and the online, but in addition from sources equivalent to parliamentary transcripts, incomes calls, and climate studies. Again then, AI knowledge units have been particularly curated and picked up from completely different sources to swimsuit particular person duties, Longpre says.
Then transformers, the structure underpinning language fashions, have been invented in 2017, and the AI sector began seeing efficiency get higher the larger the fashions and knowledge units have been. At present, most AI knowledge units are constructed by indiscriminately hoovering materials from the web. Since 2018, the online has been the dominant supply for knowledge units utilized in all media, equivalent to audio, photos, and video, and a spot between scraped knowledge and extra curated knowledge units has emerged and widened.
“In basis mannequin improvement, nothing appears to matter extra for the capabilities than the dimensions and heterogeneity of the information and the online,” says Longpre. The necessity for scale has additionally boosted the usage of artificial knowledge massively.
The previous few years have additionally seen the rise of multimodal generative AI fashions, which may generate movies and pictures. Like massive language fashions, they want as a lot knowledge as potential, and one of the best supply for that has develop into YouTube.
For video fashions, as you possibly can see on this chart, over 70% of information for each speech and picture knowledge units comes from one supply.
This may very well be a boon for Alphabet, Google’s father or mother firm, which owns YouTube. Whereas textual content is distributed throughout the online and managed by many various web sites and platforms, video knowledge is extraordinarily concentrated in a single platform.
