As noted by Google, the rise in algorithmic advances has been partly related to the availability of enormous, uncurated datasets that help fuel sophisticated machine learning systems. Since these datasets are scraped from the internet, they reflect some of our worst tendencies like “social stereotypes, and oppressive viewpoints.”
Using a self-developed benchmark known as DrawBench, Google found that Imagen was rated more favorably compared to existing text-to-image AI, like the aforementioned DALL-E 2 system. Drawbench is simply a side-by-side human evaluation, with the same text prompt being fed to both Imagen and the point of comparison. Users were asked to evaluate both images, and Imagen was largely preferred.