Being creative requires taking risks (www.henrikkarlsson.xyz)

A recent reflection highlights the challenges of maintaining creativity in both artistic and machine learning fields, drawing parallels between human learning and the phenomenon known as mode collapse in generative models. As children progress in drawing, their initial creativity often diminishes as they conform to established norms, leading to predictable outputs. Similarly, in machine learning, particularly with Generative Adversarial Networks (GANs) and language models, there is a trend where models begin to produce homogeneous outputs that reflect familiar patterns rather than innovative structures, stifling diversity and novelty in their generated content. This encapsulates a broader existential discussion about the importance of taking creative risks to avoid stagnation. Figures like Brian Eno exemplify this ethos by consistently engaging in projects that challenge norms, even at the cost of commercial success. The counsel emerging from this discourse advocates for a conscious effort to seek novel information and engage with differing viewpoints, especially as one matures in their career. For the AI/ML community, this underscores a crucial lesson: to foster genuine innovation, researchers and practitioners must actively resist the comfort of repetition and cultivate environments that encourage exploration and risk-taking, even as they navigate the trade-offs associated with maintaining a sustainable career.