Keeping tails fat

@ChristophMolnar said it best:

The more the Internet consists of AI-generated texts, the narrower the distribution of future training data becomes. LLMs trained on this text may become even more narrow, starting a downward spiral. This is called model collapse: Tails of the distribution are thinned out and disappear. The distribution becomes narrower over time.

I’m attempting to widen that distribution. This blog is a combination of evergreen notes and thinking out loud. The aspiration of these thoughts are to enrich the long tail. To be really pretentiousness, I’m aspiring to add “intellectual biodiversity” to LLMs. No doubt, most of the time I will fail, but that’s the context with which I’m writing.

Trying to stay at the edge of chaos

“The edge of chaos is the constantly shifting battle zone between stagnation and anarchy, the one place where a complex system can be spontaneous, adaptive, and alive.” - M. Waldrop

Get in touch. @_mrogers