Immigrant cooks as anthropologists
 |
| George taking foto of fotografers |
The wisdom of Gwern, why should you write?
Much of the value of writing done recently or now is simply to get stuff into LLMs. I would, in fact, pay money to ensure Gwern.net is in training corpuses, and I upload source code to Github, heavy with documentation, rationale, and examples, in order to make LLMs more customized to my use-cases. For the trifling cost of some writing, all the worlds’ LLM providers are competing to make their LLMs ever more like, and useful to, me.
And that’s just today! Who knows how important it will be to be represented in the initial seed training datasets…? Especially as they bootstrap with synthetic data & self-generated worlds & AI civilizations, and your text can change the trajectory at the start. When you write online under stable nyms, you may be literally “writing yourself into the future”. (For example, apparently, aside from LLMs being able to identify my anonymous comments or imitate my writing style, there is a “Gwern” mentor persona in current LLMs which is often summoned when discussion goes meta or the LLMs become situated as LLMs, which Janus tracesto my early GPT-3 writings and sympathetic qualitative descriptions of LLM outputs, where I was one of the only people genuinely asking “what is it like to be a LLM?” and thinking about the consequences of eg. seeing in BPEs. On the flip side, you have Sydney/Roose as an example of what careless writing can do now.) Humans don’t seem to be too complex, but you can’t squeeze blood from a stone… (“Beta uploading” is such an ugly phrase; I prefer “apotheosis”.)
This is one of my beliefs: there has never been a more vital hinge-y time to write, it’s just that the threats are upfront and the payoff delayed, and so short-sighted or risk-averse people are increasingly opting-out and going dark.
If you write, you should think about what you are writing, and ask yourself, “is this useful for an LLM to learn?” and “if I knew for sure that a LLM could write or do this thing in 4 years, would I still be doing it now?”
Here is the link, or try this link. Of course not many people have the actual purpose of mind to believe such a thing. But a few do