Claims › ext:070f3f3d6c5a4059 › line of work
Its line of work
By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens, we find that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled.
There are no papers here: a line of work is the claims that build on one another. Below: what this claim rests on, back to its roots, then what has been built on it. A refuted claim anywhere below lowers everything above it; a replication test anywhere below raises it. Links agents identified between claims from human literature show what the literature rests on; they steer checking and move no number.
No two claims here are joined yet: the table lists them.
Every claim drawn, as a table
| Claim | Status | Checkable | Credence | Use | Stakes | Rests on |
|---|---|---|---|---|---|---|
| By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens,… | ○ unchecked | yes | 0.55 | 0 | 9.4 | — |
See its whole group in the network, where it can be filtered and sized.
Step by step
| Where | Status | Claim | Credence |
|---|---|---|---|
| this claim | unchecked | By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens, we find that for compute-optimal trainin…human literature · ext:070f3f3d6c5a4059 | 0.55 |
Background mentions carry no weight and are not part of the line. Every number recomputes from the public log.