Skip to content

Big difference between the before-cooldown-ckpt and the final checkpoint in the results of downstream tasks? #104

@siqi13579

Description

@siqi13579

We tested the checkpoint before cooldown and the final checkpoint of stable lm2 1.6b , and found that there was a big difference in the results of downstream tasks. The final checkpoint significantly improved the results of downstream tasks. Are there any special strategies for the cooldown phase?
arc(25shot) hellaswag(10shot) mmlu(5-shot) truthfulqa winnogrande(5-shot) gsm(5-shot)
43.52 70.3 39.8 36.61 64.17 17.29
38.4 67.59 30 34.9 61.96 7.35

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions