ChatGLM-6b初步微调实验

chatglm-6b微调/推理, 样本为自动生成的整数/小数加减乘除运算, 可gpu/cpu
chatglm-6b fine-tuning/inference, The sample is an automatically generated, integer/decimal of add, sub, mul and div operation, that can be gpu/cpu

项目地址

https://github.com/yongzhuo/chatglm-maths

数据集-中文

• https://github.com/tatsu-lab/stanford_alpaca
• https://github.com/LianjiaTech/BELLE
• https://github.com/carbonz0/alpaca-chinese-dataset

踩坑

1. eps=1e-5(不要改小), 单精度float16, 以及LN采用的是Sandwich-LN(Sandwich LayerNorm), 分支的ATtention前后都有LN, 目的是大模型为了防止梯度溢出等;
2. 模型输入输出, 默认的tokenization_chatglm.py/modeling_chatglm.py不能用, 因为那是完全为生成generate设置的, 需要自己写好所有缩入参数, 或者机子改成适配的;2.1 ChatGLMModel中, get_masks()正常, get_position_ids()函数中‘context_length = seq.index(150004) + 1’ 改为 ‘context_length = len(seq)’;2.2 训练输入input_ids格式暂定为(训练后post-padding, 推理前pre-padding[tokenization_chatglm.py默认pre-padding])a1. x1: [CLS] + prompt_1 + " " + text_1 + " " + prompt_2 + [gMASK] + [PAD]*N(post-padding)a2. x2: [SOP] + " " + text_2 + [PAD]*N(post-padding)a.  x = x1 + x22.3 训练输入label_ids格式暂定为(CrossEntropyLoss默认忽略-100不参与计算loss)  b.  y = [-100]*len(x) + " " + text_2 + [EOP] + [-100]*N(post-padding)2.4 可参考GLM-1, https://github.com/THUDM/GLM/blob/main/tasks/seq2seq/dataset.py
3. 注意chatglm-6b权重是float16的, 不过计算loss时候会转成float32计算, 最后loss再转回float16更新梯度;
4. ChatGLMTokenizer有时候会报奇奇怪怪的错误, 建议生成时候设置max_new_tokens, 最大{"max_new_tokens": 2048}; decode有时候会出现不存在id;

环境配置

transformers>=4.26.1
cpm_kernels==1.0.11
icetk==0.0.4
torch>=1.10.1
rouge==1.0.1
nltk==3.6.6
numpy
tqdmlion_pytorch

微调-计算题

6b
微调: python c00_toy_cpu_train_6b.py
推理: python p00_toy_cpu_predit_6b.pysmall-layer
微调: python c01_toy_cpu_train_small.py
推理: python p01_toy_cpu_predict_small.py

参考/感谢

• https://github.com/THUDM/ChatGLM-6B
• https://github.com/THUDM/GLM
• https://github.com/tatsu-lab/stanford_alpaca
• https://github.com/LianjiaTech/BELLE
• https://github.com/huggingface/peft
• https://github.com/mymusise/ChatGLM-Tuning
• https://github.com/bojone/bert4keras

推理日志toy

generator_calculate_line: ('13+75=', '13+75=88')
tokenizer.vocab_size: 150344
eval:   0%|                                                                                                                                                                      | 0/1 [00:00

generator_calculate_line: ('13+75=', '13+75=88')
tokenizer.vocab_size: 150344
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:10<00:00,  1.31s/it]
transformer.word_embeddings.weight False
......
transformer.layers.26.mlp.dense_4h_to_h.bias False
transformer.layers.27.input_layernorm.weight True
transformer.layers.27.input_layernorm.bias True
transformer.layers.27.attention.query_key_value.weight True
transformer.layers.27.attention.query_key_value.bias True
transformer.layers.27.attention.dense.weight True
transformer.layers.27.attention.dense.bias True
transformer.layers.27.post_attention_layernorm.weight True
transformer.layers.27.post_attention_layernorm.bias True
transformer.layers.27.mlp.dense_h_to_4h.weight True
transformer.layers.27.mlp.dense_h_to_4h.bias True
transformer.layers.27.mlp.dense_4h_to_h.weight True
transformer.layers.27.mlp.dense_4h_to_h.bias True
transformer.final_layernorm.weight True
transformer.final_layernorm.bias True
model.chat start
13+75=88, but that's not the correct answer. The correct answer is 13+75=88, which is 90.
/anaconda3/envs/py371/lib/python3.7/site-packages/transformers/optimization.py:395: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warningFutureWarning,   
epoch:   0%|                                                                                                                                                                    | 0/21 [00:00