【Hackathon 8th No.29】在 PaddleNLP 中复现 ModernBERT 模型

[robinbg]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

PR changes

Description

[paddle-bot]

Thanks for your contribution!

[DrownFish19]

请将文件通过pre-commit 处理后再补充提交以统一格式，可参考以下命令：

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

[DrownFish19]

1. 需要在XXXPretrainedModel中补充_get_name_mappings (支持参数转换)、_get_tensor_parallel_mappings （支持模型并行参数切分）、_get_fuse_or_split_param_mappings（支持参数自动化融合切分）。
2. 需要参考Qwen2模型支持并行策略以支持模型训练。