AutoModel with ClinicalBERT gives UNEXPECTED warning
I’m using the Python ‘transformers’ package, version 5.3.0 to interact with an existing graph database. The script seems to work, giving me sensible output, but produces a warning that concerns me.
The warning is produced when this statement is executed:
model = AutoModel.from_pretrained(“emilyalsentzer/Bio_ClinicalBERT”, token=HF_TOKEN)
The text of the warning is as follows:
Loading weights: 100%|
BertModel LOAD REPORT from: emilyalsentzer/Bio_ClinicalBERT Key | Status | | -------------------------------------------±-----------±-± cls.predictions.transform.dense.weight | UNEXPECTED | | cls.predictions.transform.LayerNorm.bias | UNEXPECTED | | cls.predictions.transform.LayerNorm.weight | UNEXPECTED | | cls.predictions.bias | UNEXPECTED | | cls.seq_relationship.weight | UNEXPECTED | | cls.seq_relationship.bias | UNEXPECTED | | cls.predictions.transform.dense.bias | UNEXPECTED | | cls.predictions.decoder.weight | UNEXPECTED | |
Notes:
- UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
I don’t understand what is meant by the phrase “when loading from different task/architecture”. Can someone please explain it? I’m running Python version 3.13.6 in a venv on a Macbook M1 processor.
What changes do I need to make to my system/environment such that this ‘UNEXPECTED’ issue is resolved?
Discussion in the ATmosphere