{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreig7hpkzemhkfz4hszvnrcipbvdno6gah6kznniv6pd4p24sknp6be",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mgvux3wzonz2"
  },
  "path": "/t/valueerror-loading-helsinki-nlp-tokenizers/174192#post_4",
  "publishedAt": "2026-03-12T13:02:44.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "174223"
  ],
  "textContent": "Thank you. Yes, I did get the sentencepiece error when I initially switched to MarianTokenizer.from_pretrained(), but all hunky dory once I installed it.\n\nNow my issue is that these models don’t work with DataParallel anymore, but that is another issue (174223)(throws a StopTermination attempting to access self.model.device in its bowels). I may have to hand-jam my own threads for parallel inference (I had to do that for the ModernBERT models a while back, so code reuse is king!).",
  "title": "ValueError loading Helsinki-NLP tokenizers"
}