cannot find tensor cls.predictions.decoder.weight

[jetnet]

System Info

System Info:

# Ubuntu 24.04.2 LTS
# podman version 4.9.3

podman run --device nvidia.com/gpu=all --rm --gpus all nvidia/cuda:12.8.1-base-ubuntu22.04 nvidia-smi -L
GPU 0: NVIDIA L40 (UUID: GPU-xxx)
GPU 1: NVIDIA L40 (UUID: GPU-xxx)

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

Trying to start the model:

image="ghcr.io/huggingface/text-embeddings-inference:89-1.6"

model=google-bert/bert-base-german-dbmdz-uncased
name=german-dbmdz-uncased

podman run -d \
  --device nvidia.com/gpu=all \
  -e CUDA_VISIBLE_DEVICES=0,1 \
  -v $modeldir:/data \
  -p 18080:80 \
  --name $name $image \
  --model-id $model --pooling splade

Error:

ERROR text_embeddings_backend: backends/src/lib.rs:388:
Could not start Candle backend:
Could not start backend: cannot find tensor cls.predictions.decoder.weight

Expected behavior

container up and running