AI Models
MIST 1 140B 4bit
MIST family · open weights on Hugging Face.
4-bit NF4 quantised release of MIST-1-140B: 140B-scale weights on accessible hardware.
TransformersSafetensorsLlamaText Generation4 BitQuantizedNf4BitsandbytesLlama 3.1MistConversationalEnglishBase: olaverse/MIST-1-140BBase: quantized:olaverse/MIST-1-140BLicence: llama3.1Endpoints Compatible
0
Likes
178
Downloads
May 2026
Created
/ Model card
Built for production use
Open-weights repository on Hugging Face
Integrated ecosystem protocol tier.
Compatible with Transformers library
Integrated ecosystem protocol tier.
Optimised for low-latency inference
Integrated ecosystem protocol tier.
Community engagement: 0 likes
Integrated ecosystem protocol tier.
/ Quick start
Use MIST 1 140B 4bit, straight from its model card.
example.pyfrom the model card
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type='nf4'
)
model = AutoModelForCausalLM.from_pretrained(
"olaverse/MIST-1-140B-4bit",
quantization_config=quantization_config,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("olaverse/MIST-1-140B-4bit")
messages = [{"role": "user", "content": "Your question here"}]
text = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
/ Model card
Official repository README
Dynamically loaded from Hugging Face
Model card metadata is available on Hugging Face.
/ Built with
Transformers PyTorch Python
Ready to try MIST 1 140B 4bit?
4-bit NF4 quantised release of MIST-1-140B: 140B-scale weights on accessible hardware.
/ Model ecosystem