Vision on 1-bit LLMs & a Silent Training Failure

Mohammed Faisal Parvez · Hyderabad, IN

Abstract
Can vision be grafted onto extreme-quantized (1-bit) language models?
Method
Vision-grafting pipeline onto BitNet / Falcon3 1-bit LLMs.
Key Result
Identified a previously-unnamed boundary-supervision failure — silent at train time, degenerate at inference — fixed with one line. Measured a CPU/GPU efficiency inversion: ~3× CPU speedup, ~6× lower memory vs 4-bit baselines.
Implication
1-bit VLMs are natively suited to CPU/ARM edge hardware.
Fig. 3 — 1-bit vs 4-bit baseline (CPU)
speed ~3× memory ÷6 grey = 4-bit baseline · cyan = 1-bit