← MIC coverage of Transformers model types

state-spaces/mamba-130m-hf

huggingface-mic/mamba-130m-hf-mic@ba38c62c9ee6 ← state-spaces/mamba-130m-hf@1e76775f628f · MambaForCausalLM · random weights

CI: no failure on main in 30 days · MIC failures dashboard

Original129.1Mparameters
Fixture3.3Mparameters
Layer stacks1compared
Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

Model tester (advisory)

The model's own small test config in its ModelTester (16 literal values, 16 set by the model's own tester, the rest generic base-tester defaults). Testers are usually representative but not always, so these notes are for review, not rules.

The fixture covers every per-layer kind the tester sets, and shrinks every size it does.

Layer stacks

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

backbone.layers

Original · 24
Fixture · 2
KindOriginalFixture
Block242✓ kept

Where the parameters are

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original
Fixture
ComponentOriginalShareFixtureShare
backbone.layers90.5M70.1%87.9K2.7%
backbone.embeddings38.6M29.9%3.2M97.3%
backbone.norm_f7680.0%640.0%

Preserved properties

ConfigPropertyOriginalFixture
modelmodel_typemambamamba✓ kept
modelvocab_size5028050280✓ kept
modeltie_word_embeddingsTrueTrue✓ kept
modelhidden_actsilusilu✓ kept

Module classes

Instances of each class in the built model (prefix Mamba omitted).

ClassOriginalFixture
Linear979
RMSNorm253
Conv1d242
Block242
Mixer242
SiLUActivation242
Embedding11
ForCausalLM11
Model11
ModuleList11

Config changes

5 fields changed, 38 unchanged.

FieldOriginalFixture
d_model76864
hidden_size76864
intermediate_size1536128
n_layer242
num_hidden_layers242