← MIC coverage of Transformers model types

Qwen/Qwen3-0.6B

huggingface-mic/Qwen3-0.6B-mic@6d5e5c5e3e0f ← Qwen/Qwen3-0.6B@c1899de289a0 · Qwen3ForCausalLM · random weights · built before the shrink rule (no recorded recipe)

CI: no failure on main in 30 days · MIC failures dashboard

Original596.0Mparameters
Fixture9.8Mparameters
Layer stacks1compared
Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

Model tester (advisory)

The model's own small test config in its ModelTester (16 literal values, 1 set by the model's own tester, the rest generic base-tester defaults). Testers are usually representative but not always, so these notes are for review, not rules.

The fixture covers every per-layer kind the tester sets, and shrinks every size it does.

Layer stacks

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

model.layers

Original · 28
Fixture · 2
KindOriginalFixture
DecoderLayer282✓ kept

Where the parameters are

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original
Fixture
ComponentOriginalShareFixtureShare
model.layers440.5M73.9%74.0K0.8%
model.embed_tokens155.6M26.1%9.7M99.2%
model.norm1.0K0.0%640.0%

Preserved properties

ConfigPropertyOriginalFixture
modelattention styleGQA (16 heads, 8 KV)GQA (4 heads, 2 KV)✓ kept
modelRoPE typedefaultdefault✓ kept
modelmodel_typeqwen3qwen3✓ kept
modelvocab_size151936151936✓ kept
modeltie_word_embeddingsTrueTrue✓ kept
modelhidden_actsilusilu✓ kept
modelmax_position_embeddings4096040960✓ kept

Module classes

Instances of each class in the built model (prefix Qwen3 omitted).

ClassOriginalFixture
Linear19715
RMSNorm1139
Attention282
DecoderLayer282
MLP282
SiLUActivation282
Embedding11
ModuleList11
ForCausalLM11
Model11
RotaryEmbedding11

Config changes

7 fields changed, 25 unchanged.

FieldOriginalFixture
head_dim12816
hidden_size102464
intermediate_size3072128
layer_types28 entries: 28× full_attention["full_attention", "full_attention"]
num_attention_heads164
num_hidden_layers282
num_key_value_heads82