← MIC coverage of Transformers model types

openai/whisper-tiny

huggingface-mic/whisper-mic@1f40ed2753f5 ← openai/whisper-tiny@169d4a4341b3 · WhisperForConditionalGeneration · overfit weights · built before the shrink rule (no recorded recipe)

CI: passing now; 7 failed checks in 30 days · MIC failures dashboard

Original37.8Mparameters
Fixture3.6Mparameters
Layer stacks2compared
Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

Layer stacks

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

model.encoder.layers

Original · 4
Fixture · 2
KindOriginalFixture
EncoderLayer42✓ kept

model.decoder.layers

Original · 4
Fixture · 2
KindOriginalFixture
DecoderLayer42✓ kept

Where the parameters are

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original
Fixture
ComponentOriginalShareFixtureShare
model.decoder20.1M53.2%3.3M92.0%
model.decoder.layers9.5M25.1%100.2K2.8%
model.encoder.layers7.1M18.8%66.8K1.8%
model.encoder1.1M2.9%123.9K3.4%

Preserved properties

ConfigPropertyOriginalFixture
modelmodel_typewhisperwhisper✓ kept
modelvocab_size5186551865✓ kept
modeltie_word_embeddingsTrueTrue✓ kept
modelnum_mel_bins8080✓ kept

Module classes

Instances of each class in the built model (prefix Whisper omitted).

ClassOriginalFixture
Linear6533
LayerNorm2212
Attention126
GELUActivation84
DecoderLayer42
EncoderLayer42
Conv1d22
Embedding22
ModuleList22
Decoder11
Encoder11
ForConditionalGeneration11
Model11
PositionalEmbedding11

Config changes

7 fields changed, 40 unchanged.

FieldOriginalFixture
d_model38464
decoder_attention_heads64
decoder_ffn_dim1536128
decoder_layers42
encoder_attention_heads64
encoder_ffn_dim1536128
encoder_layers42