← MIC coverage of Transformers model types

google/pegasus-x-base

huggingface-mic/pegasus-x-base-mic@95198fe8fb30 ← google/pegasus-x-base@cd8a69a4c88a · PegasusXForConditionalGeneration · random weights

CI: no failure on main in 30 days · MIC failures dashboard

Original272.3Mparameters
Fixture6.3Mparameters
Layer stacks2compared
Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

Model tester (advisory)

The model's own small test config in its ModelTester (29 literal values, 29 set by the model's own tester, the rest generic base-tester defaults). Testers are usually representative but not always, so these notes are for review, not rules.

The fixture covers every per-layer kind the tester sets, and shrinks every size it does.

Layer stacks

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

model.encoder.layers

Original · 12
Fixture · 2
KindOriginalFixture
EncoderLayer122✓ kept

model.decoder.layers

Original · 12
Fixture · 2
KindOriginalFixture
DecoderLayer122✓ kept

Where the parameters are

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original
Fixture
ComponentOriginalShareFixtureShare
model.decoder.layers113.3M41.6%99.5K1.6%
model.encoder.layers85.0M31.2%66.7K1.1%
model.shared73.8M27.1%6.2M97.2%
model.encoder99.8K0.0%8.3K0.1%
model.decoder1.5K0.0%1280.0%

Preserved properties

ConfigPropertyOriginalFixture
modelmodel_typepegasus_xpegasus_x✓ kept
modelvocab_size9610396103✓ kept
modeltie_word_embeddingsTrueTrue✓ kept
modelmax_position_embeddings1638416384✓ kept

Module classes

Instances of each class in the built model (prefix PegasusX omitted).

ClassOriginalFixture
Linear19333
LayerNorm7414
Attention244
ReLU244
DecoderLayer122
EncoderLayer122
GlobalLocalAttention122
ScaledWordEmbedding33
ModuleList22
SinusoidalPositionalEmbedding22
Embedding11
Decoder11
Encoder11
ForConditionalGeneration11
Model11

Config changes

7 fields changed, 39 unchanged.

FieldOriginalFixture
d_model76864
decoder_attention_heads124
decoder_ffn_dim3072128
decoder_layers122
encoder_attention_heads124
encoder_ffn_dim3072128
encoder_layers122