← MIC coverage of Transformers model types

google/long-t5-tglobal-base

huggingface-mic/long-t5-tglobal-base-mic@a9f680620a0b ← google/long-t5-tglobal-base@aecb1376e5bd · LongT5ForConditionalGeneration · random weights

CI: no failure on main in 30 days · MIC failures dashboard

Original222.9Mparameters
Fixture2.3Mparameters
Layer stacks2compared
Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

Model tester (advisory)

The model's own small test config in its ModelTester (22 literal values, 22 set by the model's own tester, the rest generic base-tester defaults). Testers are usually representative but not always, so these notes are for review, not rules.

The fixture covers every per-layer kind the tester sets, and shrinks every size it does.

Layer stacks

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

encoder.block

Original · 12
Fixture · 2
KindOriginalFixture
Embedding11✓ kept
no Embedding111✓ kept

decoder.block

Original · 12
Fixture · 2
KindOriginalFixture
Embedding11✓ kept
no Embedding111✓ kept

Where the parameters are

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original
Fixture
ComponentOriginalShareFixtureShare
decoder.block113.3M50.8%115.2K5.1%
encoder.block85.0M38.1%82.6K3.7%
shared24.7M11.1%2.1M91.2%
encoder.final_layer_norm7680.0%640.0%
decoder.final_layer_norm7680.0%640.0%

Preserved properties

ConfigPropertyOriginalFixture
modelmodel_typelongt5longt5✓ kept
modelvocab_size3212832128✓ kept
modeltie_word_embeddingsTrueTrue✓ kept

Module classes

Instances of each class in the built model (prefix LongT5 omitted).

ClassOriginalFixture
Linear21737
Dropout8616
LayerNorm7414
ModuleList266
Attention244
Block244
DenseGatedActDense244
LayerFF244
NewGELUActivation244
LayerCrossAttention122
LayerSelfAttention122
LayerTransientGlobalSelfAttention122
TransientGlobalAttention122
Embedding66
Stack22
ForConditionalGeneration11

Config changes

6 fields changed, 31 unchanged.

FieldOriginalFixture
d_ff2048128
d_kv6416
d_model76864
num_decoder_layers122
num_heads124
num_layers122