microsoft/deberta-large-mnli

Original406.2Mparameters

Fixture3.4Mparameters

Layer stacks1compared

Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

Original · 24

Fixture · 2

Kind	Original	Fixture
Layer	24	2	✓ kept

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original

Fixture

Component	Original	Share	Fixture	Share
deberta.encoder.layer	352.6M	86.8%	83.3K	2.5%
deberta.embeddings	51.5M	12.7%	3.2M	95.5%
pooler.dense	1.0M	0.3%	4.2K	0.1%
deberta.encoder	1.0M	0.3%	65.5K	1.9%
classifier	3.1K	0.0%	195	0.0%

Config	Property	Original	Fixture
model	model_type	deberta	deberta	✓ kept
model	vocab_size	50265	50265	✓ kept
model	tie_word_embeddings	True	True	✓ kept
model	hidden_act	gelu	gelu	✓ kept
model	max_position_embeddings	512	512	✓ kept

Instances of each class in the built model (prefix Deberta omitted).

Class	Original	Fixture
Linear	146	14
Dropout	99	11
LayerNorm	49	5
Attention	24	2
Intermediate	24	2
Layer	24	2
Output	24	2
SelfOutput	24	2
DisentangledSelfAttention	24	2
GELUActivation	24	2
Embedding	2	2
ContextPooler	1	1
Embeddings	1	1
Encoder	1	1
ForSequenceClassification	1	1
Model	1	1
ModuleList	1	1

5 fields changed, 28 unchanged.