FacebookAI/roberta-base

huggingface-mic/roberta-base-mic@ff3c8d1292ae ← FacebookAI/roberta-base@e2da8e2f811d · RobertaForMaskedLM · random weights · built before the shrink rule (no recorded recipe)

Original124.7Mparameters

Fixture3.4Mparameters

Layer stacks1compared

Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

Original · 12

Fixture · 2

Kind	Original	Fixture
Layer	12	2	✓ kept

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original

Fixture

Component	Original	Share	Fixture	Share
roberta.encoder.layer	85.1M	68.2%	66.9K	2.0%
roberta.embeddings	39.0M	31.3%	3.3M	96.4%
lm_head.dense	590.6K	0.5%	4.2K	0.1%
lm_head	50.3K	0.0%	50.3K	1.5%
lm_head.layer_norm	1.5K	0.0%	128	0.0%

Config	Property	Original	Fixture
model	model_type	roberta	roberta	✓ kept
model	vocab_size	50265	50265	✓ kept
model	tie_word_embeddings	True	True	✓ kept
model	hidden_act	gelu	gelu	✓ kept
model	max_position_embeddings	514	514	✓ kept

Instances of each class in the built model (prefix Roberta omitted).

4 fields changed, 25 unchanged.