studio-ousia/luke-base

Original314.4Mparameters

Fixture135.2Mparameters

Layer stacks1compared

Findings0to review

✓ Same architecture, smaller

Every module class, layer kind and checked property of the original is in the fixture.

One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.

Original · 12

Fixture · 2

Kind	Original	Fixture
Layer	12	2	✓ kept

Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.

Original

Fixture

Component	Original	Share	Fixture	Share
luke.entity_embeddings	128.6M	40.9%	128.0M	94.7%
luke.encoder.layer	106.3M	33.8%	91.9K	0.1%
luke.embeddings	39.0M	12.4%	3.3M	2.4%
lm_head.decoder	38.6M	12.3%	3.2M	2.4%
lm_head.dense	590.6K	0.2%	4.2K	0.0%
luke.pooler	590.6K	0.2%	4.2K	0.0%
entity_predictions	500.0K	0.2%	500.0K	0.4%
3 others	249.2K	0.1%	67.5K	0.0%

Config	Property	Original	Fixture
model	model_type	luke	luke	✓ kept
model	vocab_size	50267	50267	✓ kept
model	tie_word_embeddings	True	True	✓ kept
model	hidden_act	gelu	gelu	✓ kept
model	max_position_embeddings	514	514	✓ kept

Instances of each class in the built model (prefix Luke omitted).

Class	Original	Fixture
Linear	114	24
Dropout	38	8
LayerNorm	28	8
GELUActivation	13	3
Attention	12	2
Intermediate	12	2
Layer	12	2
Output	12	2
SelfAttention	12	2
SelfOutput	12	2
Embedding	6	6
CrossEntropyLoss	1	1
EntityPredictionHead	1	1
EntityPredictionHeadTransform	1	1
Embeddings	1	1
Encoder	1	1
EntityEmbeddings	1	1
ForMaskedLM	1	1
LMHead	1	1
Model	1	1
Pooler	1	1
ModuleList	1	1
Tanh	1	1

4 fields changed, 29 unchanged.