florence-community/Florence-2-base
Original231.4Mparameters
Fixture6.4Mparameters
Layer stacks4compared
Findings0to review
✓ Same architecture, smaller
Every module class, layer kind and checked property of the original is in the fixture.
Layer stacks
One square per layer, colored by kind: the layer's module classes and type attributes. Hover a square for its index and kind.
model.vision_tower.convs
Original · 4
Fixture · 4
| Kind | Original | Fixture | |
|---|---|---|---|
| Conv2d | 4 | 4 | ✓ kept |
model.vision_tower.blocks
Original · 4
Fixture · 4
| Kind | Original | Fixture | |
|---|---|---|---|
| VisionBlock | 4 | 4 | ✓ kept |
model.language_model.encoder.layers
Original · 6
Fixture · 1
| Kind | Original | Fixture | |
|---|---|---|---|
| BartEncoderLayer | 6 | 1 | ✓ kept |
model.language_model.decoder.layers
Original · 6
Fixture · 1
| Kind | Original | Fixture | |
|---|---|---|---|
| BartDecoderLayer | 6 | 1 | ✓ kept |
Where the parameters are
Share of each model's parameters per component. Vocabulary and patch sizes are kept, so embeddings usually take a larger share of the fixture.
Original
Fixture
| Component | Original | Share | Fixture | Share |
|---|---|---|---|---|
| model.vision_tower.blocks | 84.2M | 36.4% | 2.5M | 39.1% |
| model.language_model.decoder.layers | 56.7M | 24.5% | 50.2K | 0.8% |
| model.language_model.encoder.layers | 42.5M | 18.4% | 33.5K | 0.5% |
| model.language_model | 41.0M | 17.7% | 3.4M | 53.0% |
| model.vision_tower.convs | 6.2M | 2.7% | 392.8K | 6.1% |
| model.multi_modal_projector | 839.2K | 0.4% | 29.3K | 0.5% |
Preserved properties
| Config | Property | Original | Fixture | |
|---|---|---|---|---|
| model | model_type | florence2 | florence2 | ✓ kept |
| model | tie_word_embeddings | True | True | ✓ kept |
| text_config | model_type | bart | bart | ✓ kept |
| text_config | vocab_size | 51328 | 51328 | ✓ kept |
| text_config | tie_word_embeddings | True | True | ✓ kept |
| text_config | max_position_embeddings | 1024 | 1024 | ✓ kept |
| vision_config | model_type | florence_vision | florence_vision | ✓ kept |
| vision_config | max_position_embeddings | 50 | 50 | ✓ kept |
| vision_config | patch_size | [7, 3, 3, 3] | [7, 3, 3, 3] | ✓ kept |
Module classes
Instances of each class in the built model (prefix Florence2 omitted).
| Class | Original | Fixture |
|---|---|---|
| Linear | 194 | 58 |
| LayerNorm | 85 | 32 |
| Conv2d | 52 | 24 |
| DropPath | 46 | 18 |
| GELUActivation | 36 | 12 |
| VisionMLP | 24 | 10 |
| BartAttention | 18 | 3 |
| VisionBlock | 12 | 5 |
| VisionChannelAttention | 12 | 5 |
| VisionChannelBlock | 12 | 5 |
| VisionSpatialBlock | 12 | 5 |
| VisionWindowAttention | 12 | 5 |
| ModuleList | 8 | 8 |
| BartDecoderLayer | 6 | 1 |
| BartEncoderLayer | 6 | 1 |
| VisionConvEmbed | 4 | 4 |
| BartScaledWordEmbedding | 3 | 3 |
| BartLearnedPositionalEmbedding | 2 | 2 |
| Embedding | 2 | 2 |
| Identity | 2 | 2 |
| BartDecoder | 1 | 1 |
| BartEncoder | 1 | 1 |
| BartModel | 1 | 1 |
| ForConditionalGeneration | 1 | 1 |
| Model | 1 | 1 |
| MultiModalProjector | 1 | 1 |
| VisionBackbone | 1 | 1 |
| VisionLearnedAbsolutePositionEmbedding2D | 1 | 1 |
| VisionPositionalEmbeddingCosine1D | 1 | 1 |
Config changes
12 fields changed, 72 unchanged.
| Field | Original | Fixture |
|---|---|---|
| text_config.d_model | 768 | 64 |
| text_config.decoder_attention_heads | 12 | 2 |
| text_config.decoder_ffn_dim | 3072 | 128 |
| text_config.decoder_layers | 6 | 1 |
| text_config.encoder_attention_heads | 12 | 2 |
| text_config.encoder_ffn_dim | 3072 | 128 |
| text_config.encoder_layers | 6 | 1 |
| vision_config.depths | [1, 1, 9, 1] | [1, 1, 2, 1] |
| vision_config.embed_dim | [128, 256, 512, 1024] | [32, 64, 128, 256] |
| vision_config.num_groups | [4, 8, 16, 32] | [2, 4, 8, 16] |
| vision_config.num_heads | [4, 8, 16, 32] | [2, 4, 8, 16] |
| vision_config.projection_dim | 768 | 64 |