Model Parameter Calculator

Enter shapes in HuggingFace style like [151 552, 5 120]. Numbers may include spaces or narrow no‑break spaces; the calculator multiplies dimensions per tensor and sums across lines.

Model Structure

Choose Model

Total layers 0

Computed automatically from the four layer counts below.

A: Dense attention-only Z01

B: Dense SSM+attention Z02

C: MoE attention-only Z03

D: MoE SSM+attention Z04

E: Embedding/output matrix shapes (one per line) Z05

F: Pre/post first/last layer norms/others (usually F32) (optional, one per line) Z06

Dense attention-only layers

G: Norms, small SSM tensors, etc (usually F32) (optional, per dense layer, one per line) Z07

H: Attention/SSM tensors (per dense layer, one per line) Z08

I: FFN tensors (per dense layer, one per line) Z09

Dense SSM+attention layers

J: Norms and small tensors (per dense SSM+attention layer, one per line) Z10

K: Attention/SSM tensors (per dense SSM+attention layer, one per line) Z11

L: FFN tensors (per dense SSM+attention layer, one per line) Z12

MoE Layer Settings

M: Experts per MoE layer Z13

N: Active experts per MoE layer Z14

Include shared expert Z43

O: Shared expert scope Z15

P: Shared expert tensors Z16

Shared expert parameters are counted under FFN totals.

MoE attention-only layers

Q: Attention/SSM tensors (per MoE layer, one per line) Z17

R: Norms, small SSM tensors, etc (usually F32) (optional, per MoE layer, one per line) Z18

S: Gate input, biases, other FFN (always active) Z19

Included in FFN parameter totals.

T: MoE experts tensors (per MoE layer; may include expert dim E) Z20 Tensors include expert dimension (E) Z44

If unchecked, shapes are treated as per‑expert and multiplied by the number of experts.

MoE SSM+attention layers

U: Attention/SSM tensors (per MoE SSM+attention layer, one per line) Z21

V: Norms, small SSM tensors, etc (per MoE SSM+attention layer, one per line) Z22

W: Gate input, biases, other FFN (always active) Z23

Included in FFN parameter totals.

X: MoE experts tensors (per MoE SSM+attention layer; may include expert dim E) Z24