Estimate memory requirements for different model architectures and platforms
| Device | VRAM/RAM | Inference | Training |
|---|---|---|---|
| NVIDIA RTX 4090 | 24.0 GB | ✓ | ✓ |
| NVIDIA RTX 4080 | 16.0 GB | ✗ | ✗ |
| NVIDIA RTX 3090 | 24.0 GB | ✓ | ✓ |
| NVIDIA L40 | 48.0 GB | ✓ | ✓ |
| NVIDIA L4 | 24.0 GB | ✓ | ✓ |
| NVIDIA A100 (40GB) | 40.0 GB | ✓ | ✓ |
| NVIDIA A100 (80GB) | 80.0 GB | ✓ | ✓ |
| NVIDIA H100 (80GB) | 80.0 GB | ✓ | ✓ |
| NVIDIA B200 | 128.0 GB | ✓ | ✓ |
| AMD MI300X | 192.0 GB | ✓ | ✓ |
| AMD MI300A | 128.0 GB | ✓ | ✓ |
| AMD MI250X | 128.0 GB | ✓ | ✓ |
| Google TPU v5e | 16.0 GB | ✗ | ✗ |
| Google TPU v5p | 128.0 GB | ✓ | ✓ |
| Apple M3 Ultra | 192.0 GB | ✓ | ✓ |
| Apple M2 Ultra | 192.0 GB | ✓ | ✓ |
| CPU (32GB RAM) | 32.0 GB | ✓ | ✓ |
| CPU (64GB RAM) | 64.0 GB | ✓ | ✓ |
| CPU (128GB RAM) | 128.0 GB | ✓ | ✓ |
Your model configuration appears efficient for your current parameters.