github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-03-08 23:03:55 -05:00
Code Issues Packages Projects Releases Wiki Activity

Add FP8 to Vol 1 precision list and tidy Vol 2 notation ending.

Browse Source 
FP8 is a data format relevant to single-node systems (H100 supports
it natively), so it belongs in Vol 1's precision list rather than as
an orphaned bullet in Vol 2. Network throughput units (Gbps vs GB/s)
remain Vol 2-only under a proper "Additional Units" subsection.
This commit is contained in:
Vijay Janapa Reddi
2026-03-05 15:34:02 -05:00
parent 7992c8ff13
commit 01a7ee2a18
2 changed files with 3 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
book/quarto/contents/vol1/frontmatter/notation.qmd
View File

@@ -94,6 +94,7 @@ We follow standard deep learning conventions (@goodfellow2016deep) with explicit
* **FP32**: Single precision (4 bytes)
* **FP16**: Half precision (2 bytes, standard range)
* **BF16**: Brain float (2 bytes, wide dynamic range)
* **FP8**: Quarter precision (1 byte, E4M3 or E5M2 format)
* **INT8**: 8-bit integer (1 byte)
## Quick Reference: Resolving Collisions {#sec-notation-conventions-quick-reference-resolving-collisions-a9cf}

6
book/quarto/contents/vol2/frontmatter/notation.qmd
View File

@@ -85,8 +85,6 @@ The distributed context introduces further symbol collisions beyond those in Vol
| $\lambda$ | Sensitivity | Failure Rate | **Context-dependent**. Sensitivity in degradation; failure rate in reliability. |
| $d$ | Hidden Dimension | Data Parallelism Degree | **Context-dependent**. Parallelism in 3D notation; hidden dim in architectures. |
### Network Throughput Units {#sec-notation-network-units}
### Additional Units {#sec-notation-distributed-units}
Network throughput is reported in both bytes/s (GB/s, TB/s) and bits/s (Gbps) depending on convention. InfiniBand and Ethernet specifications use Gbps; application-level throughput uses GB/s. We note the convention on first use.
**FP8**: Quarter precision (1 byte, E4M3 or E5M2 format) is introduced in this volume for large-scale quantized training.
* **Network throughput**: Reported in both bytes/s (GB/s, TB/s) and bits/s (Gbps) depending on convention. InfiniBand and Ethernet specifications use Gbps; application-level throughput uses GB/s. We note the convention on first use.