Subword Segmentation
Byte Pair Encoding
Introduced by Sennrich et al. in Neural Machine Translation of Rare Words with Subword UnitsByte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations).
Lei Mao has a detailed blog post that explains how this works.
Source: Neural Machine Translation of Rare Words with Subword Units
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Language Modelling | 52 | 7.25% |
| Retrieval | 39 | 5.44% |
| Question Answering | 30 | 4.18% |
| Decoder | 27 | 3.77% |
| Large Language Model | 25 | 3.49% |
| In-Context Learning | 14 | 1.95% |
| Semantic Segmentation | 14 | 1.95% |
| Image Classification | 12 | 1.67% |
| Code Generation | 12 | 1.67% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
