ELM Dataset Standard

01 · Overview

One standard, applied to every dataset we publish. #

# format
ELM-<LANGUAGE>-<MODALITY>-<TASK>-<STATUS>-<SEQUENCE>

# example
ELM-SWA-SPH-ASR-VAL-001
→ Swahili · speech · built for speech recognition · validated · first of its kind

# pattern
^ELM-[A-Z]{3}-(TXT|SPH|IMG|VID|MM)
     -(ASR|TTS|KWS|LID|SID|OCR|VQA|SLR|NER|CLS|SUM|POS|GEN|MT|ST|QA|LM)
     -(VAL|PAR|UNV)-[0-9]{3}$

02 · Terminology

Words we use precisely. #

03 · The code name

Six segments, in a fixed order. #

The code name is the formal identifier used in systems, citations and licensing. It is deliberately readable: someone who has never seen the dataset should be able to say what language it is in, what it contains, what it was built to be used for, and how far it has been checked — all without opening it.

code       = "ELM" "-" language "-" modality "-" task "-" status "-" sequence
language   = 3 uppercase letters        ; ISO 639-3, uppercased (§6)
modality   = "TXT" | "SPH" | "IMG" | "VID" | "MM"   ; what it contains (§7)
task       = 2-3 uppercase letters      ; what it is for  (§8)
status     = "VAL" | "PAR" | "UNV"      ; how far it is checked (§9)
sequence   = 3 digits, "001" .. "999"   ; (§3.6)

04 · The nickname

The name people will actually say out loud. #

Every dataset needs both names. The nickname is what appears in conversation, documentation and dataset listings; the code name is what appears in systems and citations. Neither one is optional.

A nickname can be corrected for accuracy or spelling. When it is, the old name is kept in the metadata as an alias so existing references still resolve.

05 · Build & check a code

Assemble one, or paste one in. #

06 · Language codes

The published registry. #

Codes below are ISO 639-3, uppercased. Any language with a valid ISO 639-3 code may be used; this table lists the ones already in active collection, plus the two special codes. To add a language to the registry, see §15.

07 · Modality codes

A closed list of five. #

08 · Task codes

What the dataset was built to be used for. #

Modality tells you a dataset contains speech. It doesn't tell you whether that speech is transcribed word-for-word for a recogniser, recorded in a studio for a voice, or paired with a translation. The task segment closes that gap, so a researcher can rule a dataset in or out before downloading it.

8.1 Choosing the right one

• a.Ask what a model could be trained or evaluated on with these labels, as they exist today. If the honest answer is "nothing yet", the code is LM or GEN.
• b.ASR and TTS both pair audio with text, but they are not interchangeable: ASR wants many speakers and real conditions, TTS wants few speakers and clean, consistent recording. Label by what the collection was designed for.
• c.Anything labelled by category — sentiment, topic, intent, spam or scam messages — is CLS. The specific label set lives in the metadata, not the code.
• d.A dataset whose task cannot be named without a compound ("ASR-CLS") is two datasets, or one dataset with an also_supports entry. Compounds are not valid codes.
• e.The list is closed. A task that isn't here needs a registry change (§15), not an improvised code.

8.2 The same language and modality, three different jobs

09 · Validation states

What each state promises. #

9.1 Criteria

10 · Lifecycle

How a dataset moves, and what moves with it. #

  collected            review begins           review complete + audit passes
      │                       │                              │
      ▼                       ▼                              ▼
   UNV  ───────────────▶  PAR  ──────────────────────────▶  VAL
                              ▲                              │
                              └──────────────────────────────┘
                                 audit fails, or new unreviewed records added

• 10.1Status transitions are recorded with a date and the annotator or lead responsible. The history stays in the metadata; the code only shows the current state.
• 10.2Adding records to a VAL dataset drops it to PAR until the new records are reviewed. Nothing silently inherits a validated status.
• 10.3Releases are numbered MAJOR.MINOR.PATCH. Major means records were removed or re-scoped; minor means records were added; patch means corrections only. The version lives in the metadata, never in the code.
• 10.4Retiring a dataset never frees its stem. A retired dataset keeps its code, is marked retired in the metadata with a reason, and stays resolvable so old citations don't break.
• 10.5Splitting a dataset creates new stems with new sequence numbers, each recording the stem it came from. Merging is not permitted — publish a new dataset that names its sources instead.

11 · Package & metadata

What ships with every dataset. #

11.1 Directory layout

ELM-SWA-SPH-ASR-001/
├── dataset.json          metadata, machine readable (§11.2)
├── README.md             the same information, credits included, for a reader
├── LICENCE.txt           the licence for this dataset specifically
├── data/
│   ├── audio/            WAV, one file per record
│   └── transcripts/
├── metadata/
│   ├── records.jsonl     one line per record, incl. review state
│   └── speakers.jsonl    pseudonymised speaker profiles
└── checksums.sha256

The folder is named with the stem, not the display code — see §3.5. Speaker identity, contact details and raw consent forms never enter the package; only pseudonymous identifiers do.

11.2 dataset.json

{
  "stem": "ELM-SWA-SPH-ASR-001",
  "display_code": "ELM-SWA-SPH-ASR-VAL-001",
  "nickname": "Swahili Everyday Speech",
  "aliases": [],
  "language": { "code": "swa", "name": "Swahili", "script": "Latn" },
  "modality": "SPH",
  "task": "ASR",
  "also_supports": ["LID"],
  "validation": {
    "status": "VAL",
    "reviewed_share": 1.0,
    "audit_share": 0.05,
    "audit_agreement": 0.97,
    "last_pass": "2026-07-14"
  },
  "content": { "records": 18420, "duration_hours": 62.4, "speakers": 214 },
  "collection": { "method": "prompted recording", "tool": "ELM Voice Studio",
                   "started": "2026-02-03", "consent": "recorded" },
  "credits": {
    "authors": [
      { "name": "Mwaka Chanda", "role": "collection lead",
        "affiliation": "Every Language Matters", "orcid": "0000-0002-1825-0097" },
      { "name": "Asha Juma", "role": "annotation lead",
        "affiliation": "Every Language Matters" }
    ],
    "contributors": [
      { "name": "ELM Swahili annotation team", "role": "validation", "count": 9 }
    ],
    "contact": "https://everylanguagematters.com/contacts",
    "acknowledgements": "Recording space provided by the Lusaka community radio network."
  },
  "release": { "version": "1.2.0", "date": "2026-07-21" },
  "licence": "ELM-DATA-1.0",
  "checksums": "checksums.sha256"
}

11.3 Credit

Datasets are work, and the people who did that work are named in the metadata so the credit travels with the data instead of living in someone's memory. credits.authors is required and ordered — it is the author list a citation uses (§13).

• a.An author is someone who shaped the dataset: designed the collection, led the recording or gathering, wrote the annotation guidelines, or led validation. Approving a budget or hosting the files is not authorship.
• b.Order is agreed by the authors themselves before the first release and recorded as given — not alphabetical, not by seniority.
• c.Annotators and reviewers who worked to the guidelines go in contributors: named individually where they've agreed to be, or collectively with a count where they'd rather not be.
• d.Speakers, writers and other data subjects are never listed as authors or contributors. They stay pseudonymous in speakers.jsonl and are credited in the aggregate.
• e.Record an orcid where the author has one. Affiliation is recorded as it stood at the release, and isn't rewritten when someone moves on.
• f.The author list belongs to a release. A later release that adds substantial new work may add authors; it never removes them and never reorders the ones already published.
• g.Partner-built datasets name the partner's people alongside ours, each with their own affiliation. The organisation segment of the code stays ELM either way (§3.1).

12 · Implementation

Enforce it in code, not in review. #

Pattern

# modality and task groups, written out once
MOD  = TXT|SPH|IMG|VID|MM
TASK = ASR|TTS|KWS|LID|SID|OCR|VQA|SLR|NER|CLS|SUM|POS|GEN|MT|ST|QA|LM

# display code
^ELM-[A-Z]{3}-(MOD)-(TASK)-(VAL|PAR|UNV)-[0-9]{3}$

# stem
^ELM-[A-Z]{3}-(MOD)-(TASK)-[0-9]{3}$

Python

import re

MODALITY = "TXT|SPH|IMG|VID|MM"
TASK = "ASR|TTS|KWS|LID|SID|OCR|VQA|SLR|NER|CLS|SUM|POS|GEN|MT|ST|QA|LM"

CODE = re.compile(
    "^ELM-(?P<language>[A-Z]{3})"
    "-(?P<modality>" + MODALITY + ")"
    "-(?P<task>" + TASK + ")"
    "-(?P<status>VAL|PAR|UNV)"
    "-(?P<sequence>[0-9]{3})$"
)

def parse(code: str) -> dict:
    match = CODE.match(code.strip().upper())
    if not match:
        raise ValueError(f"{code!r} is not a valid ELM dataset code")
    return match.groupdict()

def display_code(stem: str, status: str) -> str:
    org, language, modality, task, sequence = stem.split("-")
    return "-".join([org, language, modality, task, status, sequence])

PHP · Laravel

// config/datasets.php
'modalities' => ['TXT', 'SPH', 'IMG', 'VID', 'MM'],
'tasks' => ['ASR', 'TTS', 'ST', 'KWS', 'SID', 'LID', 'MT', 'CLS', 'NER',
             'POS', 'QA', 'SUM', 'OCR', 'VQA', 'SLR', 'LM', 'GEN'],

// validation
$mod = implode('|', config('datasets.modalities'));
$task = implode('|', config('datasets.tasks'));

'stem' => ['required', "regex:/^ELM-[A-Z]{3}-($mod)-($task)-[0-9]{3}$/"],
'status' => ['required', Rule::in(['VAL', 'PAR', 'UNV'])],

// App\Models\Dataset — never store the display code
protected function displayCode(): Attribute
{
    return Attribute::get(function () {
        [$org, $language, $modality, $task, $sequence] = explode('-', $this->stem);

        return implode('-', [$org, $language, $modality, $task, $this->status, $sequence]);
    });
}

PostgreSQL

ALTER TABLE datasets
  ADD CONSTRAINT datasets_stem_format
  CHECK (stem ~ '^ELM-[A-Z]{3}-(TXT|SPH|IMG|VID|MM)'
              '-(ASR|TTS|KWS|LID|SID|OCR|VQA|SLR|NER|CLS|SUM|POS|GEN|MT|ST|QA|LM)-[0-9]{3}$');

ALTER TABLE datasets ADD CONSTRAINT datasets_stem_unique UNIQUE (stem);

13 · Citation

Cite the stem, name the release. #

A citation has to stay true after a dataset is revalidated, so it points at the stem and the release, and states the status as it stood on the day the work was done. It opens with the author list from credits.authors (§11.3), in the order recorded there.

# with named authors
Chanda, M., & Juma, A. (2026). Swahili Everyday Speech (Version 1.2.0)
  [Data set]. Every Language Matters. ELM-SWA-SPH-ASR-001, status VAL
  as of 2026-07-21. https://everylanguagematters.com/datasets/ELM-SWA-SPH-ASR-001

# seven or more authors
Chanda, M., et al. (2026). Swahili Everyday Speech (Version 1.2.0)
  [Data set]. Every Language Matters. ELM-SWA-SPH-ASR-001, status VAL
  as of 2026-07-21. https://everylanguagematters.com/datasets/ELM-SWA-SPH-ASR-001

• a.Papers reporting results on a PAR dataset should state the reviewed share they trained or evaluated on.
• b.The task segment is part of the stem, so a citation already states what the dataset was built for. Don't drop segments to shorten it.
• c.Every Language Matters is the publisher, not the author. A complete citation names both.
• d.Licences attach to a release, not to a stem. Check the LICENCE.txt in the copy you actually received.

14 · Checklist

Before a dataset is published. #

15 · Governance

Changing the standard. #

Adding a language to the registry, adding a modality code, or changing a validation threshold is a change to this document. Send the proposal — what you want to add, why the existing codes don't cover it, and the datasets affected — through the contacts page. Registry additions are usually decided within a week; changes to the code format itself are versioned and announced on the news page before they take effect.

15.1 Change log