format: benchmark

purpose: Measure model competence on Bantu numerals. Score model output against `reference`. Exact-match or linguist-judged.

types: cardinal arithmetic

tier_floor: C


type: cardinal

iso: bem

prompt: Write the number 1 in Bemba (cardinal form). Answer with only the Bemba words.

reference: cimo

english: one

tier: A

quality: full


type: arithmetic

iso: bem

prompt: In Bemba, what is 1 + 1? Answer with only the Bemba words for the result.

reference: fibili

english: one + one = two

op: add


type: cardinal

iso: bem

prompt: Write the number 250 in Bemba (cardinal form). Answer with only the Bemba words.

reference: imyanda yabili na amakumi yasano

english: two hundred fifty


type: arithmetic

iso: bem

prompt: In Bemba, what is 250 + 83? Answer with only the Bemba words for the result.

reference: imyanda yatatu na amakumi yatatu na fitatu

english: two hundred fifty + eighty-three = three hundred thirty-three


type: cardinal

iso: bem

prompt: Write the number 500 in Bemba (cardinal form). Answer with only the Bemba words.

reference: imyanda yasano

english: five hundred


type: arithmetic

iso: bem

prompt: In Bemba, what is 500 + 166? Answer with only the Bemba words for the result.

reference: imyanda mutanda na amakumi mutanda na mutanda

english: five hundred + one hundred sixty-six = six hundred sixty-six


type: cardinal

iso: bem

prompt: Write the number 750 in Bemba (cardinal form). Answer with only the Bemba words.

reference: imyanda cine lubali na amakumi yasano

english: seven hundred fifty


type: arithmetic

iso: bem

prompt: In Bemba, what is 750 + 250? Answer with only the Bemba words for the result.

reference: ikana limo

english: seven hundred fifty + two hundred fifty = one thousand


type: cardinal

iso: bem

prompt: Write the number 1000 in Bemba (cardinal form). Answer with only the Bemba words.

reference: ikana limo

english: one thousand


type: arithmetic

iso: bem

prompt: In Bemba, what is 1000 + 333? Answer with only the Bemba words for the result.

reference: ikana limo na imyanda yatatu na amakumi yatatu na fitatu

english: one thousand + three hundred thirty-three = one thousand three hundred thirty-three


type: cardinal

iso: swh

prompt: Write the number 1 in Swahili (cardinal form). Answer with only the Swahili words.

reference: moja

english: one


type: arithmetic

iso: swh

prompt: In Swahili, what is 1 + 1? Answer with only the Swahili words for the result.

reference: mbili

english: one + one = two


type: cardinal

iso: swh

prompt: Write the number 250 in Swahili (cardinal form). Answer with only the Swahili words.

reference: mia mbili na hamsini

english: two hundred fifty


type: arithmetic

iso: swh

prompt: In Swahili, what is 250 + 83? Answer with only the Swahili words for the result.

reference: mia tatu na thelathini na tatu

english: two hundred fifty + eighty-three = three hundred thirty-three


type: cardinal

iso: swh

prompt: Write the number 500 in Swahili (cardinal form). Answer with only the Swahili words.

reference: mia tano

english: five hundred


type: arithmetic

iso: swh

prompt: In Swahili, what is 500 + 166? Answer with only the Swahili words for the result.

reference: mia sita na sitini na sita

english: five hundred + one hundred sixty-six = six hundred sixty-six


type: cardinal

iso: swh

prompt: Write the number 750 in Swahili (cardinal form). Answer with only the Swahili words.

reference: mia saba na hamsini

english: seven hundred fifty


type: arithmetic

iso: swh

prompt: In Swahili, what is 750 + 250? Answer with only the Swahili words for the result.

reference: elfu moja

english: seven hundred fifty + two hundred fifty = one thousand


type: cardinal

iso: swh

prompt: Write the number 1000 in Swahili (cardinal form). Answer with only the Swahili words.

reference: elfu moja

english: one thousand


type: arithmetic

iso: swh

prompt: In Swahili, what is 1000 + 333? Answer with only the Swahili words for the result.

reference: elfu moja na mia tatu na thelathini na tatu

english: one thousand + three hundred thirty-three = one thousand three hundred thirty-three