solution / multilingual datasets

Multilingual datasets, written in the language, not translated into it.

A model extended with machine-translated English learns to sound like a translation. eeko's multilingual datasets are composed natively in each target language by native speakers and professional translators, checked by a second native speaker, and tagged with language, dialect, and origin on every item. Fine-tuning, preference, evaluation, and speech data, in the languages your users actually write.

Native authorship Second-native review Origin recorded Pilot batch first
Hero image · 840 × 1050
items checked by a second native speaker
100%
reviewer decisions stored with each item
style guide
1/lang
register, script, orthography, and dialect coverage in writing
data types per language
6
SFT, preference, evaluation, red-team prompts, speech, annotation
first reply
1day
a data lead confirms coverage before anyone quotes

What is a multilingual dataset?

A multilingual dataset is training, preference, or evaluation material that exists in several languages and was made by people who speak each of them natively. The distinction that matters is not how many languages are listed on the box but how the text got there. Data written in a language from the start carries that language's idiom, its politeness system, its way of asking for help, and its cultural references. Data pushed through machine translation carries English sentence structure wearing a different vocabulary.

The second kind is cheap and plentiful, which is why so many models sound stilted outside English: literal idioms, the wrong formality, references that do not land, and poor handling of dialects and mixed-language messages.

eeko's data practice commissions the first kind. Native-speaker writers and professional translators in the eeko expert network compose prompts, responses, comparisons, and evaluation items directly in the target language, following a style guide written for that language. A second native speaker checks every item before it is accepted, and each item records how it was made, so anything machine-translated or AI-assisted is labeled and can be excluded. The same discipline we apply to supervised fine-tuning data and preference data in English is available in every language we can staff, within the eeko AI training data practice, and it is what the agents we deploy run on when they serve customers in more than one language.

Multilingual overview image · 2100 × 900
One specification, many languages. Each language gets its own writers, its own reviewers, and its own style guide.

Multilingual datasets for every stage of training.

Everything we produce in English is available across languages, each scoped per language in the specification.

01 / sftCORE
Native-written fine-tuning data
Instruction and response demonstrations composed directly in the target language by native writers, to a per-language style guide that fixes register, script, and orthography.
  • Single-turn and multi-turn demonstrations
  • Domain-specific where experts can be staffed
  • Authorship and origin on every item
02 / preferenceCORE
Multilingual preference comparisons
Pairwise and ranked judgments by native speakers on correctness, fluency, register, and cultural fit, with the written rationale kept alongside the choice.
  • Rubrics adapted per language
  • Agreement measured on overlapping items
  • Rationales delivered with each judgment
03 / evaluationHELD-OUT
Multilingual evaluation sets
Held-out test items written natively and never published, so a benchmark score in Portuguese or Hindi means the model handled the language, not that it memorized the test.
  • Exclusive to you by default
  • Per-language difficulty calibration
  • Refreshable on a standing order
04 / translationCORE
Professional translation and parallel text
When you need aligned pairs, professional translators build them, then a second native speaker checks adequacy and fluency. Source, target, and reviewer are recorded per segment.
  • Sentence and document-level alignment
  • Terminology lists maintained per project
  • Machine-translation assistance disclosed
05 / localizationCORE
Localization
Content adapted rather than translated: local examples, currencies, institutions, holidays, and conventions, so a prompt about filing taxes or booking a train reads as local, because it is.
  • Locale-specific examples and entities
  • Number, date, and address formats
  • Regional product and service names
06 / variantsCORE
Dialect and code-switching data
Regional varieties and mixed-language text written by speakers who use them daily, with dialect tags on each item so you can measure and weight coverage.
  • Writers selected for the variety
  • Language and dialect metadata per item
  • Coverage confirmed before quoting

One style guide per language, written before the first item.

Most multilingual data problems are decided before any writing starts. The style guide is where we decide them, with you.

Style guide image · 1200 × 900
Writers and reviewers work from the same document, so disagreements are about the guide, not about taste.

What the guide settles for each language

  • Register and formality — which pronoun and verb forms the assistant uses, when it switches, and how it addresses a stranger versus a returning user.
  • Script and orthography — simplified or traditional, Latin or native script, pre- or post-reform spelling, and how loanwords are written.
  • Dialect coverage — which regional varieties are in scope, in what proportions, and how each is tagged.
  • Local conventions — dates, numbers, currencies, units, honorifics, and the institutions a local user would mention.
  • Sensitive topics — what the assistant should decline or handle carefully in that culture, which is not always what English guidelines assume.
  • Review rubric — the criteria the second native speaker scores against, so acceptance is consistent across reviewers and batches.

The guide is versioned. Edge-case rulings made during the pilot are added to it, and each item notes the guide version it was written against. Our linguistics experts and professional translators draft it with your team.

Native speaker review, with the decisions on record.

You may not read the language. Your audit trail should not depend on it.

Native writer image · 800 × 1000
Written — by a native speaker, in the language, to the style guide.
Native reviewer image · 800 × 1000
Reviewed — by a second native speaker, with comments stored on the item.
Distribution report image · 800 × 1000
Reported — counts, agreement, and dialect coverage per language, against the spec.

Origin on every item

Each item records whether it was written natively, translated by a professional, or touched by machine translation or an AI assistant, so you can filter on origin at training time.

Agreement as a number

Overlapping items are rated by more than one native speaker and agreement is reported per language and per batch. Low agreement points at the guide, not just the writer, and we fix the guide.

Fingerprinted and signed

Every item is fingerprinted against everything already registered to catch duplicates across writers and languages, then signed with a provenance record. Your receipt lists each item by hash, as on every eeko dataset.

From a language list to a delivered set.

Coverage confirmed before the quote, pilot priced on production terms, no volume commitment until you have seen a batch.

STEP 01
Brief and coverage check
Send the languages, domains, data types, and volumes. A data lead checks the bench for each pairing and replies inside one business day with what we can staff.
STEP 02
Style guides and pilot
A style guide per language, a target distribution, and a review rubric, then a pilot batch in each language so your bilingual reviewers can test it before volume.
STEP 03
Production
Native writers produce, second native speakers review, and distribution reports arrive per language as batches complete. Guide rulings are versioned as edge cases appear.
STEP 04
Delivery
JSON Lines, CSV, or Parquet with language, dialect, origin, and reviewer fields, Croissant metadata, and a signed receipt listing every item by hash.

Multilingual dataset questions.

What is a multilingual dataset for AI?

A multilingual dataset is training, preference, or evaluation material that exists in more than one language and was produced by people who speak each of those languages natively. The strong form is composed in each language from the start, so prompts and responses reflect how speakers really phrase a question, pick a register, and refer to their own culture. The weak form is English data run through machine translation, which teaches a model to sound translated. eeko builds the strong form: native writers, a style guide per language, and a second native speaker checking every item.

Which languages can you cover?

Coverage follows the native speakers and translators in the eeko expert network, and it varies by language and by domain. Widely spoken languages are straightforward to staff, including for specialist subjects such as medicine or law. For less common languages we check the bench for your exact language-and-domain pairing during scoping and tell you what we can staff before we quote, rather than promising coverage and finding the gap mid-project. During production you get distribution reports showing actual counts per language against the specification.

Is the data translated from English?

Not unless you order parallel data. The default is native authorship: writers compose prompts, responses, and judgments directly in the target language, which keeps source-language sentence structure and idiom out of the set. When you do need parallel text, professional translators produce it and a second native speaker reviews it for accuracy and fluency. Every item records how it was made, including any machine-translation or AI assistance, so translated and native-written material can be separated at training time.

How do you check quality in a language we cannot read?

Every item is checked by a second native speaker who works from the same per-language style guide as the writer, and reviewer comments and decisions are stored with the item. Where you want a measurable signal, overlapping items are rated by more than one person so agreement can be computed and reported per language. Language-specific gold items are seeded into production work. You receive the review outcomes, not just the accepted items, so your own bilingual staff can audit decisions at any time.

Can you cover dialects and code-switching?

Yes, where native speakers of that variety can be staffed. The variety you need is set down in the style guide and the coverage targets, writers are chosen for that variety rather than the standard form, and each item carries language and dialect metadata so you can filter or weight by variant. Mixed-language text is written the way people actually type it, not assembled from two monolingual halves. If a variety cannot be staffed reliably, we tell you during scoping.

How is multilingual data priced?

Price is quoted per item and per language after scoping. The main drivers are the language itself and the depth of our native-speaker bench for it, the domain expertise required, the task type, whether you need native writing or professional translation, the share of items sent for review, and whether the set is exclusive to you. Specialist content in a less widely spoken language sits at the top of the range. The pilot batch is priced on the same terms, so you see the real number before committing to volume.

Send us the languages your model still gets wrong.

List the languages, domains, and data types. A data lead confirms coverage inside one business day and returns a pilot proposal priced per language. No deck. A coverage table and a quote.

Experienced within

  • SOC 2 Type II
  • ISO 27001
  • HIPAA
  • HITRUST r2
  • ISO 27701
  • PCI DSS
  • NIST CSF 2.0
  • CMMC
  • FedRAMP
  • GDPR

Markets served.

As an enterprise AI agency, eeko systems delivers production AI systems remote-first across the United States and internationally — including these markets:

New York City, New York (NY)

Los Angeles, California (CA)

Chicago, Illinois (IL)

Houston, Texas (TX)

Phoenix, Arizona (AZ)

Philadelphia, Pennsylvania (PA)

San Antonio, Texas (TX)

San Diego, California (CA)

Dallas, Texas (TX)

San Jose, California (CA)

Austin, Texas (TX)

Jacksonville, Florida (FL)

Fort Worth, Texas (TX)

Columbus, Ohio (OH)

Charlotte, North Carolina (NC)

Indianapolis, Indiana (IN)

San Francisco, California (CA)

Seattle, Washington (WA)

Denver, Colorado (CO)

Washington, District of Columbia (DC)

Boston, Massachusetts (MA)

El Paso, Texas (TX)

Nashville, Tennessee (TN)

Detroit, Michigan (MI)

Oklahoma City, Oklahoma (OK)

Portland, Oregon (OR)

Las Vegas, Nevada (NV)

Memphis, Tennessee (TN)

Louisville, Kentucky (KY)

Baltimore, Maryland (MD)

Milwaukee, Wisconsin (WI)

Albuquerque, New Mexico (NM)

Tucson, Arizona (AZ)

Fresno, California (CA)

Sacramento, California (CA)

Kansas City, Missouri (MO)

Atlanta, Georgia (GA)

Miami, Florida (FL)

Colorado Springs, Colorado (CO)

Raleigh, North Carolina (NC)

Omaha, Nebraska (NE)

Long Beach, California (CA)

Virginia Beach, Virginia (VA)

Oakland, California (CA)

Minneapolis, Minnesota (MN)

Tulsa, Oklahoma (OK)

Arlington, Texas (TX)

New Orleans, Louisiana (LA)

Wichita, Kansas (KS)

Cleveland, Ohio (OH)

Tampa, Florida (FL)

Bakersfield, California (CA)

Aurora, Colorado (CO)

Honolulu, Hawaii (HI)

Anaheim, California (CA)

Santa Ana, California (CA)

Corpus Christi, Texas (TX)

Riverside, California (CA)

Lexington, Kentucky (KY)

St. Louis, Missouri (MO)

Stockton, California (CA)

Pittsburgh, Pennsylvania (PA)

Saint Paul, Minnesota (MN)

Cincinnati, Ohio (OH)

Greensboro, North Carolina (NC)

Anchorage, Alaska (AK)

Plano, Texas (TX)

Lincoln, Nebraska (NE)

Orlando, Florida (FL)

Irvine, California (CA)

Newark, New Jersey (NJ)

Toledo, Ohio (OH)

Durham, North Carolina (NC)

Chula Vista, California (CA)

Fort Wayne, Indiana (IN)

Jersey City, New Jersey (NJ)

St. Petersburg, Florida (FL)

Laredo, Texas (TX)

Madison, Wisconsin (WI)

Chandler, Arizona (AZ)

Buffalo, New York (NY)

Lubbock, Texas (TX)

Scottsdale, Arizona (AZ)

Reno, Nevada (NV)

Glendale, Arizona (AZ)

Gilbert, Arizona (AZ)

Winston-Salem, North Carolina (NC)

North Las Vegas, Nevada (NV)

Norfolk, Virginia (VA)

Chesapeake, Virginia (VA)

Fremont, California (CA)

Garland, Texas (TX)

Richmond, Virginia (VA)

Baton Rouge, Louisiana (LA)

Boise, Idaho (ID)

San Bernardino, California (CA)

Spokane, Washington (WA)

Des Moines, Iowa (IA)

Modesto, California (CA)

Birmingham, Alabama (AL)

Tacoma, Washington (WA)

Fontana, California (CA)

Oxnard, California (CA)

Fayetteville, North Carolina (NC)

Huntsville, Alabama (AL)

Moreno Valley, California (CA)

Rochester, New York (NY)

Glendale, California (CA)

Yonkers, New York (NY)

Augusta, Georgia (GA)

Amarillo, Texas (TX)

Little Rock, Arkansas (AR)

Akron, Ohio (OH)

Shreveport, Louisiana (LA)

Grand Rapids, Michigan (MI)

Mobile, Alabama (AL)

Salt Lake City, Utah (UT)

Huntsville, Texas (TX)

Tallahassee, Florida (FL)

Overland Park, Kansas (KS)

Knoxville, Tennessee (TN)

Worcester, Massachusetts (MA)

Brownsville, Texas (TX)

New Port Richey, Florida (FL)

Jackson, Mississippi (MS)

Providence, Rhode Island (RI)

Fort Lauderdale, Florida (FL)

Sioux Falls, South Dakota (SD)

Tempe, Arizona (AZ)

Cape Coral, Florida (FL)

Springfield, Missouri (MO)

Pembroke Pines, Florida (FL)

Eugene, Oregon (OR)

Peoria, Arizona (AZ)

Corona, California (CA)

Lancaster, California (CA)

Rockford, Illinois (IL)

Salinas, California (CA)

Palmdale, California (CA)

Springfield, Massachusetts (MA)

Charleston, South Carolina (SC)

Duluth, Minnesota (MN)

London, England (ENG)

Dublin, Ireland (IRE)