Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions datasets/mgb.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"Name": "MGB",
"Volume": 1232.0,
"Unit": "hours",
"License": "unknown",
"Link": "https://arabicspeech.org/",
"HF_Link": "",
"Year": 2021,
"Domain": [
"news articles",
"TV channels"
],
"Form": "audio",
"Collection_Style": [
"crawling",
"human annotation",
"manual curation"
],
"Description": "Benchmark for Arabic ASR systems.",
"Ethical_Risks": "Medium",
"Provider": [
"HBKU",
"Qatar Computing Research Institute",
"Kanari AI",
"Carnegie Mellon University"
],
"Derived_From": [
"MGB2",
"MGB3",
"MGB5"
],
"Paper_Title": "Arabic Speech Recognition by End-to-End, Modular Systems and Human",
"Paper_Link": "https://arxiv.org/pdf/2101.08454v2",
"Tokenized": false,
"Host": "other",
"Access": "Free",
"Cost": "",
"Test_Split": true,
"Tasks": [
"speech recognition"
],
"Venue_Title": "Journal of Computer Speech and Language",
"Venue_Type": "journal",
"Venue_Name": "Journal of Computer Speech and Language",
"Authors": [
"Amir Hussein",
"Shinji Watanabe",
"Ahmed Ali"
],
"Affiliations": [
"HBKU",
"Qatar Computing Research Institute",
"Kanari AI",
"Carnegie Mellon University"
],
"Abstract": "Recent advances in automatic speech recognition (ASR) have achieved accuracy levels comparable to human transcribers, which led researchers to debate if the machine has reached human performance. Previous work focused on the English language and modular hidden Markov model-deep neural network (HMM-DNN) systems. In this paper, we perform a comprehensive benchmarking for end-to-end transformer ASR, modular HMM-DNN ASR, and human speech recognition (HSR) on the Arabic language and its dialects. For the HSR, we evaluate linguist performance and lay-native speaker performance on a new dataset collected as a part of this study. For ASR the end-to-end work led to 12.5%, 27.5% , 33.8% WER; a new performance milestone for the MGB2, MGB3, and MGB5 challenges respectively. Our results suggest that human performance in the Arabic language is still considerably better than the machine with an absolute WER gap of 3.5% on average.",
"Subsets": [],
"Dialect": "mixed",
"Language": "ar",
"Script": "Arab",
"Added_By": "qwen/qwen3.6-35b-a3b"
}
Loading