Firoj Alam

I am a Senior Scientist at Qatar Computing Research Institute. My research interest encompasses LLMs, multilingual, and multimodal representation learning.

Research Interests:

AI/Deep Learning/Machine Learning: Large Language Models, Multitask, Mulingual and Multimodal Learning,
Applicative Contexts: Natural Language Processing, Behavioral and Social Signal Processing (Emotion, Personality traits) from Speech and Text; Social Media Text Analysis.

Short Bio: Currently working for QCRI, Qatar, as a Senior Scientist. I received my PhD in Computer Science from the University of Trento, under the supervision of Prof. Giuseppe Riccardi, master’s from the same department, and undergrad from BRAC University.

Find me here: LinkedIn | Google-Scholar | Researchgate | Github | Bangla NLP

Affliated member: ACL, ACM, IEEE, AAAI, ISCA

news

Oct 23, 2024	Will present two papers at EMNLP 2024
Sep 18, 2024	Our summer interns won 2nd place among 25 projects for their project -- 'Native vs. Non-Native Language Prompting: A Comparative Analysis'
Aug 09, 2024	ArAIEval shared task at ArabicNLP 2024
Aug 01, 2024	Workshop on Detecting AI Generated Content at COLING 2025
May 23, 2024	Presented our paper titled “Can GPT-4 Identify Propaganda? Annotation and Detection of Propaganda Spans in News Articles” at LREC-COLING 2024

latest posts

Aug 09, 2024	ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content
May 23, 2024	Can GPT-4 Identify Propaganda?
Feb 03, 2024	LAraBench: Benchmarking Arabic AI with Large Language Models

selected publications

AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs

Basel Mousi, Nadir Durrani, Fatema Ahmad , and 6 more authors

arXiv preprint arXiv:2409.11404, 2024

Bib

@article{mousi2024aradicebenchmarksdialectalcultural,
  title = {{AraDiCE}: Benchmarks for Dialectal and Cultural Capabilities in LLMs},
  author = {Mousi, Basel and Durrani, Nadir and Ahmad, Fatema and Hasan, Md. Arid and Hasanain, Maram and Kabbani, Tameem and Dalvi, Fahim and Chowdhury, Shammur Absar and Alam, Firoj},
  year = {2024},
  journal = {arXiv preprint arXiv:2409.11404},
  archiveprefix = {arXiv},
  eprint = {2409.11404},
  primaryclass = {cs.CL},
  url = {https://arxiv.org/abs/2409.11404},
}

LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content

Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Maram Hasanain , and 3 more authors

arXiv preprint arXiv:2410.15308, 2024

Bib

@article{kmainasi2024llamalensspecializedmultilingualllm,
  title = {{LlamaLens}: Specialized Multilingual LLM for Analyzing News and Social Media Content},
  author = {Kmainasi, Mohamed Bayan and Shahroor, Ali Ezzat and Hasanain, Maram and Laskar, Sahinur Rahman and Hassan, Naeemul and Alam, Firoj},
  year = {2024},
  journal = {arXiv preprint arXiv:2410.15308},
  archiveprefix = {arXiv},
  eprint = {2410.15308},
  primaryclass = {cs.CL},
  url = {https://arxiv.org/abs/2410.15308},
}

NativQA: Multilingual Culturally-Aligned Natural Query for LLMs

Md. Arid Hasan, Maram Hasanain, Fatema Ahmad , and 6 more authors

2024

Bib

@article{hasan2024nativqa,
  title = {{NativQA}: Multilingual Culturally-Aligned Natural Query for LLMs},
  author = {Hasan, Md. Arid and Hasanain, Maram and Ahmad, Fatema and Laskar, Sahinur Rahman and Upadhyay, Sunaya and Sukhadia, Vrunda N and Kutlu, Mucahid and Chowdhury, Shammur Absar and Alam, Firoj},
  year = {2024},
  url = {https://arxiv.org/abs/2407.09823},
  publisher = {arXiv:2407.09823},
}

LAraBench: Benchmarking Arabic AI with Large Language Models

Ahmed Abdelali, Hamdy Mubarak, Shammur Chowdhury , and 14 more authors

In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , Mar 2024

Abs Bib

Recent advancements in Large Language Models (LLMs) have significantly influenced the landscape of language and speech research. Despite this progress, these models lack specific benchmarking against state-of-the-art (SOTA) models tailored to particular languages and tasks. LAraBench addresses this gap for Arabic Natural Language Processing (NLP) and Speech Processing tasks, including sequence tagging and content classification across different domains. We utilized models such as GPT-3.5-turbo, GPT-4, BLOOMZ, Jais-13b-chat, Whisper, and USM, employing zero and few-shot learning techniques to tackle 33 distinct tasks across 61 publicly available datasets. This involved 98 experimental setups, encompassing ~296K data points, ~46 hours of speech, and 30 sentences for Text-to-Speech (TTS). This effort resulted in 330+ sets of experiments. Our analysis focused on measuring the performance gap between SOTA models and LLMs. The overarching trend observed was that SOTA models generally outperformed LLMs in zero-shot learning, with a few exceptions. Notably, larger computational models with few-shot learning techniques managed to reduce these performance gaps. Our findings provide valuable insights into the applicability of LLMs for Arabic NLP and speech processing tasks.
@inproceedings{abdelali-etal-2024-larabench, address = {St. Julian{'}s, Malta}, author = {Abdelali, Ahmed and Mubarak, Hamdy and Chowdhury, Shammur and Hasanain, Maram and Mousi, Basel and Boughorbel, Sabri and Abdaljalil, Samir and El Kheir, Yassine and Izham, Daniel and Dalvi, Fahim and Hawasly, Majd and Nazar, Nizi and Elshahawy, Youssef and Ali, Ahmed and Durrani, Nadir and Milic-Frayling, Natasa and Alam, Firoj}, booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)}, date-modified = {2024-08-03 11:44:50 +0300}, editor = {Graham, Yvette and Purver, Matthew}, month = mar, pages = {487--520}, publisher = {Association for Computational Linguistics}, title = {{LA}ra{B}ench: Benchmarking {A}rabic {AI} with Large Language Models}, year = {2024}, url = {https://aclanthology.org/2024.eacl-long.30}, }
LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking

Fahim Dalvi, Maram Hasanain, Sabri Boughorbel , and 10 more authors

In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations , Mar 2024

Abs Bib

The recent development and success of Large Language Models (LLMs) necessitate an evaluation of their performance across diverse NLP tasks in different languages. Although several frameworks have been developed and made publicly available, their customization capabilities for specific tasks and datasets are often complex for different users. In this study, we introduce the LLMeBench framework, which can be seamlessly customized to evaluate LLMs for any NLP task, regardless of language. The framework features generic dataset loaders, several model providers, and pre-implements most standard evaluation metrics. It supports in-context learning with zero- and few-shot settings. A specific dataset and task can be evaluated for a given LLM in less than 20 lines of code while allowing full flexibility to extend the framework for custom datasets, models, or tasks. The framework has been tested on 31 unique NLP tasks using 53 publicly available datasets within 90 experimental setups, involving approximately 296K data points. We open-sourced LLMeBench for the community (https://github.com/qcri/LLMeBench/) and a video demonstrating the framework is available online (https://youtu.be/9cC2m_abk3A).
@inproceedings{dalvi-etal-2024-llmebench, address = {St. Julians, Malta}, author = {Dalvi, Fahim and Hasanain, Maram and Boughorbel, Sabri and Mousi, Basel and Abdaljalil, Samir and Nazar, Nizi and Abdelali, Ahmed and Chowdhury, Shammur Absar and Mubarak, Hamdy and Ali, Ahmed and Hawasly, Majd and Durrani, Nadir and Alam, Firoj}, booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations}, date-modified = {2024-08-03 11:44:50 +0300}, editor = {Aletras, Nikolaos and De Clercq, Orphee}, month = mar, pages = {214--222}, publisher = {Association for Computational Linguistics}, title = {{LLM}e{B}ench: A Flexible Framework for Accelerating {LLM}s Benchmarking}, year = {2024}, url = {https://aclanthology.org/2024.eacl-demo.23}, }
Large Language Models for Propaganda Span Annotation

Maram Hasanain, Fatema Ahmad, and Firoj Alam

In Findings of the Association for Computational Linguistics: EMNLP 2024 , Nov 2024

Abs Bib

The use of propagandistic techniques in online content has increased in recent years aiming to manipulate online audiences. Fine-grained propaganda detection and extraction of textual spans where propaganda techniques are used, are essential for more informed content consumption. Automatic systems targeting the task over lower resourced languages are limited, usually obstructed by lack of large scale training datasets. Our study investigates whether Large Language Models (LLMs), such as GPT-4, can effectively extract propagandistic spans. We further study the potential of employing the model to collect more cost-effective annotations. Finally, we examine the effectiveness of labels provided by GPT-4 in training smaller language models for the task. The experiments are performed over a large-scale in-house manually annotated dataset. The results suggest that providing more annotation context to GPT-4 within prompts improves its performance compared to human annotators. Moreover, when serving as an expert annotator (consolidator), the model provides labels that have higher agreement with expert annotators, and lead to specialized models that achieve state-of-the-art over an unseen Arabic testing set. Finally, our work is the first to show the potential of utilizing LLMs to develop annotated datasets for propagandistic spans detection task prompting it with annotations from human annotators with limited expertise. All scripts and annotations will be shared with the community.
@inproceedings{hasanain-etal-2024-large, title = {Large Language Models for Propaganda Span Annotation}, author = {Hasanain, Maram and Ahmad, Fatema and Alam, Firoj}, editor = {Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung}, booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024}, month = nov, year = {2024}, address = {Miami, Florida, USA}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.findings-emnlp.850}, pages = {14522--14532}, }

Discovering Latent Concepts Learned in BERT

Fahim Dalvi, Abdul Rafae Khan, Firoj Alam , and 3 more authors

In International Conference on Learning Representations , Nov 2022

Bib

@inproceedings{dalvi2022discovering,
  author = {Dalvi, Fahim and Khan, Abdul Rafae and Alam, Firoj and Durrani, Nadir and Xu, Jia and Sajjad, Hassan},
  booktitle = {International Conference on Learning Representations},
  title = {Discovering Latent Concepts Learned in {BERT}},
  year = {2022},
  url = {https://openreview.net/forum?id=POTMtpYI1xH},
}