Firoj Alam

I am a Senior Scientist at Qatar Computing Research Institute. My research interest encompasses LLMs, multilingual, and multimodal representation learning.

Research Interests:

AI/Deep Learning/Machine Learning: Large Language Models, Multitask, Mulingual and Multimodal Learning,
Applicative Contexts: Natural Language Processing, Behavioral and Social Signal Processing (Emotion, Personality traits) from Speech and Text; Social Media Text Analysis.

Short Bio: Currently working for QCRI, Qatar, as a Senior Scientist. I received my PhD in Computer Science from the University of Trento, under the supervision of Prof. Giuseppe Riccardi, master’s from the same department, and undergrad from BRAC University.

Find me here: LinkedIn | Google-Scholar | Researchgate | Github | Bangla NLP

Affliated member: ACL, ACM, IEEE, AAAI, ISCA

news

Aug 09, 2024	ArAIEval shared task at ArabicNLP 2024
Aug 01, 2024	Workshop on Detecting AI Generated Content at COLING 2025
May 23, 2024	Presented our paper titled “Can GPT-4 Identify Propaganda? Annotation and Detection of Propaganda Spans in News Articles” at LREC-COLING 2024
Apr 19, 2024	Attended to the media and information literacy event organized by UNESCO, Al Jazeera Media Institute Ministry of Education and Higher Education Qatar Qatar National Commission for Education, Cultural and Science
Feb 02, 2024	LAraBench at EACL 2024

latest posts

Aug 09, 2024	ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content
May 23, 2024	Can GPT-4 Identify Propaganda?
Feb 03, 2024	LAraBench: Benchmarking Arabic AI with Large Language Models

selected publications

NativQA: Multilingual Culturally-Aligned Natural Query for LLMs

Md. Arid Hasan, Maram Hasanain, Fatema Ahmad , and 6 more authors

2024

Bib

@article{hasan2024nativqa,
  title = {{NativQA}: Multilingual Culturally-Aligned Natural Query for LLMs},
  author = {Hasan, Md. Arid and Hasanain, Maram and Ahmad, Fatema and Laskar, Sahinur Rahman and Upadhyay, Sunaya and Sukhadia, Vrunda N and Kutlu, Mucahid and Chowdhury, Shammur Absar and Alam, Firoj},
  year = {2024},
  url = {https://arxiv.org/abs/2407.09823},
  publisher = {arXiv:2407.09823},
}

LAraBench: Benchmarking Arabic AI with Large Language Models

Ahmed Abdelali, Hamdy Mubarak, Shammur Chowdhury , and 14 more authors

In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , Mar 2024

Abs Bib

Recent advancements in Large Language Models (LLMs) have significantly influenced the landscape of language and speech research. Despite this progress, these models lack specific benchmarking against state-of-the-art (SOTA) models tailored to particular languages and tasks. LAraBench addresses this gap for Arabic Natural Language Processing (NLP) and Speech Processing tasks, including sequence tagging and content classification across different domains. We utilized models such as GPT-3.5-turbo, GPT-4, BLOOMZ, Jais-13b-chat, Whisper, and USM, employing zero and few-shot learning techniques to tackle 33 distinct tasks across 61 publicly available datasets. This involved 98 experimental setups, encompassing ~296K data points, ~46 hours of speech, and 30 sentences for Text-to-Speech (TTS). This effort resulted in 330+ sets of experiments. Our analysis focused on measuring the performance gap between SOTA models and LLMs. The overarching trend observed was that SOTA models generally outperformed LLMs in zero-shot learning, with a few exceptions. Notably, larger computational models with few-shot learning techniques managed to reduce these performance gaps. Our findings provide valuable insights into the applicability of LLMs for Arabic NLP and speech processing tasks.
@inproceedings{abdelali-etal-2024-larabench, address = {St. Julian{'}s, Malta}, author = {Abdelali, Ahmed and Mubarak, Hamdy and Chowdhury, Shammur and Hasanain, Maram and Mousi, Basel and Boughorbel, Sabri and Abdaljalil, Samir and El Kheir, Yassine and Izham, Daniel and Dalvi, Fahim and Hawasly, Majd and Nazar, Nizi and Elshahawy, Youssef and Ali, Ahmed and Durrani, Nadir and Milic-Frayling, Natasa and Alam, Firoj}, booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)}, date-modified = {2024-08-03 11:44:50 +0300}, editor = {Graham, Yvette and Purver, Matthew}, month = mar, pages = {487--520}, publisher = {Association for Computational Linguistics}, title = {{LA}ra{B}ench: Benchmarking {A}rabic {AI} with Large Language Models}, year = {2024}, url = {https://aclanthology.org/2024.eacl-long.30}, }
LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking

Fahim Dalvi, Maram Hasanain, Sabri Boughorbel , and 10 more authors

In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations , Mar 2024

Abs Bib

The recent development and success of Large Language Models (LLMs) necessitate an evaluation of their performance across diverse NLP tasks in different languages. Although several frameworks have been developed and made publicly available, their customization capabilities for specific tasks and datasets are often complex for different users. In this study, we introduce the LLMeBench framework, which can be seamlessly customized to evaluate LLMs for any NLP task, regardless of language. The framework features generic dataset loaders, several model providers, and pre-implements most standard evaluation metrics. It supports in-context learning with zero- and few-shot settings. A specific dataset and task can be evaluated for a given LLM in less than 20 lines of code while allowing full flexibility to extend the framework for custom datasets, models, or tasks. The framework has been tested on 31 unique NLP tasks using 53 publicly available datasets within 90 experimental setups, involving approximately 296K data points. We open-sourced LLMeBench for the community (https://github.com/qcri/LLMeBench/) and a video demonstrating the framework is available online (https://youtu.be/9cC2m_abk3A).
@inproceedings{dalvi-etal-2024-llmebench, address = {St. Julians, Malta}, author = {Dalvi, Fahim and Hasanain, Maram and Boughorbel, Sabri and Mousi, Basel and Abdaljalil, Samir and Nazar, Nizi and Abdelali, Ahmed and Chowdhury, Shammur Absar and Mubarak, Hamdy and Ali, Ahmed and Hawasly, Majd and Durrani, Nadir and Alam, Firoj}, booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations}, date-modified = {2024-08-03 11:44:50 +0300}, editor = {Aletras, Nikolaos and De Clercq, Orphee}, month = mar, pages = {214--222}, publisher = {Association for Computational Linguistics}, title = {{LLM}e{B}ench: A Flexible Framework for Accelerating {LLM}s Benchmarking}, year = {2024}, url = {https://aclanthology.org/2024.eacl-demo.23}, }

Large language models for propaganda span annotation

Maram Hasanain, Fatema Ahmed, and Firoj Alam

arXiv preprint arXiv:2311.09812, Mar 2023

Bib

@article{hasanain2023large,
  author = {Hasanain, Maram and Ahmed, Fatema and Alam, Firoj},
  journal = {arXiv preprint arXiv:2311.09812},
  title = {Large language models for propaganda span annotation},
  year = {2023},
}

Discovering Latent Concepts Learned in BERT

Fahim Dalvi, Abdul Rafae Khan, Firoj Alam , and 3 more authors

In International Conference on Learning Representations , Mar 2022

Bib

@inproceedings{dalvi2022discovering,
  author = {Dalvi, Fahim and Khan, Abdul Rafae and Alam, Firoj and Durrani, Nadir and Xu, Jia and Sajjad, Hassan},
  booktitle = {International Conference on Learning Representations},
  title = {Discovering Latent Concepts Learned in {BERT}},
  year = {2022},
  url = {https://openreview.net/forum?id=POTMtpYI1xH},
}