Papers
2024
- CoDi: Conversational Distillation for Grounded Question Answering
Patrick Huber*, Arash Einolghozati, Rylan Conway, Kanika Narang, Matt Smith, Waqar Nayyar, Adithya Sagar, Ahmed Aly, Akshat Shrivastava* (co-first author)
Pre-print
- MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts
Xi Victoria Lin*, Akshat Shrivastava* (co-first author), Liang Luo, Srinivasan Iyer, Mike Lewis, Gargi Ghosh, Luke Zettlemoyer, Armen Aghajanyan*
Pre-print
- PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding
Trang Le*, Daniel Lazar*, Suyoun Kim, Shan Jiang, Duc Le, Adithya Sagar, Aleksandr Livshits, Ahmed Aly, Akshat Shrivastava
EMNLP 2024 Findings
- Chameleon: Mixed-Modal Early-Fusion Foundation Models | Github | Blog | Model Checkpoints
Chameleon Team (member of pre-training team)
Pre-print
- Layer skip: Enabling early exit inference and self-speculative decoding
Mostafa Elhoushi*, Akshat Shrivastava* (co-first author), Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, Carole-Jean Wu
ACL 2024
- PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs
Charlie Hou, Akshat Shrivastava, Hongyuan Zhan, Rylan Conway, Trang Le, Adithya Sagar, Giulia Fanti, Daniel Lazar
ICML 2024 Oral Presentation
- Small But Funny: A Feedback-Driven Approach to Humor Distillation
Sahithya Ravi, Patrick Huber, Akshat Shrivastava, Aditya Sagar, Ahmed Aly, Vered Shwartz, Arash Einolghozati
ACL 2024
2023
- ICASSP 2023 SPOKEN LANGUAGE UNDERSTANDING GRAND CHALLENGE | Challenge Website
Akshat Shrivastava, Suyoun Kim, Paden Tomasello, Ali Elkahky, Daniel Lazar,Trang Le, Shan Jiang, Duc Le, Aleksandr Livshits, Ahmed Aly
ICASSP 2023
- Augmenting text for spoken language understanding with Large Language Models
Roshan Sharma, Suyoun Kim, Daniel Lazar, Trang Le, Akshat Shrivastava, Kwanghoon Ahn, Piyush Kansal, Leda Sari, Ozlem Kalinli, Michael Seltzer
Preprint
- Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding
Suyoun Kim, Akshat Shrivastava, Duc Le, Ju Lin, Ozlem Kalinli, Michael L. Seltzer
Interspeech 2022
- TreePiece: Faster Semantic Parsing via Tree Tokenization
Sid Wang, Akshat Shrivastava, Sasha Livshits
EMNLP Findings 2022
- Privately Customizing Prefinetuning to Better Match User Data in Federated Learning
Charlie Hou, Hongyuan Zhan, Akshat Shrivastava, Sida I. Wang, Sasha Livshits, Giulia C. Fanti, Daniel Lazar
Workshop on the pitfalls of limited data and computation for Trustworthy ML, ICLR 2023
2022
- Introducing Semantics into Speech Encoders
Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Alexei Baevski, Guan-Ting Lin, Hung-yi Lee, Yizhou Sun, Wei Wang.
ACL 2023
- Data-Efficiency with a Single GPU: An Exploration of Transfer Methods for Small Language Models.
Alon Albalak, Akshat Shrivastava, Chinnadhurai Sankar, Adithya Sagar, Mike Ross.
Preprint
- STOP: A dataset for Spoken Task Oriented Semantic Parsing
Paden Tomasello, Akshat Shrivastava, Daniel Lazar, Po-Chun Hsu, Duc Le, Adithya Sagar, Ali Elkahky, Jade Copet, Wei-Ning Hsu, Yossi Adi, Robin Algayres, Tu Ahn Nguyen, Emmanuel Dupoux, Luke Zettlemoyer, Abdelrahman Mohamed.
SLT 2023
- Deliberation Model for On-Device Spoken Language Understanding
Duc Le*, Akshat Shrivastava* (co-first author), Paden Tomasello, Suyoun Kim, Aleksandr Livshits, Ozlem Kalinli, Michael L. Seltzer.
Interspeech 2022
- Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing
Akshat Shrivastava, Shrey Desai, Anchit Gupta, Ali Elkahky, Aleksandr Livshits, Alexander Zotov, Ahmed Aly
EACL 2023
2021
- RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing
Vivek Gupta, Akshat Shrivastava, Adithya Sagar, Armen Aghajanyan, and Denis Savenkov
🏆 Outstanding Paper - ConvAI@ACL 2022
- Assessing Data Efficiency in Task-Oriented Semantic Parsing
Shrey Desai, Akshat Shrivastava, Justin Rill, Brian Moran, Safiyyah Saleem, Alexander Zotov, and Ahmed Aly
Preprint
- Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization
David Eriksson, Pierce I-Jen Chuang, Sam Daulton, Peng Xia, Akshat Shrivastava, Arun Babu, Shicong Zhao, Ahmed Aly, Ganesh Venkatesh, and Maximilian Balandat
ICML Workshop on Automated Machine Learning 2021
- Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing
Akshat Shrivastava, Pierce Chuang, Arun Babu, Shrey Desai, Abhinav Arora, Alexander Zotov, and Ahmed Aly
EMNLP 2021 Findings
- Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modeling
Shrey Desai, Akshat Shrivastava, Alexander Zotov, and Ahmed Aly
Preprint
- Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
Arun Babu, Akshat Shrivastava, Armen Aghajanyan, Ahmed Aly, Angela Fan, and Marjan Ghazvininejad
NAACL 2021
- MUPPET: Massive Multi-task Representations with Pre-Finetuning
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta
EMNLP 2021
2020
- Conversational Semantic Parsing
Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Mike Haeger, Haoran Li, Yashar Mehdad, Ves Stoyanov, Anuj Kumar, Mike Lewis, and Sonal Gupta
EMNLP 2020
- Cross-lingual Transfer Learning for Intent Detection of Covid-19 Utterances
Abhinav Arora, Akshat Shrivastava, Mrinal Mohit, Lorena Sainz-Maza Lecanda, and Ahmed Aly
Preprint
- Better Fine-Tuning by Reducing Representational Collapse
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta
ICLR 2021
- iSeqL Interactive Sequence Learning
Akshat Shrivastava and Jeffrey Heer
IUI 2020
Workshops and Challenges
Blog Posts
Organizer
Program Comittee
