About Me
Hi! I’m Haibin Wu. I am currently a third-year Ph.D. student at the college of Electrical Engineering and Computer Science (EECS) at National Taiwan University (NTU). I am a member of the Speech Processing Lab, working with Prof. Hung-yi Lee and Prof. Lin-shan Lee in the area of machine learning and speech processing. My expertise lies in speech foundation models, speech LLMs, speech enhancement, and speaker verification, audio event classification, multimodal learning and model compression. By the way. I was fortunate enough to be funded by a Google PhD Fellowship. I’m a main contributor for S3PRL v0.4.0 with 1.7k+ github stars.
I am now looking for full-time research scientist positions in the fields of speech processing and machine learning. If you have any opportunities or recommendations in these areas, I would greatly appreciate your assistance.
Publications
- SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts
Haibin Wu, Kai-Wei Chang, Yuan-Kuei Wu, Hung-yi Lee
Preprint
[ pdf] - The defender’s perspective on automatic speaker verification: An overview
Haibin Wu, Jiawen Kang, Lingwei Meng, Helen Meng, Hung-yi Lee
IJCAI DADA workshop 2023
[ pdf] - Rethinking complex-valued deep neural networks for monaural speech enhancement
Haibin Wu, Ke Tan, Buye Xu, Anurag Kumar, Daniel Wong
Interspeech 2023
[ pdf] Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator
Lingwei Meng, Jiawen Kang, Mingyu Cui, Haibin Wu, Xixin Wu, Helen Meng
Interspeech 2023
[ pdf]Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection
Xuanjun Chen, Haibin Wu, Helen Meng, Hung-yi Lee, Jyh-Shing Roger Jang
SLT 2023
[ pdf]SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning
SLT 2023
Tzu-hsun Feng, Annie Dong, Ching-Feng Yeh, Shu-wen Yang, Tzu-Quan Lin, Jiatong Shi, Kai-Wei Chang, Zili Huang, Haibin Wu, etc.
[ pdf]Spoofing-Aware Speaker Verification by Multi-Level Fusion
Haibin Wu, Lingwei Meng, Jiawen Kang, etc.
Interspeech 2022
[ pdf]MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification
Yang Zhang, Zhiqiang Lv, Haibin Wu, etc.
Interspeech 2022
[ pdf]Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
Haibin Wu, Jiawen Kang, Lingwei Meng, etc.
Odyssey 2022
[ pdf]Partially Fake Audio Detection by Self-Attention-Based Fake Span Discovery
Haibin Wu, Heng-Cheng Kuo, Naijun Zheng, Kuo-Hsuan Hung, Hung-Yi Lee, Yu Tsao, Hsin-Min Wang, Helen Meng
ICASSP 2022
[ pdf | video]Adversarial Sample Detection for Speaker Verification by Neural Vocoders
Haibin Wu, Po-chun Hsu, Ji Gao, Shanshan Zhang, Shen Huang, Jian Kang, Zhiyong Wu, Helen Meng, Hung-yi Lee
ICASSP 2022
[ pdf | Github | video]Characterizing the adversarial vulnerability of speech self-supervised learning
Haibin Wu, Bo Zheng, Xu Li, Xixin Wu, Hung-yi Lee, Helen Meng
ICASSP 2022
[ pdf | video]The CUHK-TENCENT speaker diarization system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge
Naijun Zheng, Na Li, Xixin Wu, Lingwei Meng, Jiawen Kang, Haibin Wu, Chao Weng, Dan Su, Helen Meng
ICASSP 2022
[ pdf]Voting for the right answer: Adversarial defense for speaker verification
Haibin Wu, Yang Zhang, Zhiyong Wu, Dong Wang, Hung-yi Lee
Interspeech 2021
[ pdf | Github]Adversarial defense for automatic speaker verification by cascaded self-supervised learning models
Haibin Wu, Xu Li, Andy T. Liu, Zhiyong Wu, Helen Meng, Hung-yi Lee
ICASSP 2021
[ pdf ]Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning
Haibin Wu, AT Liu, H Lee
Interspeech 2020
[ pdf | video ]Defense against adversarial attacks on spoofing countermeasures of ASV
Haibin Wu, Songxiang Liu, Helen Meng, Hung-yi Lee
ICASSP 2020
[ pdf ]Simplified Granger causality map for data-driven root cause diagnosis of process disturbances
Y Liu, HS Chen, Haibin Wu, Y Dai, Y Yao, Z Yan
Journal of Process Control
[ pdf]Adversarial attacks on spoofing countermeasures of automatic speaker verification
S Liu, H Wu, H Lee, H Meng
ASRU 2019
[ pdf ]Multiview Learning for Subsurface Defect Detection in Composite Products: A Challenge on Thermographic Data Analysis
Haibin Wu, K Zheng, S Sfarra, Y Liu, Y Yao
IEEE Transactions on Industrial Informatics
[ pdf ]Physically Consistent Soft-Sensor Development Using Sequence-to-Sequence Neural Networks
Cheng-Hung Chou (co-first), Haibin Wu (co-first), Jia-Lin Kang, David Shan-Hill Wong, Yuan Yao, etc.
IEEE Transactions on Industrial Informatics
[ pdf ]Process Monitoring Using a Sequence to Sequence Model
H Wu, CH Chou, Y Yao, DSH Wong, Y Liu
DDCLS 2019
[ pdf ]RIIT: Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning
J Hu, H Wu, SA Harding, S Liao
Preprint.
[ pdf ]- QR-MIX: Distributional Value Function Factorisation for Cooperative Multi-Agent Reinforcement Learning
Jian Hu, Seth Austin Harding, Haibin Wu, Siyue Hu, Shih-wei Liao
Preprint.
[ pdf ]
Research Experience
Research scientist intern at Meta May 2023 - present
Applied scientist intern at Amazon Sep 2022 - Dec 2022
Research scientist intern at Meta May 2022 - Aug 2022
Visiting Student at the Chinese University of Hong Kong May 2021 - April 2022
Visiting Student at SIGS of Tsinghua University Aug. 2020 - May 2021
Intern at Tencent Jan. 2021 - May 2021
Challenge
Honers
Interspeech travel grant Interspeech 2022
Appier Scholarship Appier 2022
Google PHD Fellowship Google 2021
Advanced Speech Technologies Scholarship NTU EECS 2020
Academic Achievement Award NCTU EECS 2019
Academic Achievement Award NCTU EECS 2018
National Scholarship Chinese Ministry of Education 2014
Teaching
TA of Machine Learning NTU EECS, Spring 2021