About Me

Hi! I’m Haibin Wu. I am currently a third-year Ph.D. student at the college of Electrical Engineering and Computer Science (EECS) at National Taiwan University (NTU). I am a member of the Speech Processing Lab, working with Prof. Hung-yi Lee and Prof. Lin-shan Lee in the area of machine learning and speech processing. My expertise lies in speech foundation models, speech LLMs, speech enhancement, and speaker verification, audio event classification, multimodal learning and model compression. By the way. I was fortunate enough to be funded by a Google PhD Fellowship. I’m a main contributor for S3PRL v0.4.0 with 1.7k+ github stars.

I am now looking for full-time research scientist positions in the fields of speech processing and machine learning. If you have any opportunities or recommendations in these areas, I would greatly appreciate your assistance.

Publications

  • SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts
    Haibin Wu, Kai-Wei Chang, Yuan-Kuei Wu, Hung-yi Lee
    Preprint
    [ pdf]
  • The defender’s perspective on automatic speaker verification: An overview
    Haibin Wu, Jiawen Kang, Lingwei Meng, Helen Meng, Hung-yi Lee
    IJCAI DADA workshop 2023
    [ pdf]
  • Rethinking complex-valued deep neural networks for monaural speech enhancement
    Haibin Wu, Ke Tan, Buye Xu, Anurag Kumar, Daniel Wong
    Interspeech 2023
    [ pdf]
  • Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator
    Lingwei Meng, Jiawen Kang, Mingyu Cui, Haibin Wu, Xixin Wu, Helen Meng
    Interspeech 2023
    [ pdf]

  • Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection
    Xuanjun Chen, Haibin Wu, Helen Meng, Hung-yi Lee, Jyh-Shing Roger Jang
    SLT 2023
    [ pdf]

  • SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning
    SLT 2023
    Tzu-hsun Feng, Annie Dong, Ching-Feng Yeh, Shu-wen Yang, Tzu-Quan Lin, Jiatong Shi, Kai-Wei Chang, Zili Huang, Haibin Wu, etc.
    [ pdf]

  • Spoofing-Aware Speaker Verification by Multi-Level Fusion
    Haibin Wu, Lingwei Meng, Jiawen Kang, etc.
    Interspeech 2022
    [ pdf]

  • MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification
    Yang Zhang, Zhiqiang Lv, Haibin Wu, etc.
    Interspeech 2022
    [ pdf]

  • Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
    Haibin Wu, Jiawen Kang, Lingwei Meng, etc.
    Odyssey 2022
    [ pdf]

  • Partially Fake Audio Detection by Self-Attention-Based Fake Span Discovery
    Haibin Wu, Heng-Cheng Kuo, Naijun Zheng, Kuo-Hsuan Hung, Hung-Yi Lee, Yu Tsao, Hsin-Min Wang, Helen Meng
    ICASSP 2022
    [ pdf | video]

  • Adversarial Sample Detection for Speaker Verification by Neural Vocoders
    Haibin Wu, Po-chun Hsu, Ji Gao, Shanshan Zhang, Shen Huang, Jian Kang, Zhiyong Wu, Helen Meng, Hung-yi Lee
    ICASSP 2022
    [ pdf | Github | video]

  • Characterizing the adversarial vulnerability of speech self-supervised learning
    Haibin Wu, Bo Zheng, Xu Li, Xixin Wu, Hung-yi Lee, Helen Meng
    ICASSP 2022
    [ pdf | video]

  • The CUHK-TENCENT speaker diarization system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge
    Naijun Zheng, Na Li, Xixin Wu, Lingwei Meng, Jiawen Kang, Haibin Wu, Chao Weng, Dan Su, Helen Meng
    ICASSP 2022
    [ pdf]

  • Voting for the right answer: Adversarial defense for speaker verification
    Haibin Wu, Yang Zhang, Zhiyong Wu, Dong Wang, Hung-yi Lee
    Interspeech 2021
    [ pdf | Github]

  • Adversarial defense for automatic speaker verification by cascaded self-supervised learning models
    Haibin Wu, Xu Li, Andy T. Liu, Zhiyong Wu, Helen Meng, Hung-yi Lee
    ICASSP 2021
    [ pdf ]

  • Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning
    Haibin Wu, AT Liu, H Lee
    Interspeech 2020
    [ pdf | video ]

  • Defense against adversarial attacks on spoofing countermeasures of ASV
    Haibin Wu, Songxiang Liu, Helen Meng, Hung-yi Lee
    ICASSP 2020
    [ pdf ]

  • Simplified Granger causality map for data-driven root cause diagnosis of process disturbances
    Y Liu, HS Chen, Haibin Wu, Y Dai, Y Yao, Z Yan
    Journal of Process Control
    [ pdf]

  • Adversarial attacks on spoofing countermeasures of automatic speaker verification
    S Liu, H Wu, H Lee, H Meng
    ASRU 2019
    [ pdf ]

  • Multiview Learning for Subsurface Defect Detection in Composite Products: A Challenge on Thermographic Data Analysis
    Haibin Wu, K Zheng, S Sfarra, Y Liu, Y Yao
    IEEE Transactions on Industrial Informatics
    [ pdf ]

  • Physically Consistent Soft-Sensor Development Using Sequence-to-Sequence Neural Networks
    Cheng-Hung Chou (co-first), Haibin Wu (co-first), Jia-Lin Kang, David Shan-Hill Wong, Yuan Yao, etc.
    IEEE Transactions on Industrial Informatics
    [ pdf ]

  • Process Monitoring Using a Sequence to Sequence Model
    H Wu, CH Chou, Y Yao, DSH Wong, Y Liu
    DDCLS 2019
    [ pdf ]

  • RIIT: Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning
    J Hu, H Wu, SA Harding, S Liao
    Preprint.
    [ pdf ]

  • QR-MIX: Distributional Value Function Factorisation for Cooperative Multi-Agent Reinforcement Learning
    Jian Hu, Seth Austin Harding, Haibin Wu, Siyue Hu, Shih-wei Liao
    Preprint.
    [ pdf ]

Research Experience

  • Research scientist intern at Meta May 2023 - present

  • Applied scientist intern at Amazon Sep 2022 - Dec 2022

  • Research scientist intern at Meta May 2022 - Aug 2022

  • Visiting Student at the Chinese University of Hong Kong May 2021 - April 2022

  • Visiting Student at SIGS of Tsinghua University Aug. 2020 - May 2021

  • Intern at Tencent Jan. 2021 - May 2021

Challenge

Honers

  • Interspeech travel grant Interspeech 2022

  • Appier Scholarship Appier 2022

  • Google PHD Fellowship Google 2021

  • Advanced Speech Technologies Scholarship NTU EECS 2020

  • Academic Achievement Award NCTU EECS 2019

  • Academic Achievement Award NCTU EECS 2018

  • National Scholarship Chinese Ministry of Education 2014

Teaching