Haibin Wu

Haibin Wu

Ph.D. student
at National Taiwan University

About Me

Hi! I’m Haibin Wu, a senior applied scientist at Microsoft. I got a Ph.D. degree at National Taiwan University, working with Prof. Hung-yi Lee and Prof. Lin-shan Lee in the area of machine learning and speech processing. My expertise lies in speech foundation models, neural audio codecs, prompt engineer, speech LLMs, speech enhancement, and deepfake detection. By the way. I was fortunate enough to be funded by a Google PhD Fellowship. I’m a main contributor for S3PRL v0.4.0 with 2200+ GitHub stars. I have a keen interest in photography, and you can find my portfolio on my homepage.

Selected Publications

  • Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech
    Haibin Wu, Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Daniel Tompkins, Chung-Hsien Tsai, Canrun Li, Zhen Xiao, Sheng Zhao, Jinyu Li, Naoyuki Kanda
    SLT 2024
    [ pdf | Webpage | Github]

  • Ultra-Low Latency Speech Enhancement - A Comprehensive Study
    Haibin Wu, Sebastian Braun
    Preprint
    [ pdf]

  • SpeechPrompt: Prompting Speech Language Models for Speech Processing Tasks
    Kai-Wei Chang, Haibin Wu, Yu-Kai Wang, Yuan-Kuei Wu, Hua Shen, Wei-Cheng Tseng, Iu-thing Kang, Shang-Wen Li, Hung-yi Lee
    TASLP
    [ pdf | Webpage | Github]

  • CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems
    Haibin Wu, Yuan Tseng, Hung-yi Lee
    Interspeech 2024
    [ pdf | Webpage]

  • Singing Voice Graph Modeling for SingFake Detection
    Xuanjun Chen, Haibin Wu, Jyh-Shing Roger Jang, Hung-yi Lee
    Interspeech 2024
    [ pdf | GitHub]

  • EMO-SUPERB: An In-depth Look at Speech Emotion Recognition
    Haibin Wu, Huang-Cheng Chou, Kai-Wei Chang, Lucas Goncalves, Jiawei Du, Jyh-Shing Roger Jang, Chi-Chun Lee, Hung-Yi Lee
    Preprint
    [ pdf | Webpage | Github]

  • Codec-SUPERB: An In-Depth Analysis of Sound Codec Models
    Haibin Wu, Ho-Lam Chung, Yi-Cheng Lin, Yuan-Kuei Wu, Xuanjun Chen, Yu-Chi Pai, Hsiu-Hsuan Wang, Kai-Wei Chang, Alexander H. Liu, Hung-yi Lee
    ACL 2024 Finding
    [ pdf | Github | Leaderboard | Huggingface]

  • Towards audio language modeling - an overview
    Haibin Wu, Xuanjun Chen, Yi-Cheng Lin, Kai-wei Chang, Ho-Lam Chung, Alexander H. Liu, Hung-yi Lee
    Preprint
    [ pdf]

  • SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts
    Haibin Wu, Kai-Wei Chang, Yuan-Kuei Wu, Hung-yi Lee
    Preprint
    [ pdf | Webpage | Github]

  • The defender’s perspective on automatic speaker verification: An overview
    Haibin Wu, Jiawen Kang, Lingwei Meng, Helen Meng, Hung-yi Lee
    IJCAI DADA workshop 2023
    [ pdf]

  • Rethinking complex-valued deep neural networks for monaural speech enhancement
    Haibin Wu, Ke Tan, Buye Xu, Anurag Kumar, Daniel Wong
    Interspeech 2023
    [ pdf]

  • Partially Fake Audio Detection by Self-Attention-Based Fake Span Discovery
    Haibin Wu, Heng-Cheng Kuo, Naijun Zheng, Kuo-Hsuan Hung, Hung-Yi Lee, Yu Tsao, Hsin-Min Wang, Helen Meng
    ICASSP 2022
    [ pdf | video]

  • Adversarial Sample Detection for Speaker Verification by Neural Vocoders
    Haibin Wu, Po-chun Hsu, Ji Gao, Shanshan Zhang, Shen Huang, Jian Kang, Zhiyong Wu, Helen Meng, Hung-yi Lee
    ICASSP 2022
    [ pdf | Github | video]

  • Adversarial attacks on spoofing countermeasures of automatic speaker verification
    S Liu, H Wu, H Lee, H Meng
    ASRU 2019
    [ pdf ]

For the complete list, please visit google scholar.

Research Experience

  • Research scientist intern at Microsoft May 2024 - Aug 2024

  • Research scientist intern at Microsoft Feb 2024 - May 2024

  • Research scientist intern at Meta May 2023 - Sep 2023

  • Applied scientist intern at Amazon Sep 2022 - Dec 2022

  • Research scientist intern at Meta May 2022 - Aug 2022

  • Visiting Student at the Chinese University of Hong Kong May 2021 - April 2022

  • Visiting Student at SIGS of Tsinghua University Aug. 2020 - May 2021

  • Intern at Tencent Jan. 2021 - May 2021

Challenge

Honers

  • Google studnet travel grant Google 2024

  • ICASSP travel grant ICASSP 2024

  • Interspeech travel grant Interspeech 2022

  • Appier Scholarship Appier 2022

  • Google PHD Fellowship Google 2021

  • Advanced Speech Technologies Scholarship NTU EECS 2020

  • Academic Achievement Award NCTU EECS 2019

  • Academic Achievement Award NCTU EECS 2018

  • National Scholarship Chinese Ministry of Education 2014