Audio source separation for various video contents

We propose a delay-free audio source separation method that estimates the source signals of human voices and background sounds by learning background sound information from the input audio signal in real time. By applying this method to digital video devices, we aim to offer users new functions while watching video content, such as making human voices easier to hear, reducing background sound, enhancing the sense of presence in sports, and supporting singing practice. We confirmed that the method requires little processing, operates without delay on a range of devices, and lets users experience its effects immediately while watching. We also confirmed that learning voices and background sounds separately enables accurate estimation of both across a variety of content.