Google AI can focus on individual speakers in a crowd
San Francisco: Just as most smartphone cameras now allow users to focus on a single object among many, it may soon be possible to pick out individual voices in a crowd by suppressing all other sounds, thanks to a new Artificial Intelligence (AI) system developed by Google researchers.
This is an important development as computers as not as good as humans at focusing their attention on a particular person in a noisy environment.
Known as the cocktail party effect, the capability to mentally "mute" all other voices and sounds comes natural to us humans.
However, automatic speech separation -- separating an audio signal into its individual speech sources -- remains a significant challenge for computers, Inbar Mosseri and Oran Lang, software engineers at Google Research, wrote in a blog post this week.
In a new paper, the researchers presented a deep learning audio-visual model for isolating a single speech signal from a mixture of sounds such as other voices and background noise.
"In this work, we are able to computationally produce videos in which speech of specific people is enhanced while all other sounds are suppressed," Mosseri and Lang said.
The method works on ordinary videos with a single audio track, and all that is required from the user is to select the face of the person in the video they want to hear, or to have such a person be selected algorithmically based on context.
The researchers believe this capability can have a wide range of applications, from speech enhancement and recognition in videos, through video conferencing, to improved hearing aids, especially in situations where there are multiple people speaking.
"A unique aspect of our technique is in combining both the auditory and visual signals of an input video to separate the speech," the researchers said.
"Intuitively, movements of a person's mouth, for example, should correlate with the sounds produced as that person is speaking, which in turn can help identify which parts of the audio correspond to that person," they explained.
The visual signal not only improves the speech separation quality significantly in cases of mixed speech, but, importantly, it also associates the separated, clean speech tracks with the visible speakers in the video, the researchers said.
The remark came in the wake of an interview Zarif gave to US' CBS TV news.
China said it welcomed plans by top US officials to visit the country to discuss trade.
Bloomberg, in a CBS interview, said he hopes by next year Trump will have changed his mind.
CPI(M) will not have a political alliance with Congress but an "understanding" inside and outside Parliament to stop the forces of communalism, Sitaram Yechury said.
China has agreed to resume sharing of hydrological data of the Brahmaputra and Sutlej rivers, External Affairs Minister Sushma Swaraj said.
Fugitive Indian diamond merchant Jatin Mehta asked the Indian government to reconsider his case after he got favourable judgments from international courts in Sharjah.
Worried his arrest is imminent, Nirav Modi has started selling off some of his international assets.
The US Attorney's Office in New York has filed federal terrorism charges against a bombing suspect who was accused of detonating a pipe bomb.
The embattled liquor baron Vijay Mallya once again offered to settle his Rs 9,000 crore debt with lenders.