Yamaha Corporation announces that it has succeeded in reproducing the singing of the late Hibari Misora, a legendary Japanese vocalist, using its own Vocaloid:AI singing synthesis technology in technical cooperation with an NHK (Japan Broadcasting Corporation) television program broadcast in Japan on September 29, serving as both the public debut and first real-world implementation of Vocaloid:AI.
The NHK-led project, which Yamaha assisted with, set out to use modern AI (artificial intelligence) technology to present a live performance of a new song by Hibari Misora—an illustrious entertainer who long stood at the forefront of popular music in Japan—to commemorate the 30th anniversary of her passing. Using cutting-edge 4K 3D video to reproduce her likeness, Hibari Misora took the stage and dazzled viewers with her rendition of the new song.
Yamaha’s role in the project was to use its Vocaloid:AI technology to faithfully reproduce Hibari Misora’s characteristic singing voice and speech for the live performance. Actual recordings of the artist’s songs and speech made while she was still alive were used as machine learning data to reproduce her singing. The singing source data used for machine learning included background musical accompaniment, but Yamaha’s accompaniment suppression technology allowed for the generation of high-quality machine learning data to further improve the quality of the singing voice.
These Yamaha technologies incorporated a relatively new and rapidly developing type of AI known as deep learning to tackle the challenge of bringing back one of Japan’s foremost late singers using modern technology.
Koichi Morita, Senior General Manager of Research and Development Division, Technology Unit, Yamaha Corporation, says “We believe it was the Yamaha technologies and sensibilities cultivated over 130 years of developing and producing musical instruments and audio equipment which enabled us to successfully capture the essence of her singing. Our cooperation in this project with this new evolution of singing synthesis technology has illuminated new possibilities in music by transcending the barriers of time to dazzle listeners with incredible singing.”
Vocaloid:AI is a singing synthesis technology developed by Yamaha which incorporates AI. By using deep learning to analyze singing characteristics such as tone and expression within recordings of singing by a predetermined vocalist in any language, it can synthesize singing which includes the unique mannerisms and nuances of that vocalist with any melodies and lyrics.
Vocaloid is a singing synthesis technology developed and released by Yamaha in 2003 which has since gained wide recognition as a technology that can produce singing using virtual singers. Vocaloid:AI uses AI to vastly improve the vocal expression of tone changes, in particular.
With the public debut of Vocaloid:AI, the Vocaloid label now encompasses all of Yamaha’s singing synthesis technologies, while Vocaloid:AI specifically refers to those which incorporate AI.
Hibari Misora
Until her passing in 1989, Hibari Misora recorded over 1500 songs, leaving behind a series of hits in her more than 40 year long career as Japan’s top singer. She posthumously became the first female recipient of the People’s Honor Award, one of the highest honors in Japan.