| layout |
|---|
default |
The Song Generation task aims to synthesize music composed of vocals and accompaniment from given lyrics. While the existing method, Jukebox, has explored this task, its constrained control over the generations often leads to deficiency in music performance. To mitigate the issue, we introduce an important concept from music composition, namely chords, to song generation networks. Chords form the foundation of accompaniment and provide vocal melody with associated harmony. Given the inaccuracy of automatic chord extractors, we devise a robust cross-attention mechanism augmented with dynamic weight sequence to integrate extracted chord information into song generations and reduce frame-level flaws, and propose a novel model termed Chord-Conditioned Song Generator (CSG) based on it. Experimental evidence demonstrates our proposed method outperforms other approaches in terms of musical performance and control precision of generated songs.
We show the songs generated by proposed CSG and other methods, conditioned by unseen lyrics and specified chords.
<style> table { width: 100%; } td { vertical-align: middle; /* 将单元格内容对齐到顶部 */ padding: 10px; /* 添加单元格内部的填充 */ } .large-cell { width: 500px; /* 调整最后一列的行距 */ } .large-img { width: 500px; /* 设置图片高度 */ } </style>| Lyrics | Chords | reference chords | Jukebox | GPT-only | concatenation | cross-attention | proposed | spectrogram of proposed |
|---|---|---|---|---|---|---|---|---|
| But thoughts, the fields to see and prize; Else may the silent feet. | 6-5-4-1 (A:min-G:maj-F:maj-C:maj) |  |
||||||
| Don't you know it's gonna be alright. | 6-4-5-1 (A:min-F:maj-G:maj-C:maj) |  |
||||||
| When I get lunch and eat my food, I hope that my model will Reason. | 1-5-6-4 (C:maj-G:maj-A:min-F:maj) |  |
||||||
| Just as long as I am there, I'll be there in your night. | 2-6-5-1 (D:min-A:min-G:maj-C:maj) |  |
||||||
| The darkest evening of the year. | 4-5-3-6 (F:maj-G:maj-E:min-A:min) |  |
||||||
| Dead puppets may, move in the bright and glorious day. | 1-2-3-4 (C:maj-D:min-E:min-F:maj) |  |