GPT-SoVITS colab training+inference tutorial

This tutorial is made by Delik. If you have any questions and/or suggestions you can contact me on discord (delik) or wechat (Dellikk)

If you need help in GPT-SoVITS, you can join the discord server for support and discussion for open source AI: https://discord.gg/CEVTUDDJyU

GPT-SoVITS is an open-source repository focused on TTS and cross language inference. Currently it only supports Chinese, English and Japanese. More languages coming soon!

Acknowledgements

First of all, I would like to thank RVC-Boss for making this repository for TTS.
There is a Chinese tutorial for GPT-SoVITS that is made by 白菜工厂1145号员工.

[Click here to jump to inference](

[TOC]Training

1, Intialize your dataset and upload your dataset to your google drive

Your dataset should be between 1 to 30 minutes. Prioritize quality over quantity.

GPT-SoVITS is made for TTS only, so it is best to remove any singing parts or parts with muffled voice. Before running GPT-SoVITS, you should perform audio denoising to enhance the quality of the model.

Bandit plus is a great model for denoising if you guys want a simple solution, you can use that model on mvsep.com. Create an account to skip queues.

2, Open training colab

Before running everything, please make sure that you are using T4 runtime. You can change the runtime by clicking change runtime type.

3, Installation

Click the installation cell, a popup will tell you to connect to your google drive.

4, Preprocessing

You only need to change these things:

Model_name: The name of the model.

Dataset_path: The path of the model. Find your audio file/folder on the left bar, right click it and click copy file/folder path and paste it in the text box.

language: The language of the dataset. Currently it only supports Chinese, Japanese and English.

Change other parameters if needed.

Finally, run the cell.

5, Text labelling


Listen to each clip and edit the text if it is not transcribed properly.
The functions are pretty self explanatory. Use next page and previous page to check the next/previous page. if you make any changes, remember to submit text. If you do not want a clip to appear in your dataset, click delete audio.

6, Run the Dataset Formatting cell

7, Training

Run the cell, adjust the parametres if needed.

DPO training for GPT model (optional)

DPO training greatly improves the performance (not audio quality wise) and stability of the model. It can infer more text at once without slicing, and there wont be any funny errors (such as repeating certain words or skipping words) when infering.

You need to have a VERY HIGH quality dataset (you need to do text labelling) to enable this. Otherwise IT WILL MAKE YOUR MODEL WORSE!!!

You need to change your batch size to 1, while keep the other settings same as above.

The final models will be saved to your drive.

Inference

1,

Edit
Pub: 09 Mar 2024 01:09 UTC
Edit: 30 Nov 2024 18:18 UTC
Views: 1770