100 lines
3.2 KiB
Markdown
100 lines
3.2 KiB
Markdown
# Project_whisperTrans01_Code
|
|
|
|
## Content
|
|
|
|
Transcriber code
|
|
|
|
## Setup
|
|
|
|
Before running this script, make sure to install the necessary python packages using the following command:
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
If your system does not have ffmpeg installed you can install it via:
|
|
|
|
```bash
|
|
sudo apt install ffmpeg
|
|
```
|
|
|
|
## Running the script
|
|
|
|
To generate transcripts create a new job with your specifications (see jobs section below).
|
|
<br/>
|
|
If you need to download the transcriber model set your huggingface token:
|
|
|
|
On Linux/Mac:
|
|
|
|
```bash
|
|
export HF_TOKEN="your-token-here"
|
|
```
|
|
|
|
On Windows (Command prompt):
|
|
|
|
```cmd
|
|
set HF_TOKEN="your-token-here"
|
|
```
|
|
|
|
Then run the following command to generate your transcript:
|
|
|
|
```bash
|
|
main.py --job {name}
|
|
```
|
|
|
|
where name is the name of your job.
|
|
|
|
## Jobs
|
|
|
|
Jobs specify tasks for transcript creation. A job contains the following information.
|
|
|
|
- `name`: string: The name of the job
|
|
- `source`: List[string]: The sources of the videos (paths, YouTube links, TibAV links)
|
|
- `modelpath`: Optional[string]: The path to the model if already saved locally (will be downloaded temporarily if not)
|
|
- `language`: Optional[string]: The language of the video (If not specified employs auto-detection)
|
|
|
|
## to-do
|
|
|
|
- seperate data production form code
|
|
- remember to accept the conditions on diarization if you use huggingface
|
|
- do a config json
|
|
- do option to have models already stored and point to them (is this necessary? > if you do not want to interact with
|
|
huggingface yes)
|
|
- also for single language models
|
|
- check warnings
|
|
- downloader for tib-av
|
|
- more optional setting
|
|
- add progress bar
|
|
|
|
## Structure
|
|
|
|
```
|
|
project_root/
|
|
├── src/
|
|
│ ├── file_manager.py # Contains FileManager class
|
|
│ ├── transcriber.py # Contains Transcriber class
|
|
│ ├── transcript_generator.py # Contains TranscriptGenerator class
|
|
│ ├── data_handling.py # Contains download_audio and convert_to_wav methods
|
|
│ └── main.py # Main script that runs the process
|
|
├── tests/
|
|
│ ├── test_file_manager.py
|
|
│ ├── test_transcriber.py
|
|
│ └── test_transcript_generator.py
|
|
├── data/
|
|
│ ├── audio/ # downloaded mp4 and output wav files
|
|
│ ├── models/ # models used for transcripts (are ignored by git and not necessarily)
|
|
│ └── transcripts/ # JSON and text transcript files
|
|
└── README.md
|
|
```
|
|
|
|
- **src/**: This directory will contain all your source code. Inside the src folder, I added:
|
|
- **main.py**: This will be the main entry point for the program. It will use the `TranscriptGenerator` class which
|
|
orchestrates the use of `FileManager` and `Transcriber`.
|
|
- **file_manager.py**: Contains the `FileManager` class implementation.
|
|
- **transcriber.py**: Contains the `Transcriber` class implementation.
|
|
- **transcript_generator.py**: Contains the `TranscriptGenerator` class implementation.
|
|
|
|
- **tests/**: This directory contains all your test scripts corresponding to your source files.
|
|
|
|
- **data/**: This directory contains all the data needed for your code. For simplicity, I categorize the files into
|
|
separate directories for audio, model and transcript files. |