# Project_whisperTrans_Code

## Overview

whisperTrans is a video2transcript tool built upon [Whisper](https://github.com/openai/whisper)
and [WhisperX](https://github.com/m-bain/whisperX). For automated speech recognition (ASR).
The project provides an efficient implementation for running transcriptions locally on open source models from local
files and YouTube videos. It is structured to handle large-scale audio transcription tasks with accuracy and ease using
diarization and alignment techniques.

## Setup

The project is tested under python 3.10.12.
In order to edit video material ffmpeg is required. Furthermore, machine learning models are employed in this project to
convert audio content to text. Which models are used depends on the language of the video content. The models are
downloaded on demand from the model hosting platform [huggingface](https://huggingface.co/). To do so you need to create
an account and set the environment variable HF_TOKEN to your token. This is necessary to accept the conditions of the
models used.

Before running this script, make sure to install the necessary python packages. We recommend using a virtual environment
to manage dependencies. You can install the required packages with the following command:

```bash
pip install -r requirements.txt  --index-url https://download.pytorch.org/whl/cu118
```

If your system does not have ffmpeg installed you can install it via:

```bash
sudo apt install ffmpeg
```

For GPU usage you need to install the CUDA toolkit. You can install it via:

```bash
sudo apt install nvidia-cuda-toolkit-11-8
```

This might require to add the package the following way:

```bash
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda-repo-ubuntu2204-11-8-local_11.8.0-520.61.05-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-11-8-local_11.8.0-520.61.05-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-11-8-local/cuda-*-keyring.gpg /usr/share/keyrings/
```

Afterward, set the environmental variables:

```bash
echo 'export PATH=/usr/local/cuda-11.8/bin${PATH:+:${PATH}}' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc
```

and reload your `~/.bashrc` file to apply these changes:

```bash
source ~/.bashrc
```

## Prerequisites

- Python 3.10
- ffmpeg
- Access to Hugging Face for model downloads
- CUDA 11.8, and cuBLAS 11.x and cuDNN 8.x
- CUDA-compatible GPU with <8GB VRAM (recommended for faster processing)

## Usage

To generate transcripts create a new job with your specifications (see [Jobs secion](#jobs) below).
<br/>
If you need to download the transcriber model set your huggingface token:

On Linux/Mac:

```bash
export HF_TOKEN="your-token-here"
```

On Windows (Command prompt):

```cmd
set HF_TOKEN="your-token-here"
```

After setting up the token, execute the script with:

```bash
python main.py --job {name}
```

Here, `name` is the identifier for your transcription job.
Note that you might need to go to huggingface to accept the conditions of the models used. Especially when running the
transcriber for the first time on your huggingface account.

## Jobs

Jobs specify tasks for transcript creation. A job contains the following information.

- `name`: string: The name of the job
- `source`: List[string]: The sources of the videos (paths, YouTube links)
- `language`: Optional[string]: The language of the video (If not specified employs auto-detection) as a language code
  (e.g. 'en', 'de', 'fr')
- `output_format`:

## Contributing

Contributions are welcome! Please fork the repository and make your changes in a separate branch. Submit a pull request
with your proposed modifications. For major changes, please open an issue first to discuss what you would like to
change.

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

## Contact

This project is maintained by [Gerrit Anders](https://www.iwm-tuebingen.de/www/de/personen/ma.html?uid=ganders).
For any questions or support, please open an issue on this repository or reach out via email
at [g.anders@iwm-tuebingen.de](mailto:g.anders@iwm-tuebingen.de).

## Future plans

- **Tib-AV Downloader**: Add a downloader capability for [TibAV](https://av.tib.eu/) video links.
- **Progress Bar**: Implement a progress bar for better user feedback during processing.
- **Local Model Directory**: Allow models to be stored locally for offline use and faster processing.
- **Extend Mock Tests**: Add actual references for tests.

## Structure

```
project_root/
├── src/
│   ├── video_manager.py   # Contains VideoManager class
│   ├── transcriber.py    # Contains Transcriber class
│   ├── transcript_generator.py    # Contains TranscriptGenerator class
│   └── data_handling.py  # Contains download_audio and convert_to_wav methods
├── tests/    
│   ├── test_file_manager.py
│   ├── test_transcriber.py
│   └── test_transcript_generator.py
├── data/
│   ├── logs/  # logs of previous jobs
│   ├── jobs/   # place to store job files
│   └── results/   # place to store results
├── main.py    # Main script that runs the process
├── requirements.txt
├── LICENSE
└── README.md
```