project_whispertrans_code/README.md

Project_whisperTrans01_Code

Content

Transcriber code

Setup

Before running this script, make sure to install the necessary python packages using the following command:

pip install -r requirements.txt

If your system does not have ffmpeg installed you can install it via:

sudo apt install ffmpeg

Running the script

To generate transcripts create a new job with your specifications (see jobs section below).
If you need to download the transcriber model set your huggingface token (initially running or using new language):

On Linux/Mac:

export HF_TOKEN="your-token-here"

On Windows (Command prompt):

set HF_TOKEN="your-token-here"

If the token is not set you will be asked for it when running the software

Then run the following command to generate your transcript:

main.py --job {name}

where name is the name of your job.

Jobs

Jobs specify tasks for transcript creation. A job contains the following information.

• name: string: The name of the job
• source: List[string]: The sources of the videos (paths, YouTube links, TibAV links)
• modelpath: Optional[string]: The path to the model if a specific model should be used.
• language: Optional[string]: The language of the video (If not specified employs auto-detection)

to-do

• seperate data production form code
• remember to accept the conditions on models if you use huggingface
• do a config json
• check warnings
• downloader for tib-av
• more optional setting
• add progress bar

Structure

project_root/
├── src/
│   ├── init.py # init so the main can load all necessary classes
│   ├── file_manager.py   # Contains FileManager class
│   ├── transcriber.py    # Contains Transcriber class
│   ├── transcript_generator.py    # Contains TranscriptGenerator class
│   ├── data_handling.py  # Contains download_audio and convert_to_wav methods
├── tests/    
│   ├── test_file_manager.py
│   ├── test_transcriber.py
│   └── test_transcript_generator.py
├── data/
│   ├── results/   # results of the transcripts (downloaded video files, audio, JSON and text transcript files)
│   ├── jobs/   # place to store jobs to run
│   └── logs/  # logs of previous jobs
├── main.py # Initial entry point for the project
└── README.md

• src/: This directory will contain all your source code. Inside the src folder, I added:

  • main.py: This will be the main entry point for the program. It will use the TranscriptGenerator class which orchestrates the use of FileManager and Transcriber.
  • file_manager.py: Contains the FileManager class implementation.
  • transcriber.py: Contains the Transcriber class implementation.
  • transcript_generator.py: Contains the TranscriptGenerator class implementation.

• tests/: This directory contains all your test scripts corresponding to your source files.

• data/: This directory contains all the data needed for your code as well as the results.