Gerrit Anders
29df782f05
save in data folder and refactor save_transcript to be part of FileManager class
107 lines
3.3 KiB
Markdown
107 lines
3.3 KiB
Markdown
# Project_whisperTrans01_Code
|
|
|
|
## Content
|
|
|
|
Transcriber code
|
|
|
|
## Setup
|
|
|
|
Before running this script, make sure to install the necessary python packages using the following command:
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
If your system does not have ffmpeg installed you can install it via:
|
|
|
|
```bash
|
|
sudo apt install ffmpeg
|
|
```
|
|
|
|
## Running the script
|
|
|
|
To generate transcripts create a new job with your specifications (see jobs section below).
|
|
<br/>
|
|
If you need to download the transcriber model set your huggingface token (initially running or using new language):
|
|
|
|
On Linux/Mac:
|
|
|
|
```bash
|
|
export HF_TOKEN="your-token-here"
|
|
```
|
|
|
|
On Windows (Command prompt):
|
|
|
|
```cmd
|
|
set HF_TOKEN="your-token-here"
|
|
```
|
|
|
|
If the token is not set you will be asked for it when running the software
|
|
|
|
Then run the following command to generate your transcript:
|
|
|
|
```bash
|
|
main.py --job {name}
|
|
```
|
|
|
|
where name is the name of your job.
|
|
|
|
## Jobs
|
|
|
|
Jobs specify tasks for transcript creation. A job contains the following information.
|
|
|
|
- `name`: string: The name of the job
|
|
- `source`: List[string]: The sources of the videos (paths, YouTube links, TibAV links)
|
|
- `language`: Optional[string]: The language of the video (If not specified employs auto-detection)
|
|
|
|
## to-do
|
|
|
|
### Feature request: specific model
|
|
|
|
add an opitonal model path to the json config
|
|
- `model_path`:Optional[string]: The path to a specific language model to be used (mainly for testing)
|
|
|
|
### Refactoring: saving transcripts
|
|
|
|
let the generate transcript return a transcript object that is the saved by file manager
|
|
give file manager an init to specify the saving folder
|
|
|
|
### Other
|
|
- remember to accept the conditions on models if you use huggingface
|
|
- check warnings
|
|
- downloader for tib-av
|
|
- more optional setting
|
|
- add progress bar
|
|
|
|
## Structure
|
|
|
|
```
|
|
project_root/
|
|
├── src/
|
|
│ ├── init.py # init so the main can load all necessary classes
|
|
│ ├── file_manager.py # Contains FileManager class
|
|
│ ├── transcriber.py # Contains Transcriber class
|
|
│ ├── transcript_generator.py # Contains TranscriptGenerator class
|
|
│ ├── data_handling.py # Contains download_audio and convert_to_wav methods
|
|
├── tests/
|
|
│ ├── test_file_manager.py
|
|
│ ├── test_transcriber.py
|
|
│ └── test_transcript_generator.py
|
|
├── data/
|
|
│ ├── results/ # results of the transcripts (downloaded video files, audio, JSON and text transcript files)
|
|
│ ├── jobs/ # place to store jobs to run
|
|
│ └── logs/ # logs of previous jobs
|
|
├── main.py # Initial entry point for the project
|
|
└── README.md
|
|
```
|
|
|
|
- **src/**: This directory will contain all your source code. Inside the src folder, I added:
|
|
- **main.py**: This will be the main entry point for the program. It will use the `TranscriptGenerator` class which
|
|
orchestrates the use of `FileManager` and `Transcriber`.
|
|
- **file_manager.py**: Contains the `FileManager` class implementation.
|
|
- **transcriber.py**: Contains the `Transcriber` class implementation.
|
|
- **transcript_generator.py**: Contains the `TranscriptGenerator` class implementation.
|
|
|
|
- **tests/**: This directory contains all your test scripts corresponding to your source files.
|
|
|
|
- **data/**: This directory contains all the data needed for your code as well as the results. |