Check-out: checkout 2023-10-09

This commit is contained in:
Orihara Izaya
2023-10-09 18:22:11 +02:00
parent df1a3c075b
commit 101f2db788
+11 -12
View File
@@ -22,7 +22,7 @@ sudo apt install ffmpeg
To generate transcripts create a new job with your specifications (see jobs section below). To generate transcripts create a new job with your specifications (see jobs section below).
<br/> <br/>
If you need to download the transcriber model set your huggingface token: If you need to download the transcriber model set your huggingface token (initially running or using new language):
On Linux/Mac: On Linux/Mac:
@@ -36,6 +36,8 @@ On Windows (Command prompt):
set HF_TOKEN="your-token-here" set HF_TOKEN="your-token-here"
``` ```
If the token is not set you will be asked for it when running the software
Then run the following command to generate your transcript: Then run the following command to generate your transcript:
```bash ```bash
@@ -50,17 +52,14 @@ Jobs specify tasks for transcript creation. A job contains the following informa
- `name`: string: The name of the job - `name`: string: The name of the job
- `source`: List[string]: The sources of the videos (paths, YouTube links, TibAV links) - `source`: List[string]: The sources of the videos (paths, YouTube links, TibAV links)
- `modelpath`: Optional[string]: The path to the model if already saved locally (will be downloaded temporarily if not) - `modelpath`: Optional[string]: The path to the model if a specific model should be used.
- `language`: Optional[string]: The language of the video (If not specified employs auto-detection) - `language`: Optional[string]: The language of the video (If not specified employs auto-detection)
## to-do ## to-do
- seperate data production form code - seperate data production form code
- remember to accept the conditions on diarization if you use huggingface - remember to accept the conditions on models if you use huggingface
- do a config json - do a config json
- do option to have models already stored and point to them (is this necessary? > if you do not want to interact with
huggingface yes)
- also for single language models
- check warnings - check warnings
- downloader for tib-av - downloader for tib-av
- more optional setting - more optional setting
@@ -71,19 +70,20 @@ Jobs specify tasks for transcript creation. A job contains the following informa
``` ```
project_root/ project_root/
├── src/ ├── src/
│ ├── init.py # init so the main can load all necessary classes
│ ├── file_manager.py # Contains FileManager class │ ├── file_manager.py # Contains FileManager class
│ ├── transcriber.py # Contains Transcriber class │ ├── transcriber.py # Contains Transcriber class
│ ├── transcript_generator.py # Contains TranscriptGenerator class │ ├── transcript_generator.py # Contains TranscriptGenerator class
│ ├── data_handling.py # Contains download_audio and convert_to_wav methods │ ├── data_handling.py # Contains download_audio and convert_to_wav methods
│ └── main.py # Main script that runs the process
├── tests/ ├── tests/
│ ├── test_file_manager.py │ ├── test_file_manager.py
│ ├── test_transcriber.py │ ├── test_transcriber.py
│ └── test_transcript_generator.py │ └── test_transcript_generator.py
├── data/ ├── data/
│ ├── audio/ # downloaded mp4 and output wav files │ ├── results/ # results of the transcripts (downloaded video files, audio, JSON and text transcript files)
│ ├── models/ # models used for transcripts (are ignored by git and not necessarily) │ ├── jobs/ # place to store jobs to run
│ └── transcripts/ # JSON and text transcript files │ └── logs/ # logs of previous jobs
├── main.py # Initial entry point for the project
└── README.md └── README.md
``` ```
@@ -96,5 +96,4 @@ project_root/
- **tests/**: This directory contains all your test scripts corresponding to your source files. - **tests/**: This directory contains all your test scripts corresponding to your source files.
- **data/**: This directory contains all the data needed for your code. For simplicity, I categorize the files into - **data/**: This directory contains all the data needed for your code as well as the results.
separate directories for audio, model and transcript files.