Check-out: checkout 2023-10-09
This commit is contained in:
@@ -22,7 +22,7 @@ sudo apt install ffmpeg
|
|||||||
|
|
||||||
To generate transcripts create a new job with your specifications (see jobs section below).
|
To generate transcripts create a new job with your specifications (see jobs section below).
|
||||||
<br/>
|
<br/>
|
||||||
If you need to download the transcriber model set your huggingface token:
|
If you need to download the transcriber model set your huggingface token (initially running or using new language):
|
||||||
|
|
||||||
On Linux/Mac:
|
On Linux/Mac:
|
||||||
|
|
||||||
@@ -36,6 +36,8 @@ On Windows (Command prompt):
|
|||||||
set HF_TOKEN="your-token-here"
|
set HF_TOKEN="your-token-here"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If the token is not set you will be asked for it when running the software
|
||||||
|
|
||||||
Then run the following command to generate your transcript:
|
Then run the following command to generate your transcript:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -50,17 +52,14 @@ Jobs specify tasks for transcript creation. A job contains the following informa
|
|||||||
|
|
||||||
- `name`: string: The name of the job
|
- `name`: string: The name of the job
|
||||||
- `source`: List[string]: The sources of the videos (paths, YouTube links, TibAV links)
|
- `source`: List[string]: The sources of the videos (paths, YouTube links, TibAV links)
|
||||||
- `modelpath`: Optional[string]: The path to the model if already saved locally (will be downloaded temporarily if not)
|
- `modelpath`: Optional[string]: The path to the model if a specific model should be used.
|
||||||
- `language`: Optional[string]: The language of the video (If not specified employs auto-detection)
|
- `language`: Optional[string]: The language of the video (If not specified employs auto-detection)
|
||||||
|
|
||||||
## to-do
|
## to-do
|
||||||
|
|
||||||
- seperate data production form code
|
- seperate data production form code
|
||||||
- remember to accept the conditions on diarization if you use huggingface
|
- remember to accept the conditions on models if you use huggingface
|
||||||
- do a config json
|
- do a config json
|
||||||
- do option to have models already stored and point to them (is this necessary? > if you do not want to interact with
|
|
||||||
huggingface yes)
|
|
||||||
- also for single language models
|
|
||||||
- check warnings
|
- check warnings
|
||||||
- downloader for tib-av
|
- downloader for tib-av
|
||||||
- more optional setting
|
- more optional setting
|
||||||
@@ -71,19 +70,20 @@ Jobs specify tasks for transcript creation. A job contains the following informa
|
|||||||
```
|
```
|
||||||
project_root/
|
project_root/
|
||||||
├── src/
|
├── src/
|
||||||
|
│ ├── init.py # init so the main can load all necessary classes
|
||||||
│ ├── file_manager.py # Contains FileManager class
|
│ ├── file_manager.py # Contains FileManager class
|
||||||
│ ├── transcriber.py # Contains Transcriber class
|
│ ├── transcriber.py # Contains Transcriber class
|
||||||
│ ├── transcript_generator.py # Contains TranscriptGenerator class
|
│ ├── transcript_generator.py # Contains TranscriptGenerator class
|
||||||
│ ├── data_handling.py # Contains download_audio and convert_to_wav methods
|
│ ├── data_handling.py # Contains download_audio and convert_to_wav methods
|
||||||
│ └── main.py # Main script that runs the process
|
|
||||||
├── tests/
|
├── tests/
|
||||||
│ ├── test_file_manager.py
|
│ ├── test_file_manager.py
|
||||||
│ ├── test_transcriber.py
|
│ ├── test_transcriber.py
|
||||||
│ └── test_transcript_generator.py
|
│ └── test_transcript_generator.py
|
||||||
├── data/
|
├── data/
|
||||||
│ ├── audio/ # downloaded mp4 and output wav files
|
│ ├── results/ # results of the transcripts (downloaded video files, audio, JSON and text transcript files)
|
||||||
│ ├── models/ # models used for transcripts (are ignored by git and not necessarily)
|
│ ├── jobs/ # place to store jobs to run
|
||||||
│ └── transcripts/ # JSON and text transcript files
|
│ └── logs/ # logs of previous jobs
|
||||||
|
├── main.py # Initial entry point for the project
|
||||||
└── README.md
|
└── README.md
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -96,5 +96,4 @@ project_root/
|
|||||||
|
|
||||||
- **tests/**: This directory contains all your test scripts corresponding to your source files.
|
- **tests/**: This directory contains all your test scripts corresponding to your source files.
|
||||||
|
|
||||||
- **data/**: This directory contains all the data needed for your code. For simplicity, I categorize the files into
|
- **data/**: This directory contains all the data needed for your code as well as the results.
|
||||||
separate directories for audio, model and transcript files.
|
|
||||||
Reference in New Issue
Block a user