315 lines
10 KiB
Markdown
315 lines
10 KiB
Markdown
# Movie Scheduler
|
|
|
|
Automated movie scheduler that reads from NocoDB, generates subtitles with whisper.cpp, burns them into videos with VAAPI encoding, and streams at scheduled times.
|
|
|
|
## Quick Links
|
|
|
|
- **[Production Deployment Guide](PRODUCTION.md)** - Complete guide for production setup
|
|
- **[Testing Guide](TESTING.md)** - Docker-based test environment
|
|
- **[Development Setup](#installation)** - Local development setup
|
|
|
|
## Features
|
|
|
|
- **NocoDB Integration**: Syncs job schedules from NocoDB database
|
|
- **Automatic Subtitle Generation**: Uses whisper.cpp to generate SRT subtitles
|
|
- **Hardware-Accelerated Encoding**: Burns subtitles into videos using VAAPI (h264_vaapi)
|
|
- **Scheduled Streaming**: Streams videos to RTMP server at scheduled times
|
|
- **Robust Error Handling**: Automatic retries with exponential backoff
|
|
- **Comprehensive Logging**: Tracks all operations in database log column
|
|
- **Process Management**: Properly tracks and manages streaming processes
|
|
|
|
## Requirements
|
|
|
|
- Python 3.7+
|
|
- `requests` library
|
|
- `whisper.cpp` installed and available in PATH
|
|
- `ffmpeg` with VAAPI support
|
|
- VAAPI-capable hardware (AMD/Intel GPU)
|
|
- NocoDB instance with scheduled jobs table
|
|
|
|
## Installation
|
|
|
|
1. Clone or download this repository
|
|
|
|
2. Install Python dependencies:
|
|
```bash
|
|
pip install requests
|
|
```
|
|
|
|
3. Ensure whisper.cpp is installed:
|
|
```bash
|
|
# Follow whisper.cpp installation instructions
|
|
# Make sure the binary is in your PATH
|
|
which whisper.cpp
|
|
```
|
|
|
|
4. Verify FFmpeg has VAAPI support:
|
|
```bash
|
|
ffmpeg -hwaccels | grep vaapi
|
|
```
|
|
|
|
## Configuration
|
|
|
|
1. Copy the example environment file:
|
|
```bash
|
|
cp .env.example .env
|
|
```
|
|
|
|
2. Edit `.env` and set the required variables:
|
|
```bash
|
|
# Required
|
|
export NOCODB_URL="https://nocodb/api/v2/tables/XXXX/records"
|
|
export NOCODB_TOKEN="your_token_here"
|
|
export RTMP_SERVER="rtmp://your_server/live"
|
|
|
|
# Optional (defaults provided)
|
|
export RAW_DIR="/root/surowe_filmy"
|
|
export FINAL_DIR="/root/przygotowane_filmy"
|
|
export WHISPER_MODEL="/root/models/ggml-base.bin"
|
|
export VAAPI_DEVICE="/dev/dri/renderD128"
|
|
export STREAM_GRACE_PERIOD_MINUTES="15"
|
|
export NOCODB_SYNC_INTERVAL_SECONDS="60"
|
|
export WATCHDOG_CHECK_INTERVAL_SECONDS="10"
|
|
```
|
|
|
|
3. Load the environment variables:
|
|
```bash
|
|
source .env
|
|
```
|
|
|
|
## NocoDB Table Structure
|
|
|
|
Your NocoDB table should have the following fields:
|
|
|
|
- `Id` - Unique identifier (Text/Primary Key)
|
|
- `Title` - Movie title matching filename (Text)
|
|
- `Date` - Scheduled run time (DateTime in ISO format)
|
|
|
|
The scheduler will automatically create additional columns in the local SQLite database:
|
|
- `prep_at` - Preparation time (6 hours before run_at)
|
|
- `prep_status` - Preparation status (pending/done/failed)
|
|
- `play_status` - Streaming status (pending/done/failed)
|
|
- `raw_path` - Path to source video file
|
|
- `final_path` - Path to converted video with subtitles
|
|
- `log` - Detailed operation log
|
|
|
|
## Usage
|
|
|
|
### Running the Scheduler
|
|
|
|
```bash
|
|
# Ensure environment variables are set
|
|
source .env
|
|
|
|
# Run the scheduler
|
|
python agent.py
|
|
```
|
|
|
|
The scheduler will:
|
|
1. Sync jobs from NocoDB every 60 seconds (configurable)
|
|
2. Check stream health every 10 seconds (configurable)
|
|
3. Prepare videos 6 hours before their scheduled time:
|
|
- Find matching video file in RAW_DIR
|
|
- Generate subtitles with whisper.cpp
|
|
- Encode video with burned subtitles using VAAPI
|
|
4. Stream videos at their scheduled time to RTMP server
|
|
|
|
### Stopping the Scheduler
|
|
|
|
Press `Ctrl+C` to gracefully stop the scheduler. It will:
|
|
- Stop any active streaming process
|
|
- Close the database connection
|
|
- Exit cleanly
|
|
|
|
### Restart & Recovery Behavior
|
|
|
|
The scheduler handles restarts, power outages, and downtime gracefully:
|
|
|
|
**On Startup:**
|
|
- ✅ Checks for overdue prep jobs and processes them immediately
|
|
- ⚠️ Skips jobs where the streaming time has already passed
|
|
- 📊 Logs recovery status (overdue/skipped jobs found)
|
|
|
|
**Grace Period for Late Streaming:**
|
|
- ⏰ If prep is done and streaming time is overdue by **up to 15 minutes**, will still start streaming
|
|
- ⚠️ Jobs more than 15 minutes late are marked as 'skipped'
|
|
- 📝 Late starts are logged with exact delay time
|
|
|
|
**Recovery Scenarios:**
|
|
|
|
1. **Short Outage (prep time missed, but streaming time not reached)**
|
|
- Movie scheduled for 21:00 (prep at 15:00)
|
|
- System down from 14:00-16:00
|
|
- ✅ Restarts at 16:00: immediately preps the movie
|
|
- ✅ Streams normally at 21:00
|
|
|
|
2. **Late Start Within Grace Period (< 15 minutes)**
|
|
- Movie scheduled for 21:00 (prep completed at 15:00)
|
|
- System down from 20:00-21:10
|
|
- ✅ Restarts at 21:10: starts streaming immediately (10 minutes late)
|
|
- 📝 Logged: "Starting stream 10.0 minutes late"
|
|
|
|
3. **Too Late to Stream (> 15 minutes)**
|
|
- Movie scheduled for 21:00 (prep completed)
|
|
- System down from 20:00-21:20
|
|
- ⚠️ Restarts at 21:20: marks streaming as 'skipped' (>15 min late)
|
|
- 📝 Logged: "Streaming skipped - more than 15 minutes late"
|
|
|
|
4. **Long Outage (both prep and streaming times passed, >15 min)**
|
|
- Movie scheduled for 21:00 (prep at 15:00)
|
|
- System down from 14:00-22:00
|
|
- ⚠️ Restarts at 22:00: marks entire job as 'skipped'
|
|
- 📝 Logged: "Job skipped - streaming time already passed"
|
|
|
|
5. **Crash During Processing**
|
|
- System crashes during subtitle generation or encoding
|
|
- ✅ On restart: retries from the beginning with full retry logic
|
|
- All operations are idempotent (safe to re-run)
|
|
|
|
**Database Persistence:**
|
|
- Job status stored in `scheduler.db` survives restarts
|
|
- Completed preps are never re-done
|
|
- Failed jobs stay marked as 'failed' (can be reset manually to 'pending' if needed)
|
|
|
|
## File Naming
|
|
|
|
Place your raw video files in `RAW_DIR` with names that contain the title from NocoDB.
|
|
|
|
Example:
|
|
- NocoDB Title: `"The Matrix"`
|
|
- Valid filenames: `The Matrix.mkv`, `the-matrix-1999.mp4`, `[1080p] The Matrix.avi`
|
|
|
|
The scheduler uses glob matching to find files containing the title.
|
|
|
|
## Streaming Watchdog
|
|
|
|
The scheduler includes an active watchdog that monitors streaming:
|
|
|
|
**Stream Monitoring (checked every 10 seconds):**
|
|
- ✅ Detects if ffmpeg stream crashes or exits unexpectedly
|
|
- ✅ Automatically restarts stream at the correct playback position
|
|
- ✅ Calculates elapsed time since stream start
|
|
- ✅ Seeks to correct position using `ffmpeg -ss` flag
|
|
- ✅ Limits restarts to 10 attempts to prevent infinite loops
|
|
- ✅ Marks stream as 'done' when video completes normally
|
|
- ✅ Marks stream as 'failed' after 10 failed restart attempts
|
|
|
|
**How It Works:**
|
|
1. Stream starts and status set to 'streaming' (not 'done')
|
|
2. Watchdog checks every 10 seconds if stream process is running
|
|
3. If crashed: calculates elapsed time and restarts with seek
|
|
4. If completed normally (exit code 0): marks as 'done'
|
|
5. If video duration exceeded: marks as 'done'
|
|
|
|
**Example:**
|
|
```
|
|
Stream starts at 21:00:00
|
|
Stream crashes at 21:05:30 (5.5 minutes in)
|
|
Watchdog detects crash within 10 seconds
|
|
Restarts stream with: ffmpeg -ss 330 -i video.mp4
|
|
Stream resumes from 5:30 mark
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
The scheduler includes robust error handling:
|
|
|
|
- **Automatic Retries**: Failed operations are retried up to 3 times with exponential backoff
|
|
- **Database Logging**: All operations and errors are logged to the `log` column
|
|
- **Status Tracking**: Jobs are marked as 'failed' if all retries are exhausted
|
|
- **File Verification**: Checks that subtitle files and encoded videos were created successfully
|
|
|
|
## Monitoring
|
|
|
|
### Check Job Status
|
|
|
|
Query the SQLite database:
|
|
```bash
|
|
sqlite3 scheduler.db "SELECT nocodb_id, title, prep_status, play_status FROM jobs;"
|
|
```
|
|
|
|
### View Job Logs
|
|
|
|
```bash
|
|
sqlite3 scheduler.db "SELECT nocodb_id, title, log FROM jobs WHERE nocodb_id='YOUR_JOB_ID';"
|
|
```
|
|
|
|
### Console Output
|
|
|
|
The scheduler logs to stdout with timestamps:
|
|
- INFO: Normal operations
|
|
- WARNING: Retry attempts
|
|
- ERROR: Failures
|
|
|
|
## Troubleshooting
|
|
|
|
### Configuration Validation Failed
|
|
- Ensure all required environment variables are set
|
|
- Check that NOCODB_URL is accessible
|
|
- Verify NOCODB_TOKEN has proper permissions
|
|
|
|
### No Files Found Matching Title
|
|
- Check that video files exist in RAW_DIR
|
|
- Ensure filenames contain the NocoDB title (case-insensitive glob matching)
|
|
- Review the log column in the database for exact error messages
|
|
|
|
### Subtitle Generation Failed
|
|
- Verify whisper.cpp is installed: `which whisper.cpp`
|
|
- Check WHISPER_MODEL path is correct
|
|
- Ensure the model file exists and is readable
|
|
|
|
### Video Encoding Failed
|
|
- Confirm VAAPI is available: `ffmpeg -hwaccels | grep vaapi`
|
|
- Check VAAPI device exists: `ls -l /dev/dri/renderD128`
|
|
- Verify subtitle file was created successfully
|
|
- Review FFmpeg error output in the log column
|
|
|
|
### Streaming Failed
|
|
- Test RTMP server connectivity: `ffmpeg -re -i test.mp4 -c copy -f flv rtmp://your_server/live`
|
|
- Ensure final video file exists
|
|
- Check network connectivity to RTMP server
|
|
|
|
## Development
|
|
|
|
### Database Schema
|
|
|
|
```sql
|
|
CREATE TABLE jobs (
|
|
nocodb_id TEXT PRIMARY KEY,
|
|
title TEXT,
|
|
run_at TIMESTAMP,
|
|
prep_at TIMESTAMP,
|
|
raw_path TEXT,
|
|
final_path TEXT,
|
|
prep_status TEXT, -- pending/done/failed/skipped
|
|
play_status TEXT, -- pending/streaming/done/failed/skipped
|
|
log TEXT,
|
|
stream_start_time TIMESTAMP, -- When streaming started (for restart seek)
|
|
stream_retry_count INTEGER -- Number of stream restart attempts
|
|
);
|
|
```
|
|
|
|
**Status Values:**
|
|
- `play_status='streaming'`: Stream is actively running
|
|
- `play_status='done'`: Stream completed successfully
|
|
- `play_status='failed'`: Stream failed after 10 restart attempts
|
|
- `play_status='skipped'`: Too late to stream (>15 min past scheduled time)
|
|
|
|
### Customization
|
|
|
|
Use environment variables to customize:
|
|
- `NOCODB_SYNC_INTERVAL_SECONDS` - How often to check NocoDB for new jobs (default: 60)
|
|
- `WATCHDOG_CHECK_INTERVAL_SECONDS` - How often to monitor streams (default: 10)
|
|
- `STREAM_GRACE_PERIOD_MINUTES` - Late start tolerance (default: 15)
|
|
|
|
Edit `agent.py` to customize:
|
|
- Retry attempts and delay (agent.py:99, 203, 232)
|
|
- Preparation time offset (currently 6 hours, see sync function)
|
|
- Stream restart limit (agent.py:572)
|
|
- Subtitle styling (agent.py:246)
|
|
- Video encoding parameters (agent.py:248-250)
|
|
|
|
## License
|
|
|
|
This project is provided as-is without warranty.
|