10 KiB

Raw Blame History

Movie Scheduler

Automated movie scheduler that reads from NocoDB, generates subtitles with whisper.cpp, burns them into videos with VAAPI encoding, and streams at scheduled times.

Quick Links

Production Deployment Guide - Complete guide for production setup
Testing Guide - Docker-based test environment
Development Setup - Local development setup

Features

NocoDB Integration: Syncs job schedules from NocoDB database
Automatic Subtitle Generation: Uses whisper.cpp to generate SRT subtitles
Hardware-Accelerated Encoding: Burns subtitles into videos using VAAPI (h264_vaapi)
Scheduled Streaming: Streams videos to RTMP server at scheduled times
Robust Error Handling: Automatic retries with exponential backoff
Comprehensive Logging: Tracks all operations in database log column
Process Management: Properly tracks and manages streaming processes

Requirements

Python 3.7+
requests library
whisper.cpp installed and available in PATH
ffmpeg with VAAPI support
VAAPI-capable hardware (AMD/Intel GPU)
NocoDB instance with scheduled jobs table

Installation

Clone or download this repository
Install Python dependencies:
```
pip install requests
```

Ensure whisper.cpp is installed:

# Follow whisper.cpp installation instructions
# Make sure the binary is in your PATH
which whisper.cpp

Verify FFmpeg has VAAPI support:
```
ffmpeg -hwaccels | grep vaapi
```

Configuration

Copy the example environment file:
```
cp .env.example .env
```

Edit .env and set the required variables:

# Required
export NOCODB_URL="https://nocodb/api/v2/tables/XXXX/records"
export NOCODB_TOKEN="your_token_here"
export RTMP_SERVER="rtmp://your_server/live"

# Optional (defaults provided)
export RAW_DIR="/root/surowe_filmy"
export FINAL_DIR="/root/przygotowane_filmy"
export WHISPER_MODEL="/root/models/ggml-base.bin"
export VAAPI_DEVICE="/dev/dri/renderD128"
export STREAM_GRACE_PERIOD_MINUTES="15"
export NOCODB_SYNC_INTERVAL_SECONDS="60"
export WATCHDOG_CHECK_INTERVAL_SECONDS="10"

Load the environment variables:
```
source .env
```

NocoDB Table Structure

Your NocoDB table should have the following fields:

Id - Unique identifier (Text/Primary Key)
Title - Movie title matching filename (Text)
Date - Scheduled run time (DateTime in ISO format)

The scheduler will automatically create additional columns in the local SQLite database:

prep_at - Preparation time (6 hours before run_at)
prep_status - Preparation status (pending/done/failed)
play_status - Streaming status (pending/done/failed)
raw_path - Path to source video file
final_path - Path to converted video with subtitles
log - Detailed operation log

Usage

Running the Scheduler

# Ensure environment variables are set
source .env

# Run the scheduler
python agent.py

The scheduler will:

Sync jobs from NocoDB every 60 seconds (configurable)
Check stream health every 10 seconds (configurable)
Prepare videos 6 hours before their scheduled time:
- Find matching video file in RAW_DIR
- Generate subtitles with whisper.cpp
- Encode video with burned subtitles using VAAPI
Stream videos at their scheduled time to RTMP server

Stopping the Scheduler

Press Ctrl+C to gracefully stop the scheduler. It will:

Stop any active streaming process
Close the database connection
Exit cleanly

Restart & Recovery Behavior

The scheduler handles restarts, power outages, and downtime gracefully:

On Startup:

✅ Checks for overdue prep jobs and processes them immediately
⚠️ Skips jobs where the streaming time has already passed
📊 Logs recovery status (overdue/skipped jobs found)

Grace Period for Late Streaming:

⏰ If prep is done and streaming time is overdue by up to 15 minutes, will still start streaming
⚠️ Jobs more than 15 minutes late are marked as 'skipped'
📝 Late starts are logged with exact delay time

Recovery Scenarios:

Short Outage (prep time missed, but streaming time not reached)
- Movie scheduled for 21:00 (prep at 15:00)
- System down from 14:00-16:00
- ✅ Restarts at 16:00: immediately preps the movie
- ✅ Streams normally at 21:00
Late Start Within Grace Period (< 15 minutes)
- Movie scheduled for 21:00 (prep completed at 15:00)
- System down from 20:00-21:10
- ✅ Restarts at 21:10: starts streaming immediately (10 minutes late)
- 📝 Logged: "Starting stream 10.0 minutes late"
Too Late to Stream (> 15 minutes)
- Movie scheduled for 21:00 (prep completed)
- System down from 20:00-21:20
- ⚠️ Restarts at 21:20: marks streaming as 'skipped' (>15 min late)
- 📝 Logged: "Streaming skipped - more than 15 minutes late"
Long Outage (both prep and streaming times passed, >15 min)
- Movie scheduled for 21:00 (prep at 15:00)
- System down from 14:00-22:00
- ⚠️ Restarts at 22:00: marks entire job as 'skipped'
- 📝 Logged: "Job skipped - streaming time already passed"
Crash During Processing
- System crashes during subtitle generation or encoding
- ✅ On restart: retries from the beginning with full retry logic
- All operations are idempotent (safe to re-run)

Database Persistence:

Job status stored in scheduler.db survives restarts
Completed preps are never re-done
Failed jobs stay marked as 'failed' (can be reset manually to 'pending' if needed)

File Naming

Place your raw video files in RAW_DIR with names that contain the title from NocoDB.

Example:

NocoDB Title: "The Matrix"
Valid filenames: The Matrix.mkv, the-matrix-1999.mp4, [1080p] The Matrix.avi

The scheduler uses glob matching to find files containing the title.

Streaming Watchdog

The scheduler includes an active watchdog that monitors streaming:

Stream Monitoring (checked every 10 seconds):

✅ Detects if ffmpeg stream crashes or exits unexpectedly
✅ Automatically restarts stream at the correct playback position
✅ Calculates elapsed time since stream start
✅ Seeks to correct position using ffmpeg -ss flag
✅ Limits restarts to 10 attempts to prevent infinite loops
✅ Marks stream as 'done' when video completes normally
✅ Marks stream as 'failed' after 10 failed restart attempts

How It Works:

Stream starts and status set to 'streaming' (not 'done')
Watchdog checks every 10 seconds if stream process is running
If crashed: calculates elapsed time and restarts with seek
If completed normally (exit code 0): marks as 'done'
If video duration exceeded: marks as 'done'

Example:

Stream starts at 21:00:00
Stream crashes at 21:05:30 (5.5 minutes in)
Watchdog detects crash within 10 seconds
Restarts stream with: ffmpeg -ss 330 -i video.mp4
Stream resumes from 5:30 mark

Error Handling

The scheduler includes robust error handling:

Automatic Retries: Failed operations are retried up to 3 times with exponential backoff
Database Logging: All operations and errors are logged to the log column
Status Tracking: Jobs are marked as 'failed' if all retries are exhausted
File Verification: Checks that subtitle files and encoded videos were created successfully

Monitoring

Check Job Status

Query the SQLite database:

sqlite3 scheduler.db "SELECT nocodb_id, title, prep_status, play_status FROM jobs;"

View Job Logs

sqlite3 scheduler.db "SELECT nocodb_id, title, log FROM jobs WHERE nocodb_id='YOUR_JOB_ID';"

Console Output

The scheduler logs to stdout with timestamps:

INFO: Normal operations
WARNING: Retry attempts
ERROR: Failures

Troubleshooting

Configuration Validation Failed

Ensure all required environment variables are set
Check that NOCODB_URL is accessible
Verify NOCODB_TOKEN has proper permissions

No Files Found Matching Title

Check that video files exist in RAW_DIR
Ensure filenames contain the NocoDB title (case-insensitive glob matching)
Review the log column in the database for exact error messages

Subtitle Generation Failed

Verify whisper.cpp is installed: which whisper.cpp
Check WHISPER_MODEL path is correct
Ensure the model file exists and is readable

Video Encoding Failed

Confirm VAAPI is available: ffmpeg -hwaccels | grep vaapi
Check VAAPI device exists: ls -l /dev/dri/renderD128
Verify subtitle file was created successfully
Review FFmpeg error output in the log column

Streaming Failed

Test RTMP server connectivity: ffmpeg -re -i test.mp4 -c copy -f flv rtmp://your_server/live
Ensure final video file exists
Check network connectivity to RTMP server

Development

Database Schema

CREATE TABLE jobs (
    nocodb_id TEXT PRIMARY KEY,
    title TEXT,
    run_at TIMESTAMP,
    prep_at TIMESTAMP,
    raw_path TEXT,
    final_path TEXT,
    prep_status TEXT,  -- pending/done/failed/skipped
    play_status TEXT,  -- pending/streaming/done/failed/skipped
    log TEXT,
    stream_start_time TIMESTAMP,  -- When streaming started (for restart seek)
    stream_retry_count INTEGER    -- Number of stream restart attempts
);

Status Values:

play_status='streaming': Stream is actively running
play_status='done': Stream completed successfully
play_status='failed': Stream failed after 10 restart attempts
play_status='skipped': Too late to stream (>15 min past scheduled time)

Customization

Use environment variables to customize:

NOCODB_SYNC_INTERVAL_SECONDS - How often to check NocoDB for new jobs (default: 60)
WATCHDOG_CHECK_INTERVAL_SECONDS - How often to monitor streams (default: 10)
STREAM_GRACE_PERIOD_MINUTES - Late start tolerance (default: 15)

Edit agent.py to customize:

Retry attempts and delay (agent.py:99, 203, 232)
Preparation time offset (currently 6 hours, see sync function)
Stream restart limit (agent.py:572)
Subtitle styling (agent.py:246)
Video encoding parameters (agent.py:248-250)

License

This project is provided as-is without warranty.

10 KiB Raw Blame History