Maxime Roy (new.blacc)

---
name: "YouTube Model Feeder"
description: "Food for your model — extract transcripts, key frames, OCR, slides, and LLM summaries from YouTube videos into structured AI-ready knowledge."
version: "1.0.0"
emoji: "🧠"
homepage: "https://github.com/celstnblacc/youtube-model-feeder"
user-invocable: true
disable-model-invocation: false

requires:
  bins: ["docker"]
  anyBins: ["ffmpeg"]
  env: []
---

# YouTube Model Feeder

> **Food for your model.**

Stop pausing videos every 30 seconds to screenshot, paste into Obsidian, and caption. Every 20-minute tutorial shouldn't take an hour to document.

YouTube Model Feeder extracts everything from a YouTube video — timestamped transcript, key frame snapshots, OCR of code and slides, presentation slide detection, and LLM-generated summaries — and packages it into structured knowledge your AI assistant can search, reference, and reason about.

## Why This Exists

The problem isn't transcription — ten tools do that. The problem is **structured context**. When you feed a raw transcript to a model, it has no visual context. It doesn't know what was on screen when the speaker said "as you can see here." It can't read the code in the terminal, the diagram on the slide, or the config file being edited.

YouTube Model Feeder captures all of that. The output isn't just text — it's a knowledge bundle: transcript segments aligned to timestamps, screenshots of every key moment, OCR text from code snippets and slides, and an LLM summary that ties it all together.

**Combined with [obsidian-semantic-search](https://clawhub.ai/skills/obsidian-semantic-search)** (also on ClawHub), every video you watch becomes permanently searchable by meaning in your Obsidian vault.

## What It Extracts

### Full Pipeline

| Step | Tool | What it produces |
|------|------|-----------------|
| **Download** | yt-dlp | Video + audio + metadata (title, duration, thumbnail) |
| **Transcribe** | Whisper (Ollama) or YouTube captions | Timestamped transcript segments |
| **Frame Extraction** | FFmpeg | Key frame snapshots every 5s (configurable) |
| **Slide Detection** | SSIM analysis (OpenCV) | Identifies presentation slides via structural similarity between frames |
| **OCR** | Tesseract | Reads code, terminal output, and text from captured frames |
| **LLM Summary** | Ollama / OpenAI / Anthropic | Structured markdown with sections, code blocks, and key takeaways |

### Slide Detection (Deep)

Not just frame captures — intelligent slide boundary detection:

1. **Layout detection** — classifies video as full-frame, picture-in-picture, or split panel
2. **SSIM transition scan** — compares consecutive frames for structural changes (threshold: SSIM < 0.85)
3. **LLM disambiguation** — borderline transitions (0.85–0.93 SSIM) sent to LLM for classification
4. **Slide grouping** — merges transitions into slides with enforced minimum duration (3s)
5. **Final-state capture** — saves the last frame of each slide as JPEG
6. **OCR extraction** — runs Tesseract on each slide image
7. **Transcript alignment** — maps transcript segments to slide time ranges

### Output Formats

| Format | What you get |
|--------|-------------|
| **Markdown** | Timestamped sections with headings, code blocks, image references |
| **HTML** | Styled single-page doc with embedded screenshots |
| **Obsidian bundle** | ZIP export: markdown + images, ready to drop into your vault |

## Installation

### Prerequisites

```bash
# macOS
brew install ffmpeg tesseract

# Linux
apt install ffmpeg tesseract-ocr
```

Docker Desktop must be running for the full backend.

### Start the Stack

```bash
git clone https://github.com/celstnblacc/youtube-model-feeder.git
cd youtube-model-feeder
docker-compose up -d
```

This starts 5 services:

| Service | Port | Purpose |
|---------|------|---------|
| **api** | 8000 | FastAPI backend + Swagger docs at `/docs` |
| **celery_worker** | — | Background video processing |
| **postgres** | 5432 | Job tracking, transcripts, documents |
| **redis** | 6379 | Task queue (Celery broker) |
| **web** | 3000 | Next.js frontend (optional) |

### Verify

Open `http://localhost:8000/docs` — you should see the Swagger API documentation.

## Usage

### Via AI Assistant

**Extract a video:**
> "Extract everything from this YouTube video and save it to my vault: https://youtube.com/watch?v=..."

**Transcript only:**
> "Get the timestamped transcript for this video"

**Slides and code screenshots:**
> "Extract all the code screenshots and presentation slides from this tutorial"

**Obsidian export:**
> "Convert this video into an Obsidian note with screenshots and timestamps"

### Via API

```bash
# Submit a video for processing
curl -X POST http://localhost:8000/jobs \
  -H "Content-Type: application/json" \
  -d '{"url": "https://youtube.com/watch?v=dQw4w9WgXcQ"}'

# Check job status
curl http://localhost:8000/jobs/{job_id}

# Get the generated document
curl http://localhost:8000/videos/{video_id}
```

### Via Web UI

Open `http://localhost:3000`, paste a YouTube URL, and watch the extraction happen in real time with progress tracking.

## LLM Provider Selection

Per-user configuration — choose your summarization engine:

| Provider | Model (default) | Setup | Cost |
|----------|----------------|-------|------|
| **Ollama** (default) | Mistral 7B | Pre-installed locally | Free |
| **OpenAI** | GPT-4o-mini | Set `OPENAI_API_KEY` | Per-token |
| **Anthropic** | Claude Sonnet 4.6 | Set `ANTHROPIC_API_KEY` | Per-token |

Configure via the API: `PATCH /settings/me` with your preferred provider and API key (encrypted at rest with Fernet).

## The Knowledge Pipeline

YouTube Model Feeder is designed to work with other ClawHub skills:

```
YouTube Video
  → youtube-model-feeder (extract transcript + snapshots + OCR + summary)
    → Obsidian vault (structured markdown + images)
      → obsidian-semantic-search (index with embeddings, search by meaning)
        → Your AI assistant (full context from every video you've ever watched)
```

Every video becomes permanent, searchable knowledge. Not buried in a playlist — indexed and queryable.

## Architecture

```
youtube-model-feeder/
├── backend/           # FastAPI + Celery (Python 3.12)
│   ├── app/
│   │   ├── services/  # Core extraction pipeline (6372 LOC)
│   │   │   ├── youtube.py          # yt-dlp wrapper
│   │   │   ├── transcript.py       # Whisper + YouTube captions
│   │   │   ├── snapshot.py         # FFmpeg frame extraction
│   │   │   ├── slide_detection.py  # SSIM + OCR + alignment (643 LOC)
│   │   │   ├── llm.py             # Summarization engine (1229 LOC)
│   │   │   └── llm_providers.py   # Ollama/OpenAI/Anthropic abstraction
│   │   ├── routes/    # REST API endpoints
│   │   └── tasks.py   # Celery background jobs
├── frontend/          # Next.js 14 + React 18 + Tailwind + Zustand
├── docker-compose.yml # 5 services
└── migrations/        # Alembic database migrations
```

## Troubleshooting

| Problem | Fix |
|---------|-----|
| VirusTotal "suspicious" warning on install | False positive — skill describes video extraction patterns. Use `clawhub install youtube-model-feeder --force` |
| Docker services won't start | Ensure Docker Desktop is running. Check `docker-compose logs api` for errors |
| Transcription is slow | First run pulls the Whisper model (~1.5 GB). Subsequent runs are fast. Try YouTube captions first (faster, no model needed) |
| No slides detected | SSIM threshold may need tuning for your video type. Presentation-style videos work best |
| LLM summary is empty | Check LLM provider config. Default is Ollama — ensure Ollama is running with a model pulled |
| FFmpeg not found | `brew install ffmpeg` (macOS) or `apt install ffmpeg` (Linux) |

## Links

- **Source:** https://github.com/celstnblacc/youtube-model-feeder
- **Obsidian Semantic Search:** https://clawhub.ai/skills/obsidian-semantic-search
- **License:** MIT-0 (this skill) / Apache 2.0 (source)

---

*Built by [celstnblacc](https://github.com/celstnblacc) — food for your model. 226 tests, 6 extraction stages, 3 LLM providers, Obsidian-ready output.*

---
name: "Git Security Scanner"
description: "Unified security scanner that catches leaked secrets, credentials, and code vulnerabilities before they reach your remote. Wraps gitleaks (400+ secret patterns) and shipguard (48+ SAST rules) into a single tool with pre-commit hooks, on-demand scans, and full git history audits."
version: "1.0.0"
emoji: "🛡️"
homepage: "https://github.com/celstnblacc"
user-invocable: true
disable-model-invocation: false

requires:
  bins: ["gitleaks", "python3"]
  anyBins: ["shipguard"]
  env: []
---

# Git Security Scanner

Scan your git repositories for leaked secrets, credentials, and security vulnerabilities in one command. Combines **gitleaks** (pattern-based secret detection) and **shipguard** (48+ SAST rules across 7 security layers) into a unified scanner with merged reporting.

## What You Get

### Two Scanning Engines

| Engine | What it does | Rules |
|--------|-------------|-------|
| **gitleaks** | Pattern-based secret detection across files and git history | 400+ built-in rules, custom `.gitleaks.toml` support |
| **shipguard** | Static analysis for secrets, shell injection, code injection, supply chain, config issues | 48+ rules: SEC-001–015, SHELL-001–009, PY-001–012, JS-001–008, GHA-001–005, CFG-001–003, SC-001–006 |

### Scanning Modes

| Mode | Command | What it checks |
|------|---------|---------------|
| **Quick scan** | `git-security-scan` | Current working tree |
| **Staged only** | `git-security-scan --staged-only` | Only staged files — for pre-commit hooks |
| **Full history** | `git-security-scan --full-history` | Entire git history — finds secrets in old commits |
| **Custom severity** | `git-security-scan --severity critical` | Filter by minimum severity level |

### What It Catches

**Secrets (gitleaks + shipguard SEC rules):**
- API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, Slack, etc.)
- Database connection strings with embedded passwords
- SSH private keys and PEM files
- JWT tokens and session secrets
- Hardcoded passwords in config files
- `.env` files accidentally staged
- Credentials in comments or docstrings

**Code vulnerabilities (shipguard SAST rules):**
- Shell command injection (`SHELL-001–009`)
- Python code injection: `eval()`, `exec()`, unsafe pickle, SQL injection (`PY-001–012`)
- JavaScript injection: `innerHTML`, `eval()`, prototype pollution (`JS-001–008`)
- GitHub Actions injection: script injection, unpinned actions (`GHA-001–005`)
- Config issues: debug mode in production, permissive CORS, exposed admin routes (`CFG-001–003`)
- Supply chain: unpinned dependencies, missing lockfiles, unsigned artifacts (`SC-001–006`)

### Output Formats

| Format | Flag | Use case |
|--------|------|----------|
| **Terminal** (default) | `--format terminal` | Color-coded findings with severity icons |
| **Markdown** | `--format markdown` | PR comments, documentation, reports |
| **JSON** | `--format json` | CI/CD integration, programmatic analysis |
| **SARIF** | `--format sarif` | GitHub Security tab integration |

## Installation

### Prerequisites

```bash
# macOS
brew install gitleaks
pipx install shipguard  # or: pip install shipguard

# Linux
# gitleaks: download from https://github.com/gitleaks/gitleaks/releases
# shipguard:
pipx install shipguard
```

### Install the Skill

```bash
clawhub install git-security-scanner
```

This adds the `git-security-scan` wrapper script and the skill definition.

### Set Up Pre-Commit Hook

```bash
git-security-scan --install-hooks
```

This installs a pre-commit hook in the current repo that runs `git-security-scan --staged-only --severity high` on every commit. Commits with critical or high severity findings are blocked.

## Usage

### CLI

```bash
# Scan current directory
git-security-scan

# Scan a specific project
git-security-scan /path/to/project

# Pre-commit mode (staged files only, block on high+)
git-security-scan --staged-only --severity high

# Full git history audit
git-security-scan --full-history

# Generate a markdown report
git-security-scan --format markdown --output report.md

# JSON for CI pipelines
git-security-scan --format json --output .security-reports/scan.json

# Skip one engine
git-security-scan --skip-gitleaks   # shipguard only
git-security-scan --skip-shipguard  # gitleaks only
```

### AI Assistant Prompts

**Quick scan:**
> "Scan this repo for leaked secrets and security vulnerabilities"

**Pre-commit setup:**
> "Set up pre-commit hooks to block secrets before they're committed"

**Full history audit:**
> "Audit the entire git history for any credentials that were ever committed"

**Custom rules:**
> "Add a gitleaks rule to catch hardcoded Proxmox API tokens"

**Targeted scan:**
> "Run shipguard on just the Python files with severity high or above"

## Configuration

### gitleaks (`.gitleaks.toml`)

Create in your repo root to add custom secret patterns:

```toml
[extend]
useDefault = true

[[rules]]
id = "proxmox-api-token"
description = "Proxmox API Token"
regex = '''PVEAPIToken=[A-Za-z0-9@!]+:[A-Za-z0-9-]+'''
```

### shipguard (`.shipguard.yml`)

Create in your repo root to configure SAST rules:

```yaml
severity_threshold: medium
exclude_paths:
  - tests/fixtures/
  - node_modules/
  - "*.min.js"
disable_rules:
  - JS-008   # skip specific rule
```

### Pre-commit config (`.shipguard-precommit.yml`)

Stricter settings for the pre-commit hook:

```yaml
severity_threshold: high
exclude_paths:
  - tests/
  - docs/
```

## Report Example

Terminal output:

```
============================================================
  Git Security Scan Report
============================================================
  Directory: /home/user/my-project
  Timestamp: 2026-03-19T10:30:00Z
  Tools:     gitleaks (2 findings)
             shipguard (3 findings)

  Findings: 5  critical: 1  high: 1  medium: 3

  🔴 [CRITICAL] aws-access-key-id (gitleaks)
     config/settings.py:42
     AWS access key ID detected
     > AKIAIOSFODNN7EXAMPLE
     Fix: Rotate this credential immediately. Remove from git history.

  🟠 [HIGH] SEC-003 (shipguard)
     src/api.py:18
     Hardcoded database password in connection string
     > db_url = "postgres://admin:p4ssw0rd@localhost/app"
     Fix: Use environment variables instead of hardcoded credentials.

  🟡 [MEDIUM] PY-003 (shipguard)
     src/utils.py:55
     Use of eval() with user-controlled input
     > result = eval(user_expression)
     Fix: Use ast.literal_eval() or a safe expression parser.
```

## Remediation Workflow

When critical/high findings are detected:

1. **Rotate the credential** — Assume it's compromised if it was ever in git history
2. **Remove from history** — `git filter-repo --invert-paths --path <file>` or BFG Repo Cleaner
3. **Add to `.gitignore`** — Prevent re-staging of secret files
4. **Use environment variables** — Move secrets to `.env` (excluded from git) or a secret manager
5. **Re-scan** — Run `git-security-scan --full-history` to confirm the fix

## Troubleshooting

| Problem | Fix |
|---------|-----|
| VirusTotal "suspicious" warning on install | This is a false positive — the skill *describes* secret patterns and vulnerability types, which triggers pattern detection. Use `clawhub install git-security-scanner --force` to proceed. Review the SKILL.md yourself to confirm. |
| `gitleaks` not found | `brew install gitleaks` (macOS) or download from [gitleaks releases](https://github.com/gitleaks/gitleaks/releases) |
| `shipguard` not found | `pipx install shipguard` or `pip install shipguard` |
| No findings but secrets exist | Check if `.gitleaks.toml` or `.shipguard.yml` is excluding the path. Try `--severity low` to see all findings. |
| Scan is slow | `--full-history` scans every commit. Use default mode (working tree only) for quick checks. |

## Links

- **gitleaks:** https://github.com/gitleaks/gitleaks
- **shipguard:** https://github.com/celstnblacc (part of ai_spec_sec)
- **License:** MIT-0 (this skill) / Apache 2.0 (source tools)

---

*Built by [celstnblacc](https://github.com/celstnblacc) — gitleaks 8.30.0 + shipguard 0.3.2 (48+ SAST rules, 4 output formats).*

---
name: "Obsidian Semantic Search"
description: "Semantic search across your Obsidian vaults using local embeddings (Ollama + pgvector). 10 MCP tools: hybrid/semantic/keyword search, file CRUD, batch reads, live re-indexing, and a monitoring dashboard. Fully local — no API keys, no cloud, zero cost."
version: "1.0.0"
emoji: "🧠"
homepage: "https://github.com/celstnblacc/obsidian-semantic-mcp"
user-invocable: true
disable-model-invocation: false

requires:
  bins: ["docker", "uv"]
  anyBins: ["python3", "python"]
  env: ["OBSIDIAN_VAULT"]
---

# Obsidian Semantic Search

Search your Obsidian vault by **meaning**, not just keywords. This skill installs and configures [obsidian-semantic-mcp](https://github.com/celstnblacc/obsidian-semantic-mcp) — a local-first MCP server that indexes your vault with vector embeddings (Ollama + pgvector) and exposes 10 tools to any MCP-compatible AI assistant.

## What You Get

### 10 MCP Tools

| Tool | What it does |
|------|-------------|
| `search_vault` | Semantic, keyword, or hybrid search with similarity scores |
| `simple_search` | Fast exact-text search across all files |
| `list_files` | Browse vault directories |
| `get_file` | Read a single file |
| `get_files_batch` | Read multiple files in one call |
| `append_content` | Append text to a file (creates if missing) |
| `write_file` | Overwrite a file completely |
| `recent_changes` | List recently modified files |
| `list_indexed_notes` | See all indexed notes with timestamps |
| `reindex_vault` | Force a full re-index |

### Monitoring Dashboard (port 8484)

- Real-time service health (PostgreSQL, Ollama, embedding model)
- Indexed notes count, vault coverage %, database size
- Search testing UI — test queries without leaving your browser
- Manual re-index trigger

### Search Modes

- **Hybrid** (default): Combines semantic meaning + keyword matching for best results
- **Semantic**: Search by meaning only — finds related content even with different wording
- **Keyword**: Exact text matching via PostgreSQL full-text search

## Installation

### Prerequisites

- **Docker Desktop** (running)
- **uv** (Python package manager): `curl -LsSf https://astral.sh/uv/install.sh | sh`
- **An Obsidian vault** on your local filesystem

### One-Liner Install

```bash
bash <(curl -fsSL https://raw.githubusercontent.com/celstnblacc/obsidian-semantic-mcp/main/install.sh) --mode 2 --vault /path/to/your/vault
```

This clones the repo to `~/.local/share/obsidian-semantic-mcp`, installs the `osm` CLI, and runs the setup wizard in Docker mode.

### Manual Install

```bash
git clone https://github.com/celstnblacc/obsidian-semantic-mcp.git
cd obsidian-semantic-mcp
uv sync
uv run osm init
```

The wizard detects your OS and offers setup modes:

**macOS (4 modes):**
- **Mode 1:** Native (Homebrew — no Docker needed)
- **Mode 2:** Docker + host Ollama (if Ollama already installed)
- **Mode 3:** Full Docker (recommended — everything in containers)
- **Mode 4:** Docker + remote Ollama (SSH tunnel to a GPU server)

**Linux (3 modes):**
- **Mode 1:** Docker + host Ollama
- **Mode 2:** Full Docker (recommended)
- **Mode 3:** Docker + remote Ollama

### Verify Installation

```bash
osm status
```

Should show: Docker containers running, Ollama healthy, embedding model loaded, vault indexed.

### Register with Claude Desktop

The wizard auto-configures this, but if you need to do it manually:

Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `~/.config/Claude/claude_desktop_config.json` (Linux):

```json
{
  "mcpServers": {
    "obsidian-semantic": {
      "command": "docker",
      "args": ["exec", "-i", "obsidian-semantic-mcp-mcp-server-1", "python3", "src/server.py"]
    }
  }
}
```

Restart Claude Desktop after adding.

## Configuration

Set these in `.env` or as environment variables:

| Variable | Required | Default | Notes |
|----------|----------|---------|-------|
| `OBSIDIAN_VAULT` | Yes | — | Path to your vault |
| `OBSIDIAN_VAULTS` | No | — | Comma-separated paths for multi-vault |
| `POSTGRES_PASSWORD` | Yes (Docker) | — | Database password |
| `EMBEDDING_MODEL` | No | `nomic-embed-text` | Ollama model (~7GB) |
| `RERANK_MODEL` | No | — | Optional cross-encoder for better ranking |
| `DASHBOARD_PORT` | No | `8484` | Monitoring dashboard port |

## Usage Examples

Once installed, your AI assistant can use these tools naturally:

**Semantic search:**
> "Search my vault for notes about authentication patterns"
> → Uses `search_vault` with hybrid mode, returns top 5 matches ranked by meaning

**Find and read:**
> "Find my meeting notes from last week and summarize them"
> → Uses `recent_changes` to find files, `get_files_batch` to read them

**Append to a note:**
> "Add this action item to my project-x.md file"
> → Uses `append_content` to add text without overwriting

**Re-index after bulk changes:**
> "I just reorganized my vault, re-index everything"
> → Uses `reindex_vault` to rebuild the full index

## Maintenance

```bash
osm status       # Check health
osm rebuild      # Rebuild Docker images after updates
osm remove       # Stop and clean up everything
```

## Architecture

- **PostgreSQL 17 + pgvector**: Stores 768-dim vector embeddings, IVFFlat index for fast cosine similarity search
- **Ollama (nomic-embed-text)**: Local embedding model — no API keys, no cloud calls
- **Watchdog file watcher**: Auto-re-indexes when vault files change (0.5s debounce)
- **LRU cache**: 256 entries, 10-min TTL — skips redundant Ollama calls for repeated queries

## Troubleshooting

| Problem | Fix |
|---------|-----|
| "Ollama not responding" | Run `osm status` — if Ollama is down, the dashboard has a "Start Ollama" button |
| Slow first search | First query loads the model into memory (~5s). Subsequent queries are <1s |
| Missing notes in search | Run `osm status` to check indexing coverage. Use dashboard to trigger re-index |
| Docker errors | Ensure Docker Desktop is running. Try `osm rebuild` to rebuild images |
| VirusTotal "suspicious" warning on install | This is a false positive — the skill *describes* searching vault content, which triggers pattern detection. Use `clawhub install obsidian-semantic-search --force` to proceed. The skill is MIT-0 licensed and contains no executable code beyond documentation. |

## Links

- **Source:** https://github.com/celstnblacc/obsidian-semantic-mcp
- **Changelog:** https://github.com/celstnblacc/obsidian-semantic-mcp/blob/main/CHANGELOG.md
- **Architecture:** https://github.com/celstnblacc/obsidian-semantic-mcp/blob/main/docs/ARCHITECTURE.md
- **License:** Apache 2.0 (source repo) / MIT-0 (this skill)

---

*Built by [celstnblacc](https://github.com/celstnblacc) — 207 unit tests, Docker + native install, multi-vault support.*