neo1307

---
name: flutter-release-pipeline
description: Build and package Flutter Android release artifacts (APK/AAB), collect outputs into a single folder, and produce a short release checklist. Use when the user asks to build APK/AAB, prepare a Play Store release, bump version, gather mapping/symbols, or generate a deterministic release folder for sharing.
---

# Flutter Android Release Pipeline

## Workflow

1) **Preflight**
- Confirm `flutter --version` works.
- Confirm project path contains `pubspec.yaml`.

2) **Build** (choose one)
- AAB: `flutter build appbundle --release`
- APK: `flutter build apk --release` (optionally `--split-per-abi`)

3) **Collect artifacts**
Create `out/flutter_release_<timestamp>/` and copy:
- `build/app/outputs/flutter-apk/*.apk` (if APK build)
- `build/app/outputs/bundle/release/*.aab` (if AAB build)
- `build/app/outputs/mapping/release/mapping.txt` (if present)
- `pubspec.yaml` (snapshot)

4) **Report**
- Print paths + sizes + SHA256 for each artifact.
- Print a short checklist (versionCode/versionName sanity, signing, Play Console notes).

## Script
Run (PowerShell):
- `powershell -ExecutionPolicy Bypass -File scripts/flutter_release.ps1 -Project "<path>" -Mode aab|apk -SplitPerAbi:$false`

## Notes
- Avoid changing app code unless explicitly requested.
- If build fails, return the exact error + suggested fix.

---
name: drive-file-relay
description: Copy (relay) a local file from this machine into Google Drive for desktop (GoogleDriveFS) so it can be downloaded from a phone when the phone is not on the same LAN. Use when the user says they are on the phone, cannot access the PC, and needs an APK/ZIP/PDF/any file moved into Drive; also use when you must verify size+SHA256 after copy.
---

# Drive File Relay (Google Drive for desktop)

## Workflow

1) **Preflight**
- Confirm source path exists.
- Detect Drive mount:
  - Prefer `G:\Drive'ım\` (TR) and `G:\My Drive\` (EN).
  - If not found, list filesystem drives and look for a drive containing `.shortcut-targets-by-id`.

2) **Copy + verify** (no partial success)
- Create destination folder if missing.
- Copy file.
- Compute SHA256 of source and destination and ensure they match.
- Report:
  - src path
  - dst path
  - size bytes + MB
  - sha256

3) **User instruction**
- Tell the user where to find it on the phone: Google Drive → My Drive/Drive'ım → `<folder>`.

## Script
Run (PowerShell):
- `powershell -ExecutionPolicy Bypass -File scripts/drive_relay.ps1 -Src "<path>" -DstFolder "ArgusShare" -DstName "<optional>"`

### Notes
- Do **not** attempt provider messaging via curl/Telegram Bot API.
- This skill does **not** create share links; it only ensures the file is present in the user’s Drive.

---
name: browser-agent
description: Browser automation with Playwright for opening pages, taking screenshots, finding or clicking DOM elements, filling forms, extracting text, and managing cookies/session state. Use when Codex needs deterministic browser control against real web pages, smoke-testing a site, grabbing DOM evidence, or automating straightforward UI flows.
---

# Browser Agent

Use Playwright-backed browser control for reproducible page actions.

## Workflow

1. Run `index.js` with a target URL.
2. Use action flags to open, screenshot, extract title/text, click selectors, fill inputs, or save/load session state.
3. Keep selectors explicit when clicking or filling.
4. Save evidence (screenshots / extracted text) to `out/`.

## Supported actions
- page open
- screenshot
- title read
- text extraction by selector
- click by selector
- fill input/textarea by selector
- save cookies/storage state
- load cookies/storage state

## Example

```bash
node skills/browser-agent/index.js --url https://example.com --screenshot out/example.png --title --extract h1
```

FILE:index.js
const fs = require('fs');
const path = require('path');
const { chromium } = require('playwright');

function arg(name) {
  const i = process.argv.indexOf(name);
  return i >= 0 ? process.argv[i + 1] : null;
}
function has(name) { return process.argv.includes(name); }
function args(name) {
  const out = [];
  for (let i = 0; i < process.argv.length; i++) if (process.argv[i] === name && process.argv[i + 1]) out.push(process.argv[i + 1]);
  return out;
}
(async () => {
  const url = arg('--url');
  if (!url) throw new Error('Usage: node index.js --url <url> [--screenshot <path>] [--title] [--extract <selector>] [--click <selector>] [--fill <selector=value>] [--save-session <path>] [--load-session <path>]');
  const headless = !has('--headed');
  const browser = await chromium.launch({ headless });
  const contextOpts = {};
  const loadSession = arg('--load-session');
  if (loadSession && fs.existsSync(path.resolve(loadSession))) contextOpts.storageState = path.resolve(loadSession);
  const context = await browser.newContext(contextOpts);
  const page = await context.newPage();
  await page.goto(url, { waitUntil: 'domcontentloaded' });
  const result = { ok: true, url, title: null, extracted: [], screenshot: null, clicked: [], filled: [], sessionSaved: null };
  if (has('--title')) result.title = await page.title();
  for (const selector of args('--click')) {
    await page.locator(selector).first().click();
    result.clicked.push(selector);
  }
  for (const pair of args('--fill')) {
    const idx = pair.indexOf('=');
    if (idx <= 0) continue;
    const selector = pair.slice(0, idx);
    const value = pair.slice(idx + 1);
    await page.locator(selector).first().fill(value);
    result.filled.push({ selector, value });
  }
  for (const selector of args('--extract')) {
    const text = await page.locator(selector).first().textContent();
    result.extracted.push({ selector, text });
  }
  const screenshot = arg('--screenshot');
  if (screenshot) {
    const out = path.resolve(screenshot);
    fs.mkdirSync(path.dirname(out), { recursive: true });
    await page.screenshot({ path: out, fullPage: true });
    result.screenshot = out;
  }
  const saveSession = arg('--save-session');
  if (saveSession) {
    const out = path.resolve(saveSession);
    fs.mkdirSync(path.dirname(out), { recursive: true });
    await context.storageState({ path: out });
    result.sessionSaved = out;
  }
  await context.close();
  await browser.close();
  process.stdout.write(JSON.stringify(result, null, 2) + '\n');
})().catch(err => {
  process.stderr.write(String(err && err.stack || err) + '\n');
  process.exit(1);
});

---
name: env-health-check
description: Inspect environment variables, critical directories, and write permissions, then produce a health report. Use when validating deployment readiness, local runtime setup, workspace health, or missing env/config prerequisites.
---

# Env Health Check

Inspect runtime prerequisites and emit a compact health report.

## Workflow

1. Pass required env var names with `--env` flags.
2. Pass critical directories with `--dir` flags.
3. Run `index.js` and review OK/WARN/FAIL output.
4. Treat missing write access as a blocker unless the path is intentionally read-only.

FILE:index.js
const fs=require('fs'); const path=require('path');
function args(name){const out=[]; for(let i=0;i<process.argv.length;i++) if(process.argv[i]===name && process.argv[i+1]) out.push(process.argv[i+1]); return out;}
const envs=args('--env'); const dirs=args('--dir'); const out=process.argv.includes('--out')?process.argv[process.argv.indexOf('--out')+1]:path.join(process.cwd(),'out','env_health_report.md');
const findings=[];
for(const name of envs){findings.push({type:'env',name,status:process.env[name]?'OK':'WARN'});}
for(const d of dirs){const abs=path.resolve(d); const exists=fs.existsSync(abs); let writable=false; if(exists){ try{ const test=path.join(abs,'.env-health-check.tmp'); fs.writeFileSync(test,'ok','utf8'); fs.unlinkSync(test); writable=true; }catch{} }
findings.push({type:'dir',path:abs,status:!exists?'FAIL':(writable?'OK':'WARN')}); }
const summary=findings.reduce((m,f)=>(m[f.status]=(m[f.status]||0)+1,m),{});
const lines=['# Environment Health Report','',`- OK: summary.OK||0`,`- WARN: summary.WARN||0`,`- FAIL: summary.FAIL||0`,'','## Findings'];
for(const f of findings){ if(f.type==='env') lines.push(`- [f.status] env f.name`); else lines.push(`- [f.status] dir f.path`); }
fs.mkdirSync(path.dirname(out),{recursive:true}); fs.writeFileSync(path.resolve(out),lines.join('\n'),'utf8');
console.log(JSON.stringify({ok:true,out:path.resolve(out),summary,findings},null,2));

FILE:README.md
# env-health-check

Checks required environment variables, critical directories, and write permissions.

## Usage

```bash
node index.js --env OPENAI_API_KEY --env ANTHROPIC_API_KEY --dir ./memory --dir ./out --out ./out/env_health_report.md
```

FILE:skill.yaml
name: env-health-check
version: 1.0.0
description: Checks environment variables, critical directories, and write permissions; produces a health report.
author: neo1307
tags:
  - environment
  - health
  - diagnostics
entrypoint: index.js

---
name: log-analyzer
description: Read a log file, extract ERROR/WARN/CRITICAL lines, group similar messages, and produce a concise summary report. Use when analyzing application logs, agent logs, service logs, or audit outputs for repeated failures, warning clusters, or critical events.
---

# Log Analyzer

Read a target log file, isolate important severity lines, group similar messages, and emit a Markdown summary.

## Workflow

1. Confirm the input log path.
2. Run `index.js` with `--input <logfile>` and optional `--out <report.md>`.
3. Review the grouped output for dominant error families.
4. Use the report as a triage artifact, not as the only source of truth.

## Output

Always include:
- total scanned lines
- WARN / ERROR / CRITICAL counts
- grouped issue buckets
- sample lines
- suggested first checks

FILE:index.js
const fs = require('fs');
const path = require('path');
function arg(name){const i=process.argv.indexOf(name);return i>=0?process.argv[i+1]:null;}
const input=arg('--input');
const out=arg('--out') || path.join(process.cwd(),'out','log_analysis_report.md');
if(!input){console.error('Usage: node index.js --input <logfile> [--out <report.md>]');process.exit(1);}
const lines=fs.readFileSync(path.resolve(input),'utf8').replace(/^\uFEFF/,'').split(/\r?\n/);
const sev=['CRITICAL','ERROR','WARN'];
const matches=[];
for(const line of lines){const s=sev.find(x=>line.includes(x)); if(s) matches.push({severity:s,line});}
const normalize=s=>s.replace(/\d+/g,'<num>').replace(/[a-f0-9]{8,}/gi,'<id>').replace(/\s+/g,' ').trim();
const groups=new Map();
for(const m of matches){const key=`m.severity|normalize(m.line)`; if(!groups.has(key)) groups.set(key,{severity:m.severity,count:0,sample:m.line}); groups.get(key).count++;}
const ordered=[...groups.values()].sort((a,b)=>b.count-a.count);
const counts=matches.reduce((m,x)=>(m[x.severity]=(m[x.severity]||0)+1,m),{});
const report=[];
report.push('# Log Analysis Report','',`- Total scanned lines: lines.length`,`- WARN: counts.WARN||0`,`- ERROR: counts.ERROR||0`,`- CRITICAL: counts.CRITICAL||0`,'','## Grouped findings');
if(!ordered.length) report.push('- No WARN/ERROR/CRITICAL lines found.');
else ordered.slice(0,50).forEach((g,i)=>report.push(`- i+1. [g.severity] xg.count — g.sample`));
report.push('','## Suggested first checks','- Start from CRITICAL groups, then highest-frequency ERROR groups.','- Verify whether repeated warnings are expected noise or precursors to failure.');
fs.mkdirSync(path.dirname(out),{recursive:true}); fs.writeFileSync(path.resolve(out),report.join('\n'),'utf8');
console.log(JSON.stringify({ok:true,out:path.resolve(out),groups:ordered.length,counts},null,2));

FILE:README.md
# log-analyzer

Reads a log file, extracts WARN/ERROR/CRITICAL lines, groups similar messages, and writes a Markdown report.

## Usage

```bash
node index.js --input ./app.log --out ./out/log_analysis_report.md
```

FILE:skill.yaml
name: log-analyzer
version: 1.0.0
description: Reads a log file, groups WARN/ERROR/CRITICAL lines, and produces a summary report.
author: neo1307
tags:
  - logs
  - analysis
  - diagnostics
entrypoint: index.js

---
name: skill-release-guard
description: Validate a local skill folder before publishing or sharing it. Use when Codex is about to release a skill, publish to ClawHub, audit SKILL.md quality, check naming/frontmatter/resource hygiene, or generate a release checklist for a skill package.
---

# Skill Release Guard

Use this skill right before packaging or publishing a skill.

## Workflow

1. Inspect the target skill directory.
2. Confirm `SKILL.md` exists and frontmatter has only `name` and `description`.
3. Check naming convention, folder layout, and presence of obvious clutter files.
4. Run `scripts/check_skill_release.js` to generate a compact release checklist.
5. Fix guard failures before publishing.

## Script

```bash
node skills/skill-release-guard/scripts/check_skill_release.js \
  --skill skills/my-skill \
  --out out/my-skill-release-check.json
```

## Guardrails

- Fail if the folder name and frontmatter name diverge.
- Flag clutter docs like README/CHANGELOG/INSTALL guides.
- Flag missing description or overly vague description.
- Treat script syntax failures as release blockers.

FILE:scripts/check_skill_release.js
const fs = require('fs');
const path = require('path');
const { spawnSync } = require('child_process');
function arg(name) { const i = process.argv.indexOf(name); return i >= 0 ? process.argv[i + 1] : null; }
const skillDir = path.resolve(arg('--skill'));
const outPath = path.resolve(arg('--out'));
const skillFile = path.join(skillDir, 'SKILL.md');
const issues = [];
if (!fs.existsSync(skillFile)) issues.push('missing_SKILL_md');
let frontmatter = {};
if (fs.existsSync(skillFile)) {
  const text = fs.readFileSync(skillFile, 'utf8').replace(/^\uFEFF/, '');
  const m = text.match(/^---\n([\s\S]*?)\n---/);
  if (!m) issues.push('missing_frontmatter');
  else {
    for (const line of m[1].split(/\r?\n/)) {
      const idx = line.indexOf(':');
      if (idx > 0) frontmatter[line.slice(0, idx).trim()] = line.slice(idx + 1).trim();
    }
  }
}
const folderName = path.basename(skillDir);
if (frontmatter.name && frontmatter.name !== folderName) issues.push('folder_name_mismatch');
if (!frontmatter.name) issues.push('missing_name');
if (!frontmatter.description || frontmatter.description.length < 40) issues.push('description_too_weak');
const clutter = ['README.md','CHANGELOG.md','INSTALL.md','INSTALLATION_GUIDE.md','QUICK_REFERENCE.md'].filter(f => fs.existsSync(path.join(skillDir, f)));
if (clutter.length) issues.push(`clutter_files:clutter.join(',')`);
const scriptDir = path.join(skillDir, 'scripts');
const syntax = [];
if (fs.existsSync(scriptDir)) {
  for (const file of fs.readdirSync(scriptDir)) {
    if (!file.endsWith('.js')) continue;
    const res = spawnSync(process.execPath, ['--check', path.join(scriptDir, file)], { encoding: 'utf8' });
    if (res.status !== 0) syntax.push(file);
  }
}
if (syntax.length) issues.push(`script_syntax_fail:syntax.join(',')`);
const result = { ok: issues.length === 0, skill: skillDir, issues, folderName, frontmatter, checkedScripts: fs.existsSync(scriptDir) ? fs.readdirSync(scriptDir).filter(f=>f.endsWith('.js')).length : 0 };
fs.mkdirSync(path.dirname(outPath), { recursive: true });
fs.writeFileSync(outPath, JSON.stringify(result, null, 2), 'utf8');
process.stdout.write(JSON.stringify(result, null, 2) + '\n');

---
name: queue-state-diff
description: Compare two queue or state snapshots and explain what changed between them. Use when Codex needs to analyze drift between JSON/JSONL snapshots, detect newly stuck jobs, missing queue references, changed counters, or state regressions across two points in time.
---

# Queue State Diff

Use this skill to answer: what changed, what disappeared, and what got worse?

## Workflow

1. Gather two comparable snapshots: before/after JSON, report JSON, or JSONL-derived exports.
2. Normalize them before reasoning; prefer deterministic diff over freehand comparison.
3. Run `scripts/queue_state_diff.js` for raw comparison.
4. Interpret the diff in terms of operational meaning: regressions, recoveries, or noise.

## Script

```bash
node skills/queue-state-diff/scripts/queue_state_diff.js \
  --before out/report-before.json \
  --after out/report-after.json \
  --out out/queue-state-diff.md
```

## Guardrails

- Compare like with like; do not diff unrelated report formats.
- Separate numeric drift from referential drift.
- Call out missing keys explicitly instead of treating them as zero.

FILE:scripts/queue_state_diff.js
const fs = require('fs');
const path = require('path');
function arg(name) { const i = process.argv.indexOf(name); return i >= 0 ? process.argv[i + 1] : null; }
function readJson(p) { return JSON.parse(fs.readFileSync(path.resolve(p), 'utf8').replace(/^\uFEFF/, '')); }
function flatten(obj, prefix = '', out = {}) {
  if (Array.isArray(obj)) { out[prefix || '$'] = `[array:obj.length]`; return out; }
  if (!obj || typeof obj !== 'object') { out[prefix || '$'] = obj; return out; }
  for (const [k, v] of Object.entries(obj)) flatten(v, prefix ? `prefix.k` : k, out);
  return out;
}
const before = readJson(arg('--before'));
const after = readJson(arg('--after'));
const outPath = path.resolve(arg('--out'));
const a = flatten(before), b = flatten(after);
const keys = [...new Set([...Object.keys(a), ...Object.keys(b)])].sort();
const added = [], removed = [], changed = [];
for (const k of keys) {
  if (!(k in a)) added.push({ key: k, value: b[k] });
  else if (!(k in b)) removed.push({ key: k, value: a[k] });
  else if (JSON.stringify(a[k]) !== JSON.stringify(b[k])) changed.push({ key: k, before: a[k], after: b[k] });
}
const lines = ['# Queue / State Diff', '', `- Added keys: added.length`, `- Removed keys: removed.length`, `- Changed keys: changed.length`, '', '## Changed'];
changed.slice(0, 100).forEach(x => lines.push(`- x.key: JSON.stringify(x.before) -> JSON.stringify(x.after)`));
lines.push('', '## Added'); added.slice(0, 50).forEach(x => lines.push(`- x.key: JSON.stringify(x.value)`));
lines.push('', '## Removed'); removed.slice(0, 50).forEach(x => lines.push(`- x.key: JSON.stringify(x.value)`));
fs.mkdirSync(path.dirname(outPath), { recursive: true });
fs.writeFileSync(outPath, lines.join('\n'), 'utf8');
process.stdout.write(JSON.stringify({ ok: true, out: outPath, added: added.length, removed: removed.length, changed: changed.length }, null, 2) + '\n');

---
name: incident-dossier
description: Build a concise incident dossier from operational logs, audits, JSON/JSONL files, and state snapshots. Use when investigating failures, duplicate events, stuck jobs, queue anomalies, regressions, or any situation where Codex must turn raw evidence into a structured incident report with timeline, suspected root cause, blast radius, and recommended next actions.
---

# Incident Dossier

Use this skill to turn messy operational artifacts into a usable incident report.

## Workflow

1. Identify the evidence set first: logs, JSON/JSONL audit trails, state snapshots, screenshots, or reports.
2. Prefer summarizing from parsed artifacts instead of hand-reading long raw logs.
3. Run `scripts/build_incident_dossier.js` with explicit inputs when at least one JSON/JSONL source exists.
4. Verify the generated timeline against a few raw lines before trusting it.
5. State uncertainty explicitly when timestamps are missing or conflicting.

## Output contract

Always include:
- Incident summary
- Scope / blast radius
- Timeline
- Evidence list
- Hypotheses / likely root cause
- Recovery status
- Recommended next actions

## Script

Use `scripts/build_incident_dossier.js` to parse mixed JSON/JSONL evidence and emit a Markdown dossier. Give it multiple `--input` paths and one `--out` path.

Example:

```bash
node skills/incident-dossier/scripts/build_incident_dossier.js \
  --input memory/job-audit.jsonl \
  --input out/job_consistency_audit_report.json \
  --out out/incident-dossier.md
```

## Guardrails

- Do not fabricate timestamps.
- Do not collapse distinct incidents into one unless the evidence clearly links them.
- Keep the dossier evidence-first; interpretation comes after observed facts.

FILE:scripts/build_incident_dossier.js
const fs = require('fs');
const path = require('path');

function arg(name) {
  const idx = process.argv.indexOf(name);
  return idx >= 0 ? process.argv[idx + 1] : null;
}
function args(name) {
  const out = [];
  for (let i = 0; i < process.argv.length; i++) if (process.argv[i] === name && process.argv[i + 1]) out.push(process.argv[i + 1]);
  return out;
}
function readText(p) { return fs.readFileSync(p, 'utf8').replace(/^\uFEFF/, ''); }
function tryJson(text) { try { return JSON.parse(text); } catch { return null; } }
function readStructured(p) {
  const text = readText(p);
  if (p.endsWith('.jsonl')) {
    return text.split(/\r?\n/).map(s => s.trim()).filter(Boolean).map(line => tryJson(line) || { raw: line });
  }
  return tryJson(text) ?? text;
}
function collectEvents(value, source, acc) {
  if (Array.isArray(value)) return value.forEach(v => collectEvents(v, source, acc));
  if (!value || typeof value !== 'object') return;
  const candidate = { source, ts: value.ts ?? value.created_at ?? value.timestamp ?? null, kind: value.kind ?? value.status ?? value.event ?? 'record', job_id: value.job_id ?? null };
  if (candidate.ts || candidate.kind || candidate.job_id) acc.push(candidate);
  for (const v of Object.values(value)) {
    if (Array.isArray(v) || (v && typeof v === 'object')) collectEvents(v, source, acc);
  }
}
function main() {
  const inputs = args('--input');
  const out = arg('--out');
  if (!inputs.length || !out) {
    console.error('Usage: node build_incident_dossier.js --input <path> [--input <path> ...] --out <path>');
    process.exit(1);
  }
  const events = [];
  const evidence = [];
  for (const input of inputs) {
    const abs = path.resolve(input);
    const structured = readStructured(abs);
    evidence.push({ path: abs, kind: Array.isArray(structured) ? 'array/jsonl' : typeof structured });
    collectEvents(structured, abs, events);
  }
  events.sort((a, b) => (a.ts ?? 0) - (b.ts ?? 0));
  const topKinds = Object.entries(events.reduce((m, e) => (m[e.kind] = (m[e.kind] || 0) + 1, m), {})).sort((a,b)=>b[1]-a[1]).slice(0, 8);
  const lines = [];
  lines.push('# Incident Dossier', '');
  lines.push('## Summary', `- Evidence files: evidence.length`, `- Parsed event count: events.length`, `- Dominant signals: topKinds.map(([k,v]) => `${k (v)`).join(', ') || 'none'}`, '');
  lines.push('## Evidence');
  for (const item of evidence) lines.push(`- item.path — item.kind`);
  lines.push('', '## Timeline');
  if (!events.length) lines.push('- No structured events extracted.');
  else for (const e of events.slice(0, 50)) lines.push(`- e.ts ?? 'no-ts' · e.kind · e.job_id ?? 'no-job-id' · path.basename(e.source)`);
  lines.push('', '## Hypotheses', '- Inspect dominant event kinds and concentration points above.', '- Compare timeline gaps and duplicate signals before claiming root cause.', '', '## Next actions', '- Verify the highest-frequency event family against raw source lines.', '- Confirm whether the incident is ongoing or purely historical.');
  fs.mkdirSync(path.dirname(path.resolve(out)), { recursive: true });
  fs.writeFileSync(path.resolve(out), lines.join('\n'), 'utf8');
  process.stdout.write(JSON.stringify({ ok: true, out: path.resolve(out), events: events.length, evidence: evidence.length }, null, 2) + '\n');
}
main();

---
name: Multi-API Data Pipeline to Google Sheets
description: Connects multiple REST APIs, fetches and transforms data, and pushes it to a live Google Sheets dashboard that auto-updates on a schedule.
version: 1.0.2
tags: [python, api, automation, google-sheets, pipeline, dashboard, data]
author: neo1307
requires: [python3, requests, pandas, gspread, google-auth-oauthlib]
---

# Multi-API Data Pipeline to Google Sheets

## Overview
Automated data pipeline that pulls from multiple REST APIs, transforms and
merges the data, and pushes it to a Google Sheets dashboard that updates
automatically on your chosen schedule (every 15 minutes, hourly, daily).
Replaces hours of manual copy-paste work.

## What It Does
- Connects to up to 10 REST APIs simultaneously
- Handles authentication: API keys, Bearer tokens, OAuth2
- Transforms and merges data across sources
- Pushes clean, formatted data to Google Sheets in real time
- Sends alert if any API call fails
- Logs all pipeline runs with success/failure status

## Required Environment Variables
Set these in OpenClaw's Secrets manager before running:

| Variable | Description |
|----------|-------------|
| `GOOGLE_SERVICE_ACCOUNT_JSON` | Google Service Account key (full JSON string) |
| `TARGET_SHEET_ID` | Google Sheets document ID (from the sheet URL) |
| `[SERVICE]_API_KEY` | One secret per connected API, e.g. `SHOPIFY_API_KEY`, `HUBSPOT_TOKEN` |

## Setup
1. Create a Google Service Account, download the JSON key, paste it as `GOOGLE_SERVICE_ACCOUNT_JSON`
2. Share your target Google Sheet with the service account email
3. Set `TARGET_SHEET_ID` from the sheet URL
4. Add one secret per API you want to connect
5. Set update schedule: `every 15 minutes` / `hourly` / `daily at 06:00`

## Usage
> "Connect Shopify and HubSpot APIs and sync sales data to my Google Sheet every hour"
> "Pull weather data and stock prices into a live dashboard"
> "Set up a pipeline from our internal API to Google Sheets, update every 15 minutes"
> "Add Stripe revenue data to the existing pipeline"

## Output
- Live Google Sheets dashboard with latest data
- Pipeline run log: `logs/pipeline_YYYY-MM-DD.txt`
- Alert on failure with error details

## Rules
- Never store raw API credentials in output files or logs
- Always validate API response schema before writing to Sheets
- If Google Sheets write fails, buffer data locally and retry up to 3 times
- Respect API rate limits — add delays per API documentation
- Each pipeline run must write a summary row to a `_run_log` tab in the Sheet

---
name: B2B Lead Generation Scraper
description: Extracts verified B2B leads (name, email, company, LinkedIn, job title) from target sources and exports them as CRM-ready CSV files.
version: 1.0.2
tags: [python, scraping, lead-generation, b2b, linkedin, csv, crm, automation]
author: neo1307
requires: [python3, selenium, webdriver-manager, pandas, requests, chromium]
---

# B2B Lead Generation Scraper

## Overview
Automated lead extraction tool that collects verified B2B contact data from
target sources and delivers clean, CRM-ready CSV files. Delivers 500-2,000+
leads per run depending on target criteria.

## What It Does
- Extracts: Full name, job title, company name, LinkedIn URL, email (when available)
- Filters by: industry, job title keywords, company size, location
- Deduplicates records automatically
- Outputs clean CSV ready for HubSpot, Salesforce, Pipedrive import
- Validates and removes junk/incomplete rows before delivery

## Required Environment Variables
Set these in OpenClaw's Secrets manager before running:

| Variable | Description |
|----------|-------------|
| `LI_SESSION` | LinkedIn session cookie (`li_at` value from your browser) |

## Setup
1. Log into LinkedIn in your browser, copy the `li_at` cookie value
2. Set `LI_SESSION` in OpenClaw Secrets
3. Define target criteria in OpenClaw chat (industry, job title, location, company size)
4. Chromium must be available on the host for Selenium headless mode

## Usage
> "Find 500 B2B leads: SaaS CEOs in the United States"
> "Scrape marketing directors at companies with 50-200 employees in London"
> "Generate a lead list of HR managers in healthcare companies"
> "Export leads to CSV formatted for HubSpot import"

## Output
- `leads_YYYY-MM-DD_[criteria].csv` with columns:
  - first_name, last_name, full_name, job_title, company, linkedin_url, email, location
- Summary: total found, duplicates removed, validation pass rate

## Rules
- Never scrape more than 200 profiles per hour to avoid detection
- Always deduplicate by LinkedIn URL before saving
- Mark rows with missing email as `email_status: not_found` — do not fabricate
- Save raw data before cleaning in `data/raw/`
- Output CSV must be UTF-8 encoded for CRM compatibility

---
name: Price Monitor & Daily Excel Report Bot
description: Monitors product prices across e-commerce sites daily, detects price drops, and emails a formatted Excel report automatically every morning.
version: 1.0.2
tags: [python, scraping, automation, excel, email, selenium, pandas]
author: neo1307
requires: [python3, selenium, webdriver-manager, pandas, openpyxl, chromium]
---

# Price Monitor & Daily Excel Report Bot

## Overview
Automated price tracking system that monitors products across competitor websites,
detects price drops, and delivers a formatted Excel report via email every morning at 8 AM.
Saves 3+ hours of manual work per day.

## What It Does
- Scrapes prices from target URLs using Selenium (handles JavaScript-rendered pages)
- Compares against previous day's prices automatically
- Highlights price drops >5% in red, increases in green inside Excel
- Emails formatted report to specified recipients on schedule
- Logs all runs with timestamps for auditing

## Required Environment Variables
Set these in OpenClaw's Secrets manager before running:

| Variable | Description | Example |
|----------|-------------|---------|
| `SMTP_HOST` | SMTP server hostname | `smtp.gmail.com` |
| `SMTP_PORT` | SMTP port number | `587` |
| `SMTP_USER` | Sender email address | `[email protected]` |
| `SMTP_PASS` | Email password or app-specific password | `xxxx xxxx xxxx xxxx` |
| `REPORT_RECIPIENT` | Where to send the daily report | `[email protected]` |

## Setup
1. Add target product URLs to `config/urls.txt` (one URL per line)
2. Set all environment variables above in OpenClaw Secrets
3. Set run schedule in OpenClaw: `daily at 08:00`
4. Chromium must be available on the host for Selenium headless mode

## Usage
> "Start monitoring prices for these URLs and email me a report every morning"
> "Check competitor prices and send Excel summary to [email protected]"
> "Run price tracker now and show me today's drops"
> "Add this product URL to the monitoring list"

## Output
- `price_report_YYYY-MM-DD.xlsx` — color-coded Excel report
- Email with report attached sent to configured recipient
- Console summary: total products, drops found, errors

## Rules
- Never send more than 1 request per second to any domain
- Always save raw scraped data before processing (in `data/raw/`)
- If email delivery fails, save report to `data/reports/` and retry once after 10 minutes
- If a URL returns 403/blocked, skip and log — do not retry more than 3 times
- Report must include: product name, URL, yesterday's price, today's price, % change

## Example Config (urls.txt)
```
https://www.amazon.com/dp/B08N5WRWNW
https://www.amazon.com/dp/B09G9FPHY6
https://www.bestbuy.com/site/product/123456
```

FILE:scripts/run.py
"""
Price Monitor & Daily Excel Report Bot
Argus AI Agent — OpenClaw Skill v1.0.0
"""

import os
import time
import smtplib
import logging
from datetime import datetime
from email.mime.multipart import MIMEMultipart
from email.mime.base import MIMEBase
from email.mime.text import MIMEText
from email import encoders
from pathlib import Path

import pandas as pd
import requests
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from openpyxl import load_workbook
from openpyxl.styles import PatternFill

# ── Logging ──────────────────────────────────────────────────────────
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("price-monitor")

# ── Config from env (OpenClaw secrets) ───────────────────────────────
SMTP_HOST = os.getenv("SMTP_HOST", "smtp.gmail.com")
SMTP_PORT = int(os.getenv("SMTP_PORT", 587))
SMTP_USER = os.getenv("SMTP_USER", "")
SMTP_PASS = os.getenv("SMTP_PASS", "")
RECIPIENT = os.getenv("REPORT_RECIPIENT", "")
URLS_FILE = Path("config/urls.txt")
DATA_DIR  = Path("data")
RAW_DIR   = DATA_DIR / "raw"
REPORT_DIR = DATA_DIR / "reports"

RAW_DIR.mkdir(parents=True, exist_ok=True)
REPORT_DIR.mkdir(parents=True, exist_ok=True)


def load_urls():
    if not URLS_FILE.exists():
        log.warning("urls.txt not found — using demo URLs")
        return ["https://www.amazon.com/dp/B08N5WRWNW"]
    return [u.strip() for u in URLS_FILE.read_text().splitlines() if u.strip()]


def scrape_price(driver, url: str) -> dict:
    """Scrape a single product URL and return price data."""
    try:
        driver.get(url)
        time.sleep(2)

        # Generic price selector — works for most e-commerce sites
        selectors = [
            '[data-automation="price"]',
            '.price', '#price', '.product-price',
            '[class*="price"]', '[id*="price"]',
        ]
        price_text = None
        for sel in selectors:
            elements = driver.find_elements(By.CSS_SELECTOR, sel)
            if elements:
                price_text = elements[0].text.strip()
                break

        title = driver.title[:80] if driver.title else url

        return {
            "url": url,
            "title": title,
            "price_raw": price_text or "N/A",
            "price": float(''.join(filter(lambda c: c.isdigit() or c == '.', price_text or '0')) or 0),
            "scraped_at": datetime.now().isoformat(),
            "status": "ok"
        }
    except Exception as e:
        log.error(f"Failed to scrape {url}: {e}")
        return {"url": url, "title": url, "price_raw": "ERROR", "price": 0, "status": "error"}


def run_scraper(urls: list) -> pd.DataFrame:
    """Run Selenium scraper on all URLs."""
    opts = Options()
    opts.add_argument("--headless")
    opts.add_argument("--no-sandbox")
    opts.add_argument("--disable-dev-shm-usage")
    opts.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.0.0")

    driver = webdriver.Chrome(options=opts)
    results = []

    for i, url in enumerate(urls):
        log.info(f"[{i+1}/{len(urls)}] Scraping: {url}")
        results.append(scrape_price(driver, url))
        time.sleep(1.2)  # Polite delay

    driver.quit()
    return pd.DataFrame(results)


def compare_with_yesterday(today_df: pd.DataFrame) -> pd.DataFrame:
    """Load yesterday's data and compute price changes."""
    yesterday_files = sorted(REPORT_DIR.glob("raw_*.csv"))
    if not yesterday_files:
        today_df["prev_price"] = today_df["price"]
        today_df["change_pct"] = 0.0
        return today_df

    prev_df = pd.read_csv(yesterday_files[-1])[["url", "price"]].rename(columns={"price": "prev_price"})
    merged = today_df.merge(prev_df, on="url", how="left")
    merged["prev_price"] = merged["prev_price"].fillna(merged["price"])
    merged["change_pct"] = ((merged["price"] - merged["prev_price"]) / merged["prev_price"] * 100).round(2)
    return merged


def build_excel_report(df: pd.DataFrame, date_str: str) -> Path:
    """Create color-coded Excel report."""
    report_path = REPORT_DIR / f"price_report_{date_str}.xlsx"

    cols = ["title", "url", "prev_price", "price", "change_pct", "scraped_at"]
    available = [c for c in cols if c in df.columns]
    df[available].to_excel(report_path, index=False, sheet_name="Price Report")

    wb = load_workbook(report_path)
    ws = wb.active

    red_fill   = PatternFill("solid", fgColor="FFCCCC")   # price drop
    green_fill = PatternFill("solid", fgColor="CCFFCC")   # price rise

    change_col = available.index("change_pct") + 1 if "change_pct" in available else None

    for row in ws.iter_rows(min_row=2, max_row=ws.max_row):
        if change_col:
            val = row[change_col - 1].value
            if isinstance(val, (int, float)):
                fill = red_fill if val < -5 else (green_fill if val > 5 else None)
                if fill:
                    for cell in row:
                        cell.fill = fill

    wb.save(report_path)
    log.info(f"Report saved: {report_path}")
    return report_path


def send_email(report_path: Path, date_str: str, summary: str):
    """Send Excel report via email."""
    if not all([SMTP_USER, SMTP_PASS, RECIPIENT]):
        log.warning("Email not configured — skipping send")
        return

    msg = MIMEMultipart()
    msg["From"] = SMTP_USER
    msg["To"] = RECIPIENT
    msg["Subject"] = f"📊 Price Report — {date_str}"
    msg.attach(MIMEText(f"Daily price monitoring report.\n\n{summary}", "plain"))

    with open(report_path, "rb") as f:
        part = MIMEBase("application", "octet-stream")
        part.set_payload(f.read())
        encoders.encode_base64(part)
        part.add_header("Content-Disposition", f"attachment; filename={report_path.name}")
        msg.attach(part)

    try:
        with smtplib.SMTP(SMTP_HOST, SMTP_PORT) as server:
            server.starttls()
            server.login(SMTP_USER, SMTP_PASS)
            server.sendmail(SMTP_USER, RECIPIENT, msg.as_string())
        log.info(f"Report emailed to {RECIPIENT}")
    except Exception as e:
        log.error(f"Email failed: {e} — report saved locally at {report_path}")


def main():
    date_str = datetime.now().strftime("%Y-%m-%d")
    log.info(f"=== Price Monitor started: {date_str} ===")

    urls = load_urls()
    log.info(f"Monitoring {len(urls)} URLs")

    today_df = run_scraper(urls)
    today_df.to_csv(RAW_DIR / f"raw_{date_str}.csv", index=False)

    result_df = compare_with_yesterday(today_df)
    report_path = build_excel_report(result_df, date_str)

    drops = result_df[result_df.get("change_pct", pd.Series()) < -5] if "change_pct" in result_df else pd.DataFrame()
    summary = (
        f"Total monitored: {len(result_df)}\n"
        f"Price drops (>5%): {len(drops)}\n"
        f"Errors: {len(result_df[result_df['status'] == 'error'])}\n"
    )
    print(summary)
    send_email(report_path, date_str, summary)
    log.info("=== Done ===")


if __name__ == "__main__":
    main()