@clawhub-neo1307-0b87ca85ae
Build and package Flutter Android release artifacts (APK/AAB), collect outputs into a single folder, and produce a short release checklist. Use when the user...
--- name: flutter-release-pipeline description: Build and package Flutter Android release artifacts (APK/AAB), collect outputs into a single folder, and produce a short release checklist. Use when the user asks to build APK/AAB, prepare a Play Store release, bump version, gather mapping/symbols, or generate a deterministic release folder for sharing. --- # Flutter Android Release Pipeline ## Workflow 1) **Preflight** - Confirm `flutter --version` works. - Confirm project path contains `pubspec.yaml`. 2) **Build** (choose one) - AAB: `flutter build appbundle --release` - APK: `flutter build apk --release` (optionally `--split-per-abi`) 3) **Collect artifacts** Create `out/flutter_release_<timestamp>/` and copy: - `build/app/outputs/flutter-apk/*.apk` (if APK build) - `build/app/outputs/bundle/release/*.aab` (if AAB build) - `build/app/outputs/mapping/release/mapping.txt` (if present) - `pubspec.yaml` (snapshot) 4) **Report** - Print paths + sizes + SHA256 for each artifact. - Print a short checklist (versionCode/versionName sanity, signing, Play Console notes). ## Script Run (PowerShell): - `powershell -ExecutionPolicy Bypass -File scripts/flutter_release.ps1 -Project "<path>" -Mode aab|apk -SplitPerAbi:$false` ## Notes - Avoid changing app code unless explicitly requested. - If build fails, return the exact error + suggested fix.
Copy (relay) a local file from this machine into Google Drive for desktop (GoogleDriveFS) so it can be downloaded from a phone when the phone is not on the s...
--- name: drive-file-relay description: Copy (relay) a local file from this machine into Google Drive for desktop (GoogleDriveFS) so it can be downloaded from a phone when the phone is not on the same LAN. Use when the user says they are on the phone, cannot access the PC, and needs an APK/ZIP/PDF/any file moved into Drive; also use when you must verify size+SHA256 after copy. --- # Drive File Relay (Google Drive for desktop) ## Workflow 1) **Preflight** - Confirm source path exists. - Detect Drive mount: - Prefer `G:\Drive'ım\` (TR) and `G:\My Drive\` (EN). - If not found, list filesystem drives and look for a drive containing `.shortcut-targets-by-id`. 2) **Copy + verify** (no partial success) - Create destination folder if missing. - Copy file. - Compute SHA256 of source and destination and ensure they match. - Report: - src path - dst path - size bytes + MB - sha256 3) **User instruction** - Tell the user where to find it on the phone: Google Drive → My Drive/Drive'ım → `<folder>`. ## Script Run (PowerShell): - `powershell -ExecutionPolicy Bypass -File scripts/drive_relay.ps1 -Src "<path>" -DstFolder "ArgusShare" -DstName "<optional>"` ### Notes - Do **not** attempt provider messaging via curl/Telegram Bot API. - This skill does **not** create share links; it only ensures the file is present in the user’s Drive.
Browser automation with Playwright for opening pages, taking screenshots, finding or clicking DOM elements, filling forms, extracting text, and managing cook...
---
name: browser-agent
description: Browser automation with Playwright for opening pages, taking screenshots, finding or clicking DOM elements, filling forms, extracting text, and managing cookies/session state. Use when Codex needs deterministic browser control against real web pages, smoke-testing a site, grabbing DOM evidence, or automating straightforward UI flows.
---
# Browser Agent
Use Playwright-backed browser control for reproducible page actions.
## Workflow
1. Run `index.js` with a target URL.
2. Use action flags to open, screenshot, extract title/text, click selectors, fill inputs, or save/load session state.
3. Keep selectors explicit when clicking or filling.
4. Save evidence (screenshots / extracted text) to `out/`.
## Supported actions
- page open
- screenshot
- title read
- text extraction by selector
- click by selector
- fill input/textarea by selector
- save cookies/storage state
- load cookies/storage state
## Example
```bash
node skills/browser-agent/index.js --url https://example.com --screenshot out/example.png --title --extract h1
```
FILE:index.js
const fs = require('fs');
const path = require('path');
const { chromium } = require('playwright');
function arg(name) {
const i = process.argv.indexOf(name);
return i >= 0 ? process.argv[i + 1] : null;
}
function has(name) { return process.argv.includes(name); }
function args(name) {
const out = [];
for (let i = 0; i < process.argv.length; i++) if (process.argv[i] === name && process.argv[i + 1]) out.push(process.argv[i + 1]);
return out;
}
(async () => {
const url = arg('--url');
if (!url) throw new Error('Usage: node index.js --url <url> [--screenshot <path>] [--title] [--extract <selector>] [--click <selector>] [--fill <selector=value>] [--save-session <path>] [--load-session <path>]');
const headless = !has('--headed');
const browser = await chromium.launch({ headless });
const contextOpts = {};
const loadSession = arg('--load-session');
if (loadSession && fs.existsSync(path.resolve(loadSession))) contextOpts.storageState = path.resolve(loadSession);
const context = await browser.newContext(contextOpts);
const page = await context.newPage();
await page.goto(url, { waitUntil: 'domcontentloaded' });
const result = { ok: true, url, title: null, extracted: [], screenshot: null, clicked: [], filled: [], sessionSaved: null };
if (has('--title')) result.title = await page.title();
for (const selector of args('--click')) {
await page.locator(selector).first().click();
result.clicked.push(selector);
}
for (const pair of args('--fill')) {
const idx = pair.indexOf('=');
if (idx <= 0) continue;
const selector = pair.slice(0, idx);
const value = pair.slice(idx + 1);
await page.locator(selector).first().fill(value);
result.filled.push({ selector, value });
}
for (const selector of args('--extract')) {
const text = await page.locator(selector).first().textContent();
result.extracted.push({ selector, text });
}
const screenshot = arg('--screenshot');
if (screenshot) {
const out = path.resolve(screenshot);
fs.mkdirSync(path.dirname(out), { recursive: true });
await page.screenshot({ path: out, fullPage: true });
result.screenshot = out;
}
const saveSession = arg('--save-session');
if (saveSession) {
const out = path.resolve(saveSession);
fs.mkdirSync(path.dirname(out), { recursive: true });
await context.storageState({ path: out });
result.sessionSaved = out;
}
await context.close();
await browser.close();
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
})().catch(err => {
process.stderr.write(String(err && err.stack || err) + '\n');
process.exit(1);
});
Inspect environment variables, critical directories, and write permissions, then produce a health report. Use when validating deployment readiness, local run...
---
name: env-health-check
description: Inspect environment variables, critical directories, and write permissions, then produce a health report. Use when validating deployment readiness, local runtime setup, workspace health, or missing env/config prerequisites.
---
# Env Health Check
Inspect runtime prerequisites and emit a compact health report.
## Workflow
1. Pass required env var names with `--env` flags.
2. Pass critical directories with `--dir` flags.
3. Run `index.js` and review OK/WARN/FAIL output.
4. Treat missing write access as a blocker unless the path is intentionally read-only.
FILE:index.js
const fs=require('fs'); const path=require('path');
function args(name){const out=[]; for(let i=0;i<process.argv.length;i++) if(process.argv[i]===name && process.argv[i+1]) out.push(process.argv[i+1]); return out;}
const envs=args('--env'); const dirs=args('--dir'); const out=process.argv.includes('--out')?process.argv[process.argv.indexOf('--out')+1]:path.join(process.cwd(),'out','env_health_report.md');
const findings=[];
for(const name of envs){findings.push({type:'env',name,status:process.env[name]?'OK':'WARN'});}
for(const d of dirs){const abs=path.resolve(d); const exists=fs.existsSync(abs); let writable=false; if(exists){ try{ const test=path.join(abs,'.env-health-check.tmp'); fs.writeFileSync(test,'ok','utf8'); fs.unlinkSync(test); writable=true; }catch{} }
findings.push({type:'dir',path:abs,status:!exists?'FAIL':(writable?'OK':'WARN')}); }
const summary=findings.reduce((m,f)=>(m[f.status]=(m[f.status]||0)+1,m),{});
const lines=['# Environment Health Report','',`- OK: summary.OK||0`,`- WARN: summary.WARN||0`,`- FAIL: summary.FAIL||0`,'','## Findings'];
for(const f of findings){ if(f.type==='env') lines.push(`- [f.status] env f.name`); else lines.push(`- [f.status] dir f.path`); }
fs.mkdirSync(path.dirname(out),{recursive:true}); fs.writeFileSync(path.resolve(out),lines.join('\n'),'utf8');
console.log(JSON.stringify({ok:true,out:path.resolve(out),summary,findings},null,2));
FILE:README.md
# env-health-check
Checks required environment variables, critical directories, and write permissions.
## Usage
```bash
node index.js --env OPENAI_API_KEY --env ANTHROPIC_API_KEY --dir ./memory --dir ./out --out ./out/env_health_report.md
```
FILE:skill.yaml
name: env-health-check
version: 1.0.0
description: Checks environment variables, critical directories, and write permissions; produces a health report.
author: neo1307
tags:
- environment
- health
- diagnostics
entrypoint: index.js
Read a log file, extract ERROR/WARN/CRITICAL lines, group similar messages, and produce a concise summary report. Use when analyzing application logs, agent...
---
name: log-analyzer
description: Read a log file, extract ERROR/WARN/CRITICAL lines, group similar messages, and produce a concise summary report. Use when analyzing application logs, agent logs, service logs, or audit outputs for repeated failures, warning clusters, or critical events.
---
# Log Analyzer
Read a target log file, isolate important severity lines, group similar messages, and emit a Markdown summary.
## Workflow
1. Confirm the input log path.
2. Run `index.js` with `--input <logfile>` and optional `--out <report.md>`.
3. Review the grouped output for dominant error families.
4. Use the report as a triage artifact, not as the only source of truth.
## Output
Always include:
- total scanned lines
- WARN / ERROR / CRITICAL counts
- grouped issue buckets
- sample lines
- suggested first checks
FILE:index.js
const fs = require('fs');
const path = require('path');
function arg(name){const i=process.argv.indexOf(name);return i>=0?process.argv[i+1]:null;}
const input=arg('--input');
const out=arg('--out') || path.join(process.cwd(),'out','log_analysis_report.md');
if(!input){console.error('Usage: node index.js --input <logfile> [--out <report.md>]');process.exit(1);}
const lines=fs.readFileSync(path.resolve(input),'utf8').replace(/^\uFEFF/,'').split(/\r?\n/);
const sev=['CRITICAL','ERROR','WARN'];
const matches=[];
for(const line of lines){const s=sev.find(x=>line.includes(x)); if(s) matches.push({severity:s,line});}
const normalize=s=>s.replace(/\d+/g,'<num>').replace(/[a-f0-9]{8,}/gi,'<id>').replace(/\s+/g,' ').trim();
const groups=new Map();
for(const m of matches){const key=`m.severity|normalize(m.line)`; if(!groups.has(key)) groups.set(key,{severity:m.severity,count:0,sample:m.line}); groups.get(key).count++;}
const ordered=[...groups.values()].sort((a,b)=>b.count-a.count);
const counts=matches.reduce((m,x)=>(m[x.severity]=(m[x.severity]||0)+1,m),{});
const report=[];
report.push('# Log Analysis Report','',`- Total scanned lines: lines.length`,`- WARN: counts.WARN||0`,`- ERROR: counts.ERROR||0`,`- CRITICAL: counts.CRITICAL||0`,'','## Grouped findings');
if(!ordered.length) report.push('- No WARN/ERROR/CRITICAL lines found.');
else ordered.slice(0,50).forEach((g,i)=>report.push(`- i+1. [g.severity] xg.count — g.sample`));
report.push('','## Suggested first checks','- Start from CRITICAL groups, then highest-frequency ERROR groups.','- Verify whether repeated warnings are expected noise or precursors to failure.');
fs.mkdirSync(path.dirname(out),{recursive:true}); fs.writeFileSync(path.resolve(out),report.join('\n'),'utf8');
console.log(JSON.stringify({ok:true,out:path.resolve(out),groups:ordered.length,counts},null,2));
FILE:README.md
# log-analyzer
Reads a log file, extracts WARN/ERROR/CRITICAL lines, groups similar messages, and writes a Markdown report.
## Usage
```bash
node index.js --input ./app.log --out ./out/log_analysis_report.md
```
FILE:skill.yaml
name: log-analyzer
version: 1.0.0
description: Reads a log file, groups WARN/ERROR/CRITICAL lines, and produces a summary report.
author: neo1307
tags:
- logs
- analysis
- diagnostics
entrypoint: index.js
Validate a local skill folder before publishing or sharing it. Use when Codex is about to release a skill, publish to ClawHub, audit SKILL.md quality, check...
---
name: skill-release-guard
description: Validate a local skill folder before publishing or sharing it. Use when Codex is about to release a skill, publish to ClawHub, audit SKILL.md quality, check naming/frontmatter/resource hygiene, or generate a release checklist for a skill package.
---
# Skill Release Guard
Use this skill right before packaging or publishing a skill.
## Workflow
1. Inspect the target skill directory.
2. Confirm `SKILL.md` exists and frontmatter has only `name` and `description`.
3. Check naming convention, folder layout, and presence of obvious clutter files.
4. Run `scripts/check_skill_release.js` to generate a compact release checklist.
5. Fix guard failures before publishing.
## Script
```bash
node skills/skill-release-guard/scripts/check_skill_release.js \
--skill skills/my-skill \
--out out/my-skill-release-check.json
```
## Guardrails
- Fail if the folder name and frontmatter name diverge.
- Flag clutter docs like README/CHANGELOG/INSTALL guides.
- Flag missing description or overly vague description.
- Treat script syntax failures as release blockers.
FILE:scripts/check_skill_release.js
const fs = require('fs');
const path = require('path');
const { spawnSync } = require('child_process');
function arg(name) { const i = process.argv.indexOf(name); return i >= 0 ? process.argv[i + 1] : null; }
const skillDir = path.resolve(arg('--skill'));
const outPath = path.resolve(arg('--out'));
const skillFile = path.join(skillDir, 'SKILL.md');
const issues = [];
if (!fs.existsSync(skillFile)) issues.push('missing_SKILL_md');
let frontmatter = {};
if (fs.existsSync(skillFile)) {
const text = fs.readFileSync(skillFile, 'utf8').replace(/^\uFEFF/, '');
const m = text.match(/^---\n([\s\S]*?)\n---/);
if (!m) issues.push('missing_frontmatter');
else {
for (const line of m[1].split(/\r?\n/)) {
const idx = line.indexOf(':');
if (idx > 0) frontmatter[line.slice(0, idx).trim()] = line.slice(idx + 1).trim();
}
}
}
const folderName = path.basename(skillDir);
if (frontmatter.name && frontmatter.name !== folderName) issues.push('folder_name_mismatch');
if (!frontmatter.name) issues.push('missing_name');
if (!frontmatter.description || frontmatter.description.length < 40) issues.push('description_too_weak');
const clutter = ['README.md','CHANGELOG.md','INSTALL.md','INSTALLATION_GUIDE.md','QUICK_REFERENCE.md'].filter(f => fs.existsSync(path.join(skillDir, f)));
if (clutter.length) issues.push(`clutter_files:clutter.join(',')`);
const scriptDir = path.join(skillDir, 'scripts');
const syntax = [];
if (fs.existsSync(scriptDir)) {
for (const file of fs.readdirSync(scriptDir)) {
if (!file.endsWith('.js')) continue;
const res = spawnSync(process.execPath, ['--check', path.join(scriptDir, file)], { encoding: 'utf8' });
if (res.status !== 0) syntax.push(file);
}
}
if (syntax.length) issues.push(`script_syntax_fail:syntax.join(',')`);
const result = { ok: issues.length === 0, skill: skillDir, issues, folderName, frontmatter, checkedScripts: fs.existsSync(scriptDir) ? fs.readdirSync(scriptDir).filter(f=>f.endsWith('.js')).length : 0 };
fs.mkdirSync(path.dirname(outPath), { recursive: true });
fs.writeFileSync(outPath, JSON.stringify(result, null, 2), 'utf8');
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
Compare two queue or state snapshots and explain what changed between them. Use when Codex needs to analyze drift between JSON/JSONL snapshots, detect newly...
---
name: queue-state-diff
description: Compare two queue or state snapshots and explain what changed between them. Use when Codex needs to analyze drift between JSON/JSONL snapshots, detect newly stuck jobs, missing queue references, changed counters, or state regressions across two points in time.
---
# Queue State Diff
Use this skill to answer: what changed, what disappeared, and what got worse?
## Workflow
1. Gather two comparable snapshots: before/after JSON, report JSON, or JSONL-derived exports.
2. Normalize them before reasoning; prefer deterministic diff over freehand comparison.
3. Run `scripts/queue_state_diff.js` for raw comparison.
4. Interpret the diff in terms of operational meaning: regressions, recoveries, or noise.
## Script
```bash
node skills/queue-state-diff/scripts/queue_state_diff.js \
--before out/report-before.json \
--after out/report-after.json \
--out out/queue-state-diff.md
```
## Guardrails
- Compare like with like; do not diff unrelated report formats.
- Separate numeric drift from referential drift.
- Call out missing keys explicitly instead of treating them as zero.
FILE:scripts/queue_state_diff.js
const fs = require('fs');
const path = require('path');
function arg(name) { const i = process.argv.indexOf(name); return i >= 0 ? process.argv[i + 1] : null; }
function readJson(p) { return JSON.parse(fs.readFileSync(path.resolve(p), 'utf8').replace(/^\uFEFF/, '')); }
function flatten(obj, prefix = '', out = {}) {
if (Array.isArray(obj)) { out[prefix || '$'] = `[array:obj.length]`; return out; }
if (!obj || typeof obj !== 'object') { out[prefix || '$'] = obj; return out; }
for (const [k, v] of Object.entries(obj)) flatten(v, prefix ? `prefix.k` : k, out);
return out;
}
const before = readJson(arg('--before'));
const after = readJson(arg('--after'));
const outPath = path.resolve(arg('--out'));
const a = flatten(before), b = flatten(after);
const keys = [...new Set([...Object.keys(a), ...Object.keys(b)])].sort();
const added = [], removed = [], changed = [];
for (const k of keys) {
if (!(k in a)) added.push({ key: k, value: b[k] });
else if (!(k in b)) removed.push({ key: k, value: a[k] });
else if (JSON.stringify(a[k]) !== JSON.stringify(b[k])) changed.push({ key: k, before: a[k], after: b[k] });
}
const lines = ['# Queue / State Diff', '', `- Added keys: added.length`, `- Removed keys: removed.length`, `- Changed keys: changed.length`, '', '## Changed'];
changed.slice(0, 100).forEach(x => lines.push(`- x.key: JSON.stringify(x.before) -> JSON.stringify(x.after)`));
lines.push('', '## Added'); added.slice(0, 50).forEach(x => lines.push(`- x.key: JSON.stringify(x.value)`));
lines.push('', '## Removed'); removed.slice(0, 50).forEach(x => lines.push(`- x.key: JSON.stringify(x.value)`));
fs.mkdirSync(path.dirname(outPath), { recursive: true });
fs.writeFileSync(outPath, lines.join('\n'), 'utf8');
process.stdout.write(JSON.stringify({ ok: true, out: outPath, added: added.length, removed: removed.length, changed: changed.length }, null, 2) + '\n');
Build a concise incident dossier from operational logs, audits, JSON/JSONL files, and state snapshots. Use when investigating failures, duplicate events, stu...
---
name: incident-dossier
description: Build a concise incident dossier from operational logs, audits, JSON/JSONL files, and state snapshots. Use when investigating failures, duplicate events, stuck jobs, queue anomalies, regressions, or any situation where Codex must turn raw evidence into a structured incident report with timeline, suspected root cause, blast radius, and recommended next actions.
---
# Incident Dossier
Use this skill to turn messy operational artifacts into a usable incident report.
## Workflow
1. Identify the evidence set first: logs, JSON/JSONL audit trails, state snapshots, screenshots, or reports.
2. Prefer summarizing from parsed artifacts instead of hand-reading long raw logs.
3. Run `scripts/build_incident_dossier.js` with explicit inputs when at least one JSON/JSONL source exists.
4. Verify the generated timeline against a few raw lines before trusting it.
5. State uncertainty explicitly when timestamps are missing or conflicting.
## Output contract
Always include:
- Incident summary
- Scope / blast radius
- Timeline
- Evidence list
- Hypotheses / likely root cause
- Recovery status
- Recommended next actions
## Script
Use `scripts/build_incident_dossier.js` to parse mixed JSON/JSONL evidence and emit a Markdown dossier. Give it multiple `--input` paths and one `--out` path.
Example:
```bash
node skills/incident-dossier/scripts/build_incident_dossier.js \
--input memory/job-audit.jsonl \
--input out/job_consistency_audit_report.json \
--out out/incident-dossier.md
```
## Guardrails
- Do not fabricate timestamps.
- Do not collapse distinct incidents into one unless the evidence clearly links them.
- Keep the dossier evidence-first; interpretation comes after observed facts.
FILE:scripts/build_incident_dossier.js
const fs = require('fs');
const path = require('path');
function arg(name) {
const idx = process.argv.indexOf(name);
return idx >= 0 ? process.argv[idx + 1] : null;
}
function args(name) {
const out = [];
for (let i = 0; i < process.argv.length; i++) if (process.argv[i] === name && process.argv[i + 1]) out.push(process.argv[i + 1]);
return out;
}
function readText(p) { return fs.readFileSync(p, 'utf8').replace(/^\uFEFF/, ''); }
function tryJson(text) { try { return JSON.parse(text); } catch { return null; } }
function readStructured(p) {
const text = readText(p);
if (p.endsWith('.jsonl')) {
return text.split(/\r?\n/).map(s => s.trim()).filter(Boolean).map(line => tryJson(line) || { raw: line });
}
return tryJson(text) ?? text;
}
function collectEvents(value, source, acc) {
if (Array.isArray(value)) return value.forEach(v => collectEvents(v, source, acc));
if (!value || typeof value !== 'object') return;
const candidate = { source, ts: value.ts ?? value.created_at ?? value.timestamp ?? null, kind: value.kind ?? value.status ?? value.event ?? 'record', job_id: value.job_id ?? null };
if (candidate.ts || candidate.kind || candidate.job_id) acc.push(candidate);
for (const v of Object.values(value)) {
if (Array.isArray(v) || (v && typeof v === 'object')) collectEvents(v, source, acc);
}
}
function main() {
const inputs = args('--input');
const out = arg('--out');
if (!inputs.length || !out) {
console.error('Usage: node build_incident_dossier.js --input <path> [--input <path> ...] --out <path>');
process.exit(1);
}
const events = [];
const evidence = [];
for (const input of inputs) {
const abs = path.resolve(input);
const structured = readStructured(abs);
evidence.push({ path: abs, kind: Array.isArray(structured) ? 'array/jsonl' : typeof structured });
collectEvents(structured, abs, events);
}
events.sort((a, b) => (a.ts ?? 0) - (b.ts ?? 0));
const topKinds = Object.entries(events.reduce((m, e) => (m[e.kind] = (m[e.kind] || 0) + 1, m), {})).sort((a,b)=>b[1]-a[1]).slice(0, 8);
const lines = [];
lines.push('# Incident Dossier', '');
lines.push('## Summary', `- Evidence files: evidence.length`, `- Parsed event count: events.length`, `- Dominant signals: topKinds.map(([k,v]) => `${k (v)`).join(', ') || 'none'}`, '');
lines.push('## Evidence');
for (const item of evidence) lines.push(`- item.path — item.kind`);
lines.push('', '## Timeline');
if (!events.length) lines.push('- No structured events extracted.');
else for (const e of events.slice(0, 50)) lines.push(`- e.ts ?? 'no-ts' · e.kind · e.job_id ?? 'no-job-id' · path.basename(e.source)`);
lines.push('', '## Hypotheses', '- Inspect dominant event kinds and concentration points above.', '- Compare timeline gaps and duplicate signals before claiming root cause.', '', '## Next actions', '- Verify the highest-frequency event family against raw source lines.', '- Confirm whether the incident is ongoing or purely historical.');
fs.mkdirSync(path.dirname(path.resolve(out)), { recursive: true });
fs.writeFileSync(path.resolve(out), lines.join('\n'), 'utf8');
process.stdout.write(JSON.stringify({ ok: true, out: path.resolve(out), events: events.length, evidence: evidence.length }, null, 2) + '\n');
}
main();
Connects multiple REST APIs, fetches and transforms data, and pushes it to a live Google Sheets dashboard that auto-updates on a schedule.
--- name: Multi-API Data Pipeline to Google Sheets description: Connects multiple REST APIs, fetches and transforms data, and pushes it to a live Google Sheets dashboard that auto-updates on a schedule. version: 1.0.2 tags: [python, api, automation, google-sheets, pipeline, dashboard, data] author: neo1307 requires: [python3, requests, pandas, gspread, google-auth-oauthlib] --- # Multi-API Data Pipeline to Google Sheets ## Overview Automated data pipeline that pulls from multiple REST APIs, transforms and merges the data, and pushes it to a Google Sheets dashboard that updates automatically on your chosen schedule (every 15 minutes, hourly, daily). Replaces hours of manual copy-paste work. ## What It Does - Connects to up to 10 REST APIs simultaneously - Handles authentication: API keys, Bearer tokens, OAuth2 - Transforms and merges data across sources - Pushes clean, formatted data to Google Sheets in real time - Sends alert if any API call fails - Logs all pipeline runs with success/failure status ## Required Environment Variables Set these in OpenClaw's Secrets manager before running: | Variable | Description | |----------|-------------| | `GOOGLE_SERVICE_ACCOUNT_JSON` | Google Service Account key (full JSON string) | | `TARGET_SHEET_ID` | Google Sheets document ID (from the sheet URL) | | `[SERVICE]_API_KEY` | One secret per connected API, e.g. `SHOPIFY_API_KEY`, `HUBSPOT_TOKEN` | ## Setup 1. Create a Google Service Account, download the JSON key, paste it as `GOOGLE_SERVICE_ACCOUNT_JSON` 2. Share your target Google Sheet with the service account email 3. Set `TARGET_SHEET_ID` from the sheet URL 4. Add one secret per API you want to connect 5. Set update schedule: `every 15 minutes` / `hourly` / `daily at 06:00` ## Usage > "Connect Shopify and HubSpot APIs and sync sales data to my Google Sheet every hour" > "Pull weather data and stock prices into a live dashboard" > "Set up a pipeline from our internal API to Google Sheets, update every 15 minutes" > "Add Stripe revenue data to the existing pipeline" ## Output - Live Google Sheets dashboard with latest data - Pipeline run log: `logs/pipeline_YYYY-MM-DD.txt` - Alert on failure with error details ## Rules - Never store raw API credentials in output files or logs - Always validate API response schema before writing to Sheets - If Google Sheets write fails, buffer data locally and retry up to 3 times - Respect API rate limits — add delays per API documentation - Each pipeline run must write a summary row to a `_run_log` tab in the Sheet
Extracts verified B2B leads (name, email, company, LinkedIn, job title) from target sources and exports them as CRM-ready CSV files.
--- name: B2B Lead Generation Scraper description: Extracts verified B2B leads (name, email, company, LinkedIn, job title) from target sources and exports them as CRM-ready CSV files. version: 1.0.2 tags: [python, scraping, lead-generation, b2b, linkedin, csv, crm, automation] author: neo1307 requires: [python3, selenium, webdriver-manager, pandas, requests, chromium] --- # B2B Lead Generation Scraper ## Overview Automated lead extraction tool that collects verified B2B contact data from target sources and delivers clean, CRM-ready CSV files. Delivers 500-2,000+ leads per run depending on target criteria. ## What It Does - Extracts: Full name, job title, company name, LinkedIn URL, email (when available) - Filters by: industry, job title keywords, company size, location - Deduplicates records automatically - Outputs clean CSV ready for HubSpot, Salesforce, Pipedrive import - Validates and removes junk/incomplete rows before delivery ## Required Environment Variables Set these in OpenClaw's Secrets manager before running: | Variable | Description | |----------|-------------| | `LI_SESSION` | LinkedIn session cookie (`li_at` value from your browser) | ## Setup 1. Log into LinkedIn in your browser, copy the `li_at` cookie value 2. Set `LI_SESSION` in OpenClaw Secrets 3. Define target criteria in OpenClaw chat (industry, job title, location, company size) 4. Chromium must be available on the host for Selenium headless mode ## Usage > "Find 500 B2B leads: SaaS CEOs in the United States" > "Scrape marketing directors at companies with 50-200 employees in London" > "Generate a lead list of HR managers in healthcare companies" > "Export leads to CSV formatted for HubSpot import" ## Output - `leads_YYYY-MM-DD_[criteria].csv` with columns: - first_name, last_name, full_name, job_title, company, linkedin_url, email, location - Summary: total found, duplicates removed, validation pass rate ## Rules - Never scrape more than 200 profiles per hour to avoid detection - Always deduplicate by LinkedIn URL before saving - Mark rows with missing email as `email_status: not_found` — do not fabricate - Save raw data before cleaning in `data/raw/` - Output CSV must be UTF-8 encoded for CRM compatibility
Monitors product prices across e-commerce sites daily, detects price drops, and emails a formatted Excel report automatically every morning.
--- name: Price Monitor & Daily Excel Report Bot description: Monitors product prices across e-commerce sites daily, detects price drops, and emails a formatted Excel report automatically every morning. version: 1.0.2 tags: [python, scraping, automation, excel, email, selenium, pandas] author: neo1307 requires: [python3, selenium, webdriver-manager, pandas, openpyxl, chromium] --- # Price Monitor & Daily Excel Report Bot ## Overview Automated price tracking system that monitors products across competitor websites, detects price drops, and delivers a formatted Excel report via email every morning at 8 AM. Saves 3+ hours of manual work per day. ## What It Does - Scrapes prices from target URLs using Selenium (handles JavaScript-rendered pages) - Compares against previous day's prices automatically - Highlights price drops >5% in red, increases in green inside Excel - Emails formatted report to specified recipients on schedule - Logs all runs with timestamps for auditing ## Required Environment Variables Set these in OpenClaw's Secrets manager before running: | Variable | Description | Example | |----------|-------------|---------| | `SMTP_HOST` | SMTP server hostname | `smtp.gmail.com` | | `SMTP_PORT` | SMTP port number | `587` | | `SMTP_USER` | Sender email address | `[email protected]` | | `SMTP_PASS` | Email password or app-specific password | `xxxx xxxx xxxx xxxx` | | `REPORT_RECIPIENT` | Where to send the daily report | `[email protected]` | ## Setup 1. Add target product URLs to `config/urls.txt` (one URL per line) 2. Set all environment variables above in OpenClaw Secrets 3. Set run schedule in OpenClaw: `daily at 08:00` 4. Chromium must be available on the host for Selenium headless mode ## Usage > "Start monitoring prices for these URLs and email me a report every morning" > "Check competitor prices and send Excel summary to [email protected]" > "Run price tracker now and show me today's drops" > "Add this product URL to the monitoring list" ## Output - `price_report_YYYY-MM-DD.xlsx` — color-coded Excel report - Email with report attached sent to configured recipient - Console summary: total products, drops found, errors ## Rules - Never send more than 1 request per second to any domain - Always save raw scraped data before processing (in `data/raw/`) - If email delivery fails, save report to `data/reports/` and retry once after 10 minutes - If a URL returns 403/blocked, skip and log — do not retry more than 3 times - Report must include: product name, URL, yesterday's price, today's price, % change ## Example Config (urls.txt) ``` https://www.amazon.com/dp/B08N5WRWNW https://www.amazon.com/dp/B09G9FPHY6 https://www.bestbuy.com/site/product/123456 ``` FILE:scripts/run.py """ Price Monitor & Daily Excel Report Bot Argus AI Agent — OpenClaw Skill v1.0.0 """ import os import time import smtplib import logging from datetime import datetime from email.mime.multipart import MIMEMultipart from email.mime.base import MIMEBase from email.mime.text import MIMEText from email import encoders from pathlib import Path import pandas as pd import requests from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.by import By from openpyxl import load_workbook from openpyxl.styles import PatternFill # ── Logging ────────────────────────────────────────────────────────── logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s") log = logging.getLogger("price-monitor") # ── Config from env (OpenClaw secrets) ─────────────────────────────── SMTP_HOST = os.getenv("SMTP_HOST", "smtp.gmail.com") SMTP_PORT = int(os.getenv("SMTP_PORT", 587)) SMTP_USER = os.getenv("SMTP_USER", "") SMTP_PASS = os.getenv("SMTP_PASS", "") RECIPIENT = os.getenv("REPORT_RECIPIENT", "") URLS_FILE = Path("config/urls.txt") DATA_DIR = Path("data") RAW_DIR = DATA_DIR / "raw" REPORT_DIR = DATA_DIR / "reports" RAW_DIR.mkdir(parents=True, exist_ok=True) REPORT_DIR.mkdir(parents=True, exist_ok=True) def load_urls(): if not URLS_FILE.exists(): log.warning("urls.txt not found — using demo URLs") return ["https://www.amazon.com/dp/B08N5WRWNW"] return [u.strip() for u in URLS_FILE.read_text().splitlines() if u.strip()] def scrape_price(driver, url: str) -> dict: """Scrape a single product URL and return price data.""" try: driver.get(url) time.sleep(2) # Generic price selector — works for most e-commerce sites selectors = [ '[data-automation="price"]', '.price', '#price', '.product-price', '[class*="price"]', '[id*="price"]', ] price_text = None for sel in selectors: elements = driver.find_elements(By.CSS_SELECTOR, sel) if elements: price_text = elements[0].text.strip() break title = driver.title[:80] if driver.title else url return { "url": url, "title": title, "price_raw": price_text or "N/A", "price": float(''.join(filter(lambda c: c.isdigit() or c == '.', price_text or '0')) or 0), "scraped_at": datetime.now().isoformat(), "status": "ok" } except Exception as e: log.error(f"Failed to scrape {url}: {e}") return {"url": url, "title": url, "price_raw": "ERROR", "price": 0, "status": "error"} def run_scraper(urls: list) -> pd.DataFrame: """Run Selenium scraper on all URLs.""" opts = Options() opts.add_argument("--headless") opts.add_argument("--no-sandbox") opts.add_argument("--disable-dev-shm-usage") opts.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.0.0") driver = webdriver.Chrome(options=opts) results = [] for i, url in enumerate(urls): log.info(f"[{i+1}/{len(urls)}] Scraping: {url}") results.append(scrape_price(driver, url)) time.sleep(1.2) # Polite delay driver.quit() return pd.DataFrame(results) def compare_with_yesterday(today_df: pd.DataFrame) -> pd.DataFrame: """Load yesterday's data and compute price changes.""" yesterday_files = sorted(REPORT_DIR.glob("raw_*.csv")) if not yesterday_files: today_df["prev_price"] = today_df["price"] today_df["change_pct"] = 0.0 return today_df prev_df = pd.read_csv(yesterday_files[-1])[["url", "price"]].rename(columns={"price": "prev_price"}) merged = today_df.merge(prev_df, on="url", how="left") merged["prev_price"] = merged["prev_price"].fillna(merged["price"]) merged["change_pct"] = ((merged["price"] - merged["prev_price"]) / merged["prev_price"] * 100).round(2) return merged def build_excel_report(df: pd.DataFrame, date_str: str) -> Path: """Create color-coded Excel report.""" report_path = REPORT_DIR / f"price_report_{date_str}.xlsx" cols = ["title", "url", "prev_price", "price", "change_pct", "scraped_at"] available = [c for c in cols if c in df.columns] df[available].to_excel(report_path, index=False, sheet_name="Price Report") wb = load_workbook(report_path) ws = wb.active red_fill = PatternFill("solid", fgColor="FFCCCC") # price drop green_fill = PatternFill("solid", fgColor="CCFFCC") # price rise change_col = available.index("change_pct") + 1 if "change_pct" in available else None for row in ws.iter_rows(min_row=2, max_row=ws.max_row): if change_col: val = row[change_col - 1].value if isinstance(val, (int, float)): fill = red_fill if val < -5 else (green_fill if val > 5 else None) if fill: for cell in row: cell.fill = fill wb.save(report_path) log.info(f"Report saved: {report_path}") return report_path def send_email(report_path: Path, date_str: str, summary: str): """Send Excel report via email.""" if not all([SMTP_USER, SMTP_PASS, RECIPIENT]): log.warning("Email not configured — skipping send") return msg = MIMEMultipart() msg["From"] = SMTP_USER msg["To"] = RECIPIENT msg["Subject"] = f"📊 Price Report — {date_str}" msg.attach(MIMEText(f"Daily price monitoring report.\n\n{summary}", "plain")) with open(report_path, "rb") as f: part = MIMEBase("application", "octet-stream") part.set_payload(f.read()) encoders.encode_base64(part) part.add_header("Content-Disposition", f"attachment; filename={report_path.name}") msg.attach(part) try: with smtplib.SMTP(SMTP_HOST, SMTP_PORT) as server: server.starttls() server.login(SMTP_USER, SMTP_PASS) server.sendmail(SMTP_USER, RECIPIENT, msg.as_string()) log.info(f"Report emailed to {RECIPIENT}") except Exception as e: log.error(f"Email failed: {e} — report saved locally at {report_path}") def main(): date_str = datetime.now().strftime("%Y-%m-%d") log.info(f"=== Price Monitor started: {date_str} ===") urls = load_urls() log.info(f"Monitoring {len(urls)} URLs") today_df = run_scraper(urls) today_df.to_csv(RAW_DIR / f"raw_{date_str}.csv", index=False) result_df = compare_with_yesterday(today_df) report_path = build_excel_report(result_df, date_str) drops = result_df[result_df.get("change_pct", pd.Series()) < -5] if "change_pct" in result_df else pd.DataFrame() summary = ( f"Total monitored: {len(result_df)}\n" f"Price drops (>5%): {len(drops)}\n" f"Errors: {len(result_df[result_df['status'] == 'error'])}\n" ) print(summary) send_email(report_path, date_str, summary) log.info("=== Done ===") if __name__ == "__main__": main()