--- name: renatus-icm description: Manage Renatus event marketing campaigns as an ICM. Use when running event registration campaigns, sending commercial email blasts, setting up event landing pages, exporting leads from Supabase, syncing unsubscribes to Renatus, or performing browser-based guest registration via CDP. Requires: Renatus credentials, Supabase project, SMTP sender account. Trigger phrases: "run a Renatus event campaign", "send Renatus email campaign", "register a guest for Renatus", "download Renatus leads", "set up event landing page". --- ## ⚠️ Credentials & Security **Required environment variables** (declare these before running any script): | Variable | Purpose | Minimum Scope | |---|---|---| | `RENATUS_USERNAME` | Renatus back office login | ICM-level account | | `RENATUS_PASSWORD` | Renatus back office login | ICM-level account | | `SUPABASE_URL` | Supabase project URL | Read leads | | `LEAD_ADMIN_TOKEN` | Supabase admin export | `anon` key or limited service role | | `SENDER_EMAIL` | From address for campaigns | Send-only SMTP account | | `SENDER_PASSWORD` | SMTP auth for email | Send-only | | `SUPABASE_SERVICE_ROLE_KEY` | Supabase admin actions (destructive) | Service role — do not use for read-only ops | **Security recommendations:** - Use a dedicated Supabase project for testing — do not supply your production `service_role_key` - Create a least-privilege Supabase anon key scoped to `funnel_leads` table reads only - Use a separate Chrome/Brave **profile** for CDP access (not your main browser session) - `renatus_delete_lead.py` performs deletions — always run with `--dry-run` first - Rotate credentials after use; revoke tokens that were shared or exposed - Do not commit real credentials to `config.json` — use the `.example` file and environment variables **CDP access:** Scripts connect to `http://127.0.0.1:9222` to inspect your browser's localStorage/cookies for Renatus auth tokens. This requires Chrome/Brave launched with `--remote-debugging-port=9222`. The skill does not extract your master Renatus password from CDP — it reads existing session tokens only. --- name: renatus-icm description: Run a Renatus event marketing campaign as an ICM (Independent Campaign Manager). Use when managing Renatus event registrations, sending commercial email campaigns, setting up event landing pages, downloading/exporting leads, syncing unsubscribes to Renatus, or performing browser-based guest registration via CDP. Handles: Supabase Edge Function registration (submit-renatus-registration), gws/gog email sending (commercial-core-day1), bounce detection and SMS recovery, CDP browser registration/delete scripts, weekly unsubscribe cron, and lead export via lead-admin-export. Trigger phrases: "run a Renatus event campaign", "send Renatus email campaign", "register a guest for Renatus", "download Renatus leads", "set up event landing page", "handle Renatus unsubscribes". --- # Renatus ICM Skill Complete toolkit for running a Renatus event marketing campaign: event page setup, email campaigns, lead management, and unsubscribe sync. ## Workflows Quick Reference See [references/workflows.md](references/workflows.md) for detailed step-by-step: - **New Event Setup** → [event-page-setup.md](references/event-page-setup.md) - **Email Campaign** → [email-campaign.md](references/email-campaign.md) - **CDP Registration** → scripts `renatus_register_guest.py`, `renatus_delete_lead.py` - **Lead Export** → Supabase lead-admin-export edge function - **Unsubscribe Sync** → `weekly_unsubscribe_sync.sh` ## Architecture Overview ## Generate a New Event Landing Page ```bash python3 scripts/generate_event_page.py --event-url "https://backoffice.myrenatus.com/Events/EventDetails?eventId=..." --output site/my-event/index.html ``` Requirements: Chrome/Brave CDP + active Renatus session. The script extracts event name, date(s), location, speakers, and session details from the backoffice page — then renders a complete ready-to-deploy HTML file. Preview before saving with `--dry-run`. ``` Event Landing Page (HTML) ↓ form submit Supabase Edge Function: submit-renatus-registration ↓ server-side Renatus API (no CORS) Renatus Back Office (lead created + registered) ↓ insert Supabase: funnel_leads table ↓ lead-admin-export edge function ← ICM exports here ``` **Key URLs:** - Landing page: `https://YOUR_REGISTRATION_PAGE/` (example) - Supabase edge: `https://.supabase.co/submit-renatus-registration` - Lead export: `https://.supabase.co/lead-admin-export` - Renatus back office: `https://backoffice.myrenatus.com` ## Email Campaign ### Send Batch ```bash python3 scripts/send_commercial_email_batches.py --batch-size 20 --start 0 --send ``` Resume from next batch: change `--start N`. Skip already-sent via `--skip-sent` (default on). See [email-campaign.md](references/email-campaign.md) for bounce handling, unsubscribe sync, and template customization. ### Prerequisites - `gws` CLI authenticated as sender account - `renatus_leads.csv` with `email` column - `commercial-core-day1.html` template ## Browser-Based Registration (CDP) Requires Chrome/Brave with `--remote-debugging-port=9222` and active Renatus session. ### Register Guest (dry run) ```bash python3 scripts/renatus_register_guest.py \ --first-name Jane --last-name Smith \ --email jane@example.com --phone "(555) 555-5555" \ --event-id 0817966f-b9bb-448e-bbb8-b4160115bcc8 ``` ### Execute Registration Add `--execute` to perform writes. ### Delete Lead (unsubscribe) ```bash python3 scripts/renatus_delete_lead.py --email jane@example.com --execute ``` ### Generate Events Calendar ```bash python3 scripts/generate_calendar.py --output site/calendar.html python3 scripts/generate_calendar.py --dry-run ``` Reads `events[]` from `config.json`, generates a calendar page listing all active events with registration links. ### Add Event (One-Command Setup) ```bash python3 scripts/add_event.py --event-url "https://backoffice.myrenatus.com/Events/EventDetails?eventId=..." --output site/my-event/index.html ``` Scrapes event from Renatus → adds to `config.json` → generates landing page in one step. Requires Chrome/Brave CDP + active Renatus session. ### Download Leads from Supabase ```bash python3 scripts/renatus_leads.py --export # Or: SUPABASE_URL=... LEAD_ADMIN_TOKEN=... python3 scripts/renatus_leads.py --export ``` Exports to `workspace/renatus_leads.csv` + `renatus_leads.json`. Set `SUPABASE_URL` and `LEAD_ADMIN_TOKEN` env vars, or pass `--ref` + `--token` flags. Use `--convert-json` to convert an existing JSON export to CSV format. ## Lead Export ### Supabase Admin Export ```bash curl -H "x-admin-token: $TOKEN" \ "https://.supabase.co/functions/v1/lead-admin-export?limit=500" ``` Token = `LEAD_ADMIN_TOKEN` secret in Supabase. See [supabase-setup.md](references/supabase-setup.md) for setup. ### Download via Agent If running as an OpenClaw agent: ```bash # From workspace: logs/email_send_log.json tracks sent emails # Download leads to: workspace/renatus_leads.csv ``` ## Configuration (config.json) Copy `config.json.example` to `config.json` and fill in your values. Scripts auto-load it: ## Configuration (config.json) Copy `config.json.example` to `config.json` and fill in your values. Scripts auto-load it: ```bash cp config.json.example config.json # edit config.json with your Supabase URL, Renatus credentials, sender account, etc. ``` Alternatively, set `RENATUS_*` environment variables. Scripts check (in order): `config.json` → env vars → defaults. Environment variables: `RENATUS_SUPABASE_REF`, `RENATUS_SUPABASE_URL`, `RENATUS_LEAD_TOKEN`, `RENATUS_USERNAME`, `RENATUS_PASSWORD`, `RENATUS_EVENT_ID`, `RENATUS_SENDER`, `RENATUS_PROVIDER`, `RENATUS_TEMPLATE`, `RENATUS_UNSUB_URL`, `RENATUS_SITE_URL`, `RENATUS_REGISTRATION_URL`. **Important:** `config.json` is gitignored. Your personal config stays private; `config.json.example` is what other ICMs receive. --- ## Supabase Setup (New ICM) See [supabase-setup.md](references/supabase-setup.md) for full setup: 1. Link project: `supabase link --project-ref ` 2. Push migrations: `supabase db push` 3. Deploy functions: `submit-renatus-registration`, `lead-admin-export` 4. Set secrets: `RENATUS_USERNAME`, `RENATUS_PASSWORD`, `RENATUS_EVENT_ID`, `LEAD_ADMIN_TOKEN` 5. Verify: submit test registration → check `funnel_leads` + Renatus back office ## Supabase Edge Function Reference ### submit-renatus-registration Public POST endpoint. Registers a lead in Renatus via server-side API (bypasses CORS). Auto-detects public-eligible sessions (no IMA/education requirements). Required secrets: `RENATUS_USERNAME`, `RENATUS_PASSWORD`, `RENATUS_EVENT_ID`. ### lead-admin-export Admin GET endpoint. Requires `x-admin-token` header. Params: `?partner=&limit=`. ### capture-lead General-purpose public lead intake with Turnstile bot protection. Insert into `funnel_leads`. ## Key Secrets (Supabase) | Secret | Required | Purpose | |---|---|---| | `RENATUS_USERNAME` | Registration | Back office login | | `RENATUS_PASSWORD` | Registration | Back office password | | `RENATUS_EVENT_ID` | Registration | Default event GUID | | `LEAD_ADMIN_TOKEN` | Export | Admin export auth | | `SUPABASE_URL` | Always | Project URL | | `SUPABASE_SERVICE_ROLE_KEY` | Always | DB access | ## Unsubscribe Sync (Weekly Cron) ```bash # Add to crontab 0 2 * * 0 /home/umbrel/.openclaw/workspace/scripts/weekly_unsubscribe_sync.sh # Manual run CDP_URL=http://127.0.0.1:9222 bash scripts/weekly_unsubscribe_sync.sh ``` Requirements: Chrome CDP at 9222 + active Renatus session. See [email-campaign.md](references/email-campaign.md). ## Bounce Recovery ```bash # Detect bounces python3 scripts/handle_bounced_emails.py --check # Export SMS-ready contacts python3 scripts/handle_bounced_emails.py --export-sms # Manually mark bounce python3 scripts/handle_bounced_emails.py --mark-bounced user@example.com ``` ## Authentication: Two Options ### Option A — Browser Relay (Recommended for CDP scripts) Use your existing logged-in Brave/Chrome session via Browser Relay (OpenClaw extension). 1. Open Brave → click the **OpenClaw Browser Relay** extension icon on any Renatus tab 2. Scripts connect to `http://127.0.0.1:9222` via Playwright CDP 3. No credentials needed in config — reads existing session tokens from localStorage/cookies 4. Requires the OpenClaw Browser Relay extension with an active tab **CDP scripts affected:** `renatus_register_guest.py`, `renatus_delete_lead.py`, `generate_event_page.py`, `add_event.py` ### Option B — Env Vars (Recommended for non-CDP scripts) Set credentials as environment variables — never in chat or config files. ```bash export RENATUS_USERNAME="earlvanze@gmail.com" export RENATUS_PASSWORD="YOUR_PASSWORD" export RENATUS_EVENT_ID="0817966f-b9bb-448e-bbb8-b4160115bcc8" export LEAD_ADMIN_TOKEN="YOUR_SUPABASE_TOKEN" ``` Or use a `.env` file loaded by your shell profile. Scripts read from env vars via `config_loader.py`. **Non-CDP scripts:** `renatus_leads.py`, `send_commercial_email_batches.py`, `generate_calendar.py`, `generate_email_template.py` ### Security Rules - **Never paste credentials in chat** — use env vars or `.env` files - **Bitwarden CLI** (`bw`) for programmatic credential access: `bw get password "Renatus"` - CDP reads do not extract your master password — only existing session tokens - Config file (`config.json`) should use `[REDACTED — see Nextcloud/secrets]` for credential placeholders - Rotate credentials after any shared exposure FILE:assets/sample_leads.csv name,phone,email,dateCreated "Jane Smith","+1 555-555-5555","jane.smith@example.com","Mar 1, 2026 10:00 AM (MST)" "John Doe","+1 555-123-4567","john.doe@example.com","Mar 2, 2026 2:30 PM (PST)" "Bob Wilson","","bob.wilson@example.com","Mar 3, 2026 9:15 AM (CST)" FILE:assets/site/confirmation.html Registration Confirmed | {{event_name}}

✓

You are Registered!

Your spot for {{event_name}} is reserved. Check your email for confirmation and calendar invite.

Event Details

📅

{{event_date}} {{event_time}}

📍

Location {{event_location}}

👥

Instructors {{instructors}}

What is Next

Check your email for a confirmation with calendar invite
Add the event to your calendar
You will receive Zoom link and materials 24 hours before the event

Know someone who should attend?

Share this training with fellow investors.

Back to Home

FILE:assets/site/unsubscribe.html Unsubscribe | Renatus, LLC

📧

Unsubscribe

Enter your email to unsubscribe from Renatus, LLC communications.

You have been unsubscribed.
You will no longer receive emails from Renatus, LLC.

Your email has been removed from our mailing list and will be queued for deletion from the Renatus database within 24 hours.

Something went wrong. Please try again or contact support at support@renatus.com.

FILE:references/email-campaign.md # Email Campaign Guide ## Overview Commercial email campaigns are sent from `YOUR_SENDER@gmail.com` via gws CLI. The email template is `commercial-core-day1.html`. Campaigns target leads from `renatus_leads.csv` and track delivery via `logs/email_send_log.json`. ## Campaign Assets | File | Purpose | |---|---| | `email/commercial-core-day1.html` | HTML email template | | `renatus_leads.csv` | Lead list (export via `renatus_leads.py --export`) | | `logs/email_send_log.json` | Sent log (resumable sends) | | `logs/bounced_emails.json` | Bounced email tracking | ### Getting the Leads CSV ```bash # Export fresh from Supabase python3 scripts/renatus_leads.py --export # Convert existing JSON to CSV (if you have old JSON export) python3 scripts/renatus_leads.py --convert-json ``` ## Sending Emails ### Prerequisites - `gws` CLI authenticated as `YOUR_SENDER@gmail.com` - `renatus_leads.csv` exists with at least `email` column - `commercial-core-day1.html` template ### Dry Run (Preview) ```bash python3 scripts/send_commercial_email_batches.py --dry-run ``` ### Send in Batches (with Resume) ```bash # First batch python3 scripts/send_commercial_email_batches.py --batch-size 20 --start 0 --send # Resume from batch 2 python3 scripts/send_commercial_email_batches.py --batch-size 20 --start 20 --send # Resume from batch 3 python3 scripts/send_commercial_email_batches.py --batch-size 20 --start 40 --send ``` ### Batch Script Flags | Flag | Default | Description | |---|---|---| | `--batch-size` | 20 | Emails per batch | | `--start` | 0 | Starting index | | `--dry-run` | — | Preview without sending | | `--send` | — | Actually send | | `--delay` | 2.0 | Seconds between sends | ## Personalizing the Template The template uses `{{recipient_email}}` as a placeholder for the unsubscribe link target. ### Adding Personalized Unsubscribe URL Replace `{{recipient_email}}` with a base64-encoded unsubscribe URL: ```python import base64 def encode_unsubscribe_url(email: str, base_url: str = "https://YOUR_DOMAIN/unsubscribe.html") -> str: encoded = base64.urlsafe_b64encode(email.encode()).decode().rstrip('=') return f"{base_url}?e={encoded}" ``` ### Unsubscribe URL in Email Template ```html Unsubscribe ``` When the user clicks unsubscribe, their email is pre-filled and they confirm removal. ## Subject Line Options Tested subject lines for Commercial Core: 1. `They turned $7K into $103M. Learn how (April 16)` 2. `Commercial real estate: free training from $100M operators` 3. `[Only 9 days] Learn commercial underwriting from operators` 4. `From skeptical to 90+ doors in 8 months (free training)` ## Bounce Handling ### Check for Bounces ```bash python3 scripts/handle_bounced_emails.py --check ``` ### Export Bounced Contacts with Phone Numbers for SMS ```bash python3 scripts/handle_bounced_emails.py --export-sms # Output: logs/sms_followup_list.csv ``` ### Manually Mark an Email as Bounced ```bash python3 scripts/handle_bounced_emails.py --mark-bounced user@example.com ``` ## Unsubscribe Sync (Weekly Automation) The `weekly_unsubscribe_sync.sh` script syncs unsubscribes from the web form to Renatus. ### Setup Cron Job ```bash # Run every Sunday at 2 AM 0 2 * * 0 /home/umbrel/.openclaw/workspace/scripts/weekly_unsubscribe_sync.sh >> /home/umbrel/.openclaw/workspace/logs/unsubscribe_cron.log 2>&1 ``` ### Manual Run ```bash CDP_URL=http://127.0.0.1:9222 bash scripts/weekly_unsubscribe_sync.sh ``` ### Requirements - Chrome/Brave with `--remote-debugging-port=9222` - Logged into `backoffice.myrenatus.com` - Unsubscribes stored in `data/unsubscribes_.txt` ## Deleting Leads from Renatus ### Single Lead (Dry Run) ```bash python3 scripts/renatus_delete_lead.py --email user@example.com ``` ### Single Lead (Execute) ```bash python3 scripts/renatus_delete_lead.py --email user@example.com --execute ``` ### Bulk Delete from File ```bash python3 scripts/renatus_delete_lead.py --file unsubscribes.txt --execute ``` ### Process Unsubscribe File ```bash python3 scripts/process_unsubscribes.py --file unsubscribes.txt --execute ``` ## Lead Download (Supabase Admin Export) ```bash TOKEN="your_lead_admin_token" REF="your_project_ref" curl -H "x-admin-token: $TOKEN" \ "https://$REF.supabase.co/functions/v1/lead-admin-export?limit=500" | \ jq '.rows[] | {name, email, phone: .metadata.phone}' > leads_export.csv ``` ## CSV Format (renatus_leads.csv) Expected columns: - `email` (required) — recipient address - `name` (optional) — full name or first name - `phone` (optional) — for SMS bounce recovery - `dateCreated` (optional) — lead creation date Example: ```csv email,name,phone,dateCreated jane@example.com,Jane Smith,5185551212,2026-03-01 john@example.com,John Doe,,2026-03-02 ``` FILE:references/event-page-setup.md # Event Page Setup Guide ## Overview The event landing page is a static HTML form at `site/commercial/index.html` (or similar path on your hostinger/YOUR_DOMAIN deployment). It submits directly to a Supabase Edge Function which registers the lead in Renatus Back Office. ## Page Structure ### Registration Form Fields - `firstName` (required) - `lastName` (required) - `email` (required) - `phone` (required) - `website` (honeypot — hidden, must be empty) - `source_page` (auto-populated from `window.location.href`) ### Submit Endpoint ``` POST https://.supabase.co/submit-renatus-registration ``` ### Request Body ```json { "firstName": "Jane", "lastName": "Smith", "email": "jane@example.com", "phone": "(555) 555-5555", "website": "", "source_page": "https://YOUR_DOMAIN/commercial/" } ``` ### Success Response ```json { "ok": true, "eventId": "0817966f-b9bb-448e-bbb8-b4160115bcc8", "eventName": "Commercial Core - Live Education", "registrationId": "...", "leadId": "...", "guestUserId": "...", "registeredSessions": ["Session Name 1", "Session Name 2"] } ``` ### Error Response ```json { "error": "Error message", "details": "optional details" } ``` ## Setting Up a New Event Page ### 1. Copy the Template Copy `site/commercial/index.html` to a new directory, e.g., `site//index.html` ### 2. Update the Edge Function URL In the ` """ # --------------------------------------------------------------------------- # JS: Extract event data from backoffice.myrenatus.com # --------------------------------------------------------------------------- EXTRACT_JS = r ## SECURITY NOTE: What this script reads ## - localStorage.getItem('auth') → extracts access_token for API calls ## - localStorage.getItem('__RequestVerificationToken') → XSRF token for POST requests ## - document.cookie → session cookies for authenticated requests ## NO passwords are extracted. This only reads existing browser session tokens. ## Use a dedicated Chrome profile (not your main session) for CDP access. """ async () => { const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); function cookies() { const out = {}; const raw = document.cookie || ''; for (const part of raw.split(/;\s*/)) { if (!part) continue; const idx = part.indexOf('='); const k = decodeURIComponent(idx >= 0 ? part.slice(0, idx) : part); const v = decodeURIComponent(idx >= 0 ? part.slice(idx + 1) : ''); out[k] = v; } return out; } function storageAuth() { try { return JSON.parse(localStorage.getItem('auth') || '{}') || {}; } catch (_) { return {}; } } let currentAuth = storageAuth(); function xsrfToken() { const ls = localStorage.getItem('__RequestVerificationToken'); if (ls) return ls; const c = cookies(); for (const [k, v] of Object.entries(c)) { if (k.toLowerCase().includes('requestverificationtoken') || k.toLowerCase().includes('xsrf')) return v; } return null; } function accessToken() { return currentAuth?.access_token || ''; } async function api(method, url, body = null, contentType = 'application/json; charset=UTF-8') { const headers = { 'x-requested-with': 'XMLHttpRequest' }; const xsrf = xsrfToken(); const bearer = accessToken(); if (xsrf) headers['x-xsrf-token'] = xsrf; if (bearer) headers['Authorization'] = `Bearer ${bearer}`; if (contentType) headers['content-type'] = contentType; const res = await fetch(url, { method, credentials: 'include', headers, body: body == null ? undefined : (typeof body === 'string' ? body : JSON.stringify(body)), }); const text = await res.text(); let parsed = null; try { parsed = JSON.parse(text); } catch (_) {} return { ok: res.ok, status: res.status, text, json: parsed }; } // Extract event ID from URL const urlParams = new URLSearchParams(location.search); let eventId = urlParams.get('eventId') || ''; // If no eventId in URL, try from page content / React state if (!eventId) { // Try React __STATE__ or similar const scripts = document.querySelectorAll('script'); for (const script of scripts) { const text = script.textContent || ''; const match = text.match(/eventId["\s:]+([^"',\s]+)/i); if (match) { eventId = match[1]; break; } } } if (!eventId) { return { ok: false, stage: 'event_id', message: 'Could not find eventId in URL or page. Paste a URL like: backoffice.myrenatus.com/Events/EventDetails?eventId=...' }; } // Fetch event let eventResp = await api('GET', `${location.origin}/api/queryproxy/execute?url=/api/event/getsavedevent?&Value=${encodeURIComponent(eventId)}`, null, null ); // Auth refresh retry if (eventResp.status === 401) { await sleep(1500); eventResp = await api('GET', `${location.origin}/api/queryproxy/execute?url=/api/event/getsavedevent?&Value=${encodeURIComponent(eventId)}`, null, null ); } if (eventResp.status === 401 || !eventResp.ok || !eventResp.json) { return { ok: false, stage: 'auth', message: 'Cannot access Renatus API from this browser session. Make sure you are logged into backoffice.myrenatus.com.', status: eventResp.status, hasXsrf: !!xsrfToken(), hasAccessToken: !!accessToken(), }; } const ev = eventResp.json; const sessions = (ev.Sessions || []).map((s) => ({ guid: s.SessionGuid, name: s.Name || 'Session', startDate: s.StartDate || '', endDate: s.EndDate || '', locationName: s.LocationName || 'Online', fee: s.Fee, inPersonFee: s.InPersonAttendingFee, mediumId: s.MediumId, requirements: { requireActiveIMA: !!s.RequireActiveIMA, requireXtreamEducation: !!s.RequireXtreamEducation, requireNoEducation: !!s.RequireNoEducation, isPublicEligible: !s.RequireActiveIMA && !s.RequireXtreamEducation && !s.RequireNoEducation && !s.RequireOneStarQualify && !s.RequireThreeStarQualify && !s.RequireFiveStarQualify && !s.RequireEssentialEducation && !s.RequireProfitEducation && !s.RequireAdvancedEducation && !s.RequireGeniusIn21DaysEducation && !s.IsLeadOnly && !s.IsNonLeadOnly, }, })); const publicSessions = sessions.filter((s) => s.requireXtreamEducation === false && !s.requireActiveIMA && !s.requireNoEducation); return { ok: true, eventId, eventName: ev.Name || 'Renatus Event', eventTagline: ev.Tagline || ev.Description || 'Register for this exclusive Renatus event.', eventDescription: ev.Description || 'Learn from expert instructors in this hands-on event.', locationName: ev.LocationName || publicSessions[0]?.locationName || 'Online', sessions, publicSessionCount: publicSessions.length, publicSessions, rawEvent: { name: ev.Name, description: ev.Description, tagline: ev.Tagline, startDate: ev.EventStartDate, endDate: ev.EventEndDate, locationName: ev.LocationName, }, }; } """ def parse_args() -> argparse.Namespace: ap = argparse.ArgumentParser( description="Generate a Renatus event landing page from a backoffice event URL." ) ap.add_argument( "--event-url", required=True, help="Renatus event URL, e.g. https://backoffice.myrenatus.com/Events/EventDetails?eventId=..." ) ap.add_argument( "--output", required=True, help="Output HTML path, e.g. site/my-event/index.html" ) ap.add_argument( "--edge-url", default="https://YOUR_PROJECT_REF.functions.supabase.co/submit-renatus-registration", help="Supabase edge function URL for form submission" ) ap.add_argument( "--company-name", default="Autonomous Ops Studio", help="Company name shown in the page" ) ap.add_argument( "--event-id", help="Override event ID (auto-detected from URL if omitted)" ) ap.add_argument( "--cdp-url", default=DEFAULT_CDP_URL, help="Chrome CDP URL" ) ap.add_argument( "--dry-run", action="store_true", help="Print extracted data without writing the HTML file" ) ap.add_argument( "--timeout-seconds", type=int, default=60 ) return ap.parse_args() def extract_event_id(url: str) -> str: try: from urllib.parse import parse_qs, urlparse parsed = urlparse(url) if parsed.query: params = dict(p.split("=", 1) for p in parsed.query.split("&") if "=" in p) return params.get("eventId", "") return "" except Exception: return "" def find_authenticated_page(browser, args: argparse.Namespace): """Find first browser context with active Renatus auth.""" diagnostics = [] for idx, context in enumerate(browser.contexts): page = context.new_page() try: page.goto(RENATUS_HOST, wait_until="domcontentloaded", timeout=15000) page.wait_for_load_state("networkidle", timeout=5000) except Exception: pass time.sleep(1) check = page.evaluate(""" () => { const auth = localStorage.getItem('auth'); const xsrf = localStorage.getItem('__RequestVerificationToken'); return { hasAuth: !!auth, hasXsrf: !!xsrf, url: location.href }; } """) diagnostics.append({"context": idx, **check}) if check.get("hasAuth") and check.get("hasXsrf"): return page, diagnostics try: page.close() except Exception: pass return None, diagnostics def render_page(event_data: dict, args: argparse.Namespace) -> str: ev = event_data sessions = ev.get("sessions", []) public_sessions = ev.get("publicSessions", []) # Format dates date_parts = set() location_parts = set() speaker_parts = [] for s in sessions: start = s.get("startDate", "") if start: # e.g. 2026-04-16T09:00:00 → Apr 16, 2026 try: from datetime import datetime dt = datetime.fromisoformat(start.replace("Z", "+00:00")) date_parts.add(dt.strftime("%B %d, %Y")) except Exception: date_parts.add(start[:10]) loc = s.get("locationName", "") if loc: location_parts.add(loc) date_display = "; ".join(sorted(date_parts)) if date_parts else ev.get("rawEvent", {}).get("startDate", "TBD") if len(date_parts) > 1: date_display = "See session details" location_display = " / ".join(sorted(location_parts)) if location_parts else ev.get("locationName", "Online") # Speakers (session names often contain instructor names) session_names = [s.get("name", "") for s in sessions] # De-duplicate and filter short names seen = set() for n in session_names: if n and len(n) > 2: speaker_key = n.lower() if speaker_key not in seen: seen.add(speaker_key) speaker_parts.append(n) speakers_display = ", ".join(speaker_parts[:4]) if speaker_parts else "Renatus Instructors" event_id = args.event_id or ev.get("eventId", "") event_name = ev.get("eventName", "Renatus Event") event_tagline = ev.get("eventTagline") or ev.get("rawEvent", {}).get("tagline") or f"Register for {event_name}" event_description = ev.get("eventDescription") or ev.get("rawEvent", {}).get("description") or "Learn from expert instructors in this hands-on event." # Truncate description if too long if len(event_description) > 300: event_description = event_description[:297] + "..." meta_description = f"Register for {event_name}. {event_description[:150]}" html = EVENT_PAGE_TEMPLATE.format( event_id=event_id, confirmation_url=args.confirmation_url or 'confirmation.html', event_name=event_name, event_tagline=event_tagline, event_description=event_description, meta_description=meta_description, date_display=date_display, location_display=location_display, speakers_display=speakers_display, edge_url=args.edge_url, company_name=args.company_name, s_plural="" if len(date_parts) == 1 else "s", accent_color="#22c55e", accent_color_dark="#16a34a", ) # Inject event config JSON separately to avoid format() parsing JSON braces event_config_json = json.dumps({ 'event_name': event_name, 'event_date': event_date, 'event_time': event_time, 'event_location': location_display, 'instructors': speakers_display, }, default='') html = html.replace('__EVENT_CONFIG_JSON__', event_config_json) return html def resolve_event_url(args) -> str: """Resolve the Renatus event URL: from --event-url, --event-id (config lookup), or prompt.""" if args.event_url: return args.event_url # Try config lookup by event-id if args.event_id: try: sys.path.insert(0, str(Path(__file__).parent)) from config_loader import get_event_by_id ev = get_event_by_id(args.event_id) if ev and ev.get("renatus_url"): print(f"Found event {args.event_id} in config: {ev.get('name', '?')}") return ev["renatus_url"] if ev: # Construct URL from event ID return f"https://backoffice.myrenatus.com/Events/EventDetails?eventId={args.event_id}" print(f"Warning: event {args.event_id} not found in config.json", file=sys.stderr) except Exception: pass return f"https://backoffice.myrenatus.com/Events/EventDetails?eventId={args.event_id}" # Prompt-free: construct from common pattern raise SystemExit("Error: --event-url or --event-id required. Run with --help for usage.") def main() -> int: args = parse_args() event_id_from_url = extract_event_id(args.event_url) if not args.event_id: args.event_id = event_id_from_url print(f"Event URL: {args.event_url}") print(f"Output: {args.output}") print() with sync_playwright() as p: browser = p.chromium.connect_over_cdp(args.cdp_url, timeout=args.timeout_seconds * 1000) page, diagnostics = find_authenticated_page(browser, args) if page is None: print("ERROR: No authenticated Renatus session found.", file=sys.stderr) print("Make sure Chrome/Brave is open to backoffice.myrenatus.com and you are logged in.", file=sys.stderr) print("Diagnostics:", json.dumps(diagnostics, indent=2), file=sys.stderr) return 1 # Navigate to the event URL print(f"Navigating to: {args.event_url}") page.goto(args.event_url, wait_until="domcontentloaded", timeout=args.timeout_seconds * 1000) page.wait_for_load_state("networkidle", timeout=10000) time.sleep(2) # Extract event data print("Extracting event data from Renatus...") result = page.evaluate(EXTRACT_JS) if not result.get("ok"): print(f"FAILED: {result.get('stage')} — {result.get('message')}", file=sys.stderr) if result.get("diagnostics"): print("Diagnostics:", json.dumps(result["diagnostics"], indent=2), file=sys.stderr) return 1 print(f"Event name: {result.get('eventName')}") print(f"Event ID: {result.get('eventId')}") sessions = result.get("sessions", []) print(f"Sessions: {len(sessions)} total / {result.get('publicSessionCount')} public-eligible") for s in sessions: reqs = [k for k, v in s.get("requirements", {}).items() if v] req_str = f" [{', '.join(reqs)}]" if reqs else " [PUBLIC]" print(f" - {s['name']} | {s['startDate'][:10] if s.get('startDate') else 'TBD'} | {s['locationName']}{req_str}") if args.dry_run: print("\nDry run — not writing file.") page.close() return 0 # Render HTML html = render_page(result, args) # Write output out_path = Path(args.output) out_path.parent.mkdir(parents=True, exist_ok=True) out_path.write_text(html) print(f"\nGenerated: {out_path.absolute()}") print(f"Edge URL: {args.edge_url}") print(f"Event ID: {result.get('eventId')}") print() print("Next steps:") print(f" 1. Review the file: open {out_path}") print(f" 2. Update SUPABASE secrets: RENATUS_EVENT_ID={result.get('eventId')}") print(f" 3. Deploy: python3 scripts/deploy_hostinger.py (or FTP upload)") page.close() return 0 if __name__ == "__main__": raise SystemExit(main()) FILE:scripts/renatus_delete_lead.py #!/usr/bin/env python3 """ renatus_delete_lead.py Attach to a live authenticated browser session over CDP and delete a lead from Renatus by email. This is the unsubscribe companion to renatus_register_guest.py. Requires: A running Chrome/Brave instance with --remote-debugging-port=9222 AND an active logged-in session to backoffice.myrenatus.com Examples: # Dry run - find the lead but don't delete python3 scripts/renatus_delete_lead.py --email user@example.com # Actually delete python3 scripts/renatus_delete_lead.py --email user@example.com --execute # Delete multiple from a file python3 scripts/renatus_delete_lead.py --file unsubscribes.txt --execute """ from __future__ import annotations import argparse import json import sys import time from pathlib import Path from playwright.sync_api import sync_playwright DEFAULT_CDP_URL = "http://127.0.0.1:9222" RENATUS_HOST = "https://backoffice.myrenatus.com" RENATUS_HOME_URL = f"{RENATUS_HOST}/Home/index#/index" JS_DELETE_FLOW = r""" async (args) => { const { email, execute } = args; const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); function cookies() { const out = {}; const raw = document.cookie || ''; for (const part of raw.split(/;\s*/)) { if (!part) continue; const idx = part.indexOf('='); const k = decodeURIComponent(idx >= 0 ? part.slice(0, idx) : part); const v = decodeURIComponent(idx >= 0 ? part.slice(idx + 1) : ''); out[k] = v; } return out; } function storageAuth() { try { return JSON.parse(localStorage.getItem('auth') || '{}') || {}; } catch (_) { return {}; } } let currentAuth = storageAuth(); function xsrfToken() { const ls = localStorage.getItem('__RequestVerificationToken'); if (ls) return ls; const c = cookies(); for (const [k, v] of Object.entries(c)) { if (k.toLowerCase().includes('requestverificationtoken') || k.toLowerCase().includes('xsrf')) return v; } return null; } function accessToken() { return currentAuth?.access_token || ''; } async function api(method, url, body = null, contentType = 'application/json; charset=UTF-8') { const headers = { 'x-requested-with': 'XMLHttpRequest', }; const xsrf = xsrfToken(); const bearer = accessToken(); if (xsrf) headers['x-xsrf-token'] = xsrf; if (bearer && !url.endsWith('/Token')) headers['Authorization'] = `Bearer ${bearer}`; if (contentType) headers['content-type'] = contentType; const res = await fetch(url, { method, credentials: 'include', headers, body: body == null ? undefined : (typeof body === 'string' ? body : JSON.stringify(body)), }); const text = await res.text(); let parsed = null; try { parsed = JSON.parse(text); } catch (_) {} return { ok: res.ok, status: res.status, url: res.url, text, json: parsed, }; } // Step 1: Search for lead by email const searchResp = await api('GET', `${location.origin}/api/queryproxy/execute?url=/api/lead/search?` + `&Email=${encodeURIComponent(email)}&CurrentPage=0&PageSize=10`, null, null ); if (!searchResp.ok || !searchResp.json) { return { ok: false, stage: 'search', message: 'Failed to search for lead', status: searchResp.status, }; } const results = searchResp.json.Results || searchResp.json.results || searchResp.json.Data || []; if (!results.length) { return { ok: true, stage: 'not_found', message: 'Lead not found in Renatus (may already be deleted or email mismatch)', email, }; } const lead = results[0]; const leadId = lead.Id || lead.id || lead.LeadId || lead.leadId; const leadName = lead.Name || lead.FullName || `${lead.FirstName || ''} ${lead.LastName || ''}`.trim(); if (!leadId) { return { ok: false, stage: 'parse', message: 'Could not extract lead ID from search results', lead, }; } if (!execute) { return { ok: true, stage: 'dry_run', message: 'Found lead - would delete if --execute was passed', email, leadId, leadName, totalResults: results.length, }; } // Step 2: Delete the lead // Try multiple endpoints that might work const deleteEndpoints = [ `/api/commandproxy/execute?url=/api/lead/delete?`, `/api/commandproxy/execute?url=/api/marketingLead/delete?`, `/api/lead/delete?`, ]; let deleteResp = null; let lastError = null; for (const endpoint of deleteEndpoints) { try { deleteResp = await api('POST', `${location.origin}${endpoint}`, { Value: leadId, Key: leadId, Id: leadId, LeadId: leadId, }); if (deleteResp.ok || deleteResp.status === 200) { break; } } catch (e) { lastError = e; } } if (!deleteResp || (!deleteResp.ok && deleteResp.status !== 200)) { return { ok: false, stage: 'delete', message: 'Failed to delete lead after trying multiple endpoints', leadId, lastStatus: deleteResp?.status, lastError: lastError?.message, }; } return { ok: true, stage: 'deleted', email, leadId, leadName, deleteResponse: deleteResp.json || deleteResp.text, }; } """ def parse_args() -> argparse.Namespace: ap = argparse.ArgumentParser(description="Delete a Renatus lead via a live authenticated browser session.") ap.add_argument("--email", help="Email of lead to delete") ap.add_argument("--file", help="File containing emails to delete (one per line)") ap.add_argument("--cdp-url", default=DEFAULT_CDP_URL, help="Chrome/Brave CDP URL") ap.add_argument("--execute", action="store_true", help="Actually delete (default: dry-run)") ap.add_argument("--json", action="store_true", help="Print raw JSON result") ap.add_argument("--timeout-seconds", type=int, default=45) return ap.parse_args() def delete_single(page, email: str, execute: bool) -> dict: return page.evaluate(JS_DELETE_FLOW, {"email": email, "execute": execute}) def find_authenticated_page(browser, args: argparse.Namespace): diagnostics = [] if not browser.contexts: raise RuntimeError("No browser contexts found on attached CDP browser.") for context_index, context in enumerate(browser.contexts): page = context.new_page() try: page.goto(RENATUS_HOME_URL, wait_until="domcontentloaded", timeout=args.timeout_seconds * 1000) page.wait_for_load_state("networkidle", timeout=5000) except Exception: pass time.sleep(2) # Quick auth check result = page.evaluate(""" () => { const auth = localStorage.getItem('auth'); const xsrf = localStorage.getItem('__RequestVerificationToken'); return { hasAuth: !!auth, hasXsrf: !!xsrf, currentUrl: location.href }; } """) diagnostics.append({ "contextIndex": context_index, "hasAuth": result.get("hasAuth"), "hasXsrf": result.get("hasXsrf"), "url": result.get("currentUrl"), }) if result.get("hasAuth") and result.get("hasXsrf"): return page, diagnostics try: page.close() except Exception: pass return None, diagnostics def main() -> int: args = parse_args() if not args.email and not args.file: print("Either --email or --file is required", file=sys.stderr) return 2 emails = [] if args.email: emails.append(args.email) if args.file: path = Path(args.file) if not path.exists(): print(f"File not found: {args.file}", file=sys.stderr) return 2 emails.extend([line.strip() for line in path.read_text().splitlines() if line.strip()]) print(f"Processing {len(emails)} email(s)...") print(f"Mode: {'DELETE' if args.execute else 'DRY RUN'}") print() with sync_playwright() as p: browser = p.chromium.connect_over_cdp(args.cdp_url, timeout=args.timeout_seconds * 1000) page, diagnostics = find_authenticated_page(browser, args) if page is None: print("FAILED: No authenticated Renatus session found", file=sys.stderr) print("Diagnostics:", json.dumps(diagnostics, indent=2), file=sys.stderr) print("\nMake sure you have:", file=sys.stderr) print("1. Chrome/Brave running with: --remote-debugging-port=9222", file=sys.stderr) print("2. Logged into backoffice.myrenatus.com in that browser", file=sys.stderr) return 1 results = [] for email in emails: print(f"Processing: {email}") result = delete_single(page, email, args.execute) results.append(result) if args.json: print(json.dumps(result, indent=2)) else: if result.get("ok"): if result.get("stage") == "deleted": print(f" ✓ Deleted: {result.get('leadName')} (ID: {result.get('leadId')})") elif result.get("stage") == "not_found": print(f" ℹ Not found (already deleted or different email)") elif result.get("stage") == "dry_run": print(f" → Would delete: {result.get('leadName')} (ID: {result.get('leadId')})") else: print(f" ✗ Failed: {result.get('stage')} - {result.get('message')}") print() try: page.close() except Exception: pass # Summary successful = sum(1 for r in results if r.get("ok")) print(f"\nSummary: {successful}/{len(results)} successful") return 0 if successful == len(results) else 1 if __name__ == "__main__": raise SystemExit(main()) FILE:scripts/renatus_leads.py #!/usr/bin/env python3 """ renatus_leads.py Export leads from the Renatus ICM Supabase project into workspace CSV/JSON files. Usage: # Export all leads to workspace/renatus_leads.csv and .json python3 scripts/renatus_leads.py --export # Dry run (show data without saving) python3 scripts/renatus_leads.py --dry-run # Export a specific number python3 scripts/renatus_leads.py --export --limit 200 # Convert existing workspace JSON to CSV python3 scripts/renatus_leads.py --convert-json Requirements: - SUPABASE_URL and LEAD_ADMIN_TOKEN environment variables (or --ref/--token flags) """ from __future__ import annotations import argparse import csv import json import os import sys import urllib.request import urllib.error from datetime import datetime from pathlib import Path try: sys.path.insert(0, str(Path(__file__).parent)) from config_loader import load_config _cfg = load_config() except Exception: _cfg = {} DEFAULT_WORKSPACE = Path("/home/umbrel/.openclaw/workspace") DEFAULT_CSV = DEFAULT_WORKSPACE / "renatus_leads.csv" DEFAULT_JSON = DEFAULT_WORKSPACE / "renatus_leads.json" def parse_args() -> argparse.Namespace: ap = argparse.ArgumentParser(description="Export and manage Renatus leads from Supabase.") ap.add_argument("--export", action="store_true", help="Export leads from Supabase to CSV/JSON") ap.add_argument("--convert-json", action="store_true", help="Convert renatus_leads.json to renatus_leads.csv") ap.add_argument("--dry-run", action="store_true", help="Show data without saving files") ap.add_argument("--limit", type=int, default=500, help="Max leads to export (default: 500)") supabase_url_cfg = _cfg.get("supabase_url", "") if isinstance(_cfg, dict) else "" supabase_ref_cfg = _cfg.get("supabase_ref", "") if isinstance(_cfg, dict) else "" ap.add_argument("--ref", default=os.environ.get("SUPABASE_URL", supabase_ref_cfg or "").replace("https://", "").replace(".supabase.co", ""), help="Supabase project ref (default: from config.json or env)") lead_token_cfg = _cfg.get("lead_token", "") if isinstance(_cfg, dict) else "" ap.add_argument("--token", default=os.environ.get("LEAD_ADMIN_TOKEN", lead_token_cfg), help="Lead admin token (default: from config.json or env)") ap.add_argument("--output-dir", type=Path, default=DEFAULT_WORKSPACE, help="Output directory for CSV/JSON files") ap.add_argument("--format", choices=["csv", "json", "both"], default="both", help="Output format (default: both)") return ap.parse_args() def fetch_leads(ref: str, token: str, limit: int) -> list[dict]: """Fetch leads from lead-admin-export edge function.""" url = f"https://{ref}.supabase.co/functions/v1/lead-admin-export?limit={limit}" req = urllib.request.Request(url, headers={ "x-admin-token": token, "Accept": "application/json", }) try: with urllib.request.urlopen(req, timeout=30) as resp: data = json.loads(resp.read()) except urllib.error.HTTPError as e: body = e.read().decode() if e.fp else "" raise RuntimeError(f"HTTP {e.code}: {body}. Check SUPABASE_URL / RENATUS_SUPABASE_REF and LEAD_ADMIN_TOKEN / RENATUS_LEAD_TOKEN.") from e if not data.get("ok"): raise RuntimeError(f"API error: {data.get('error', 'unknown')}") rows = data.get("rows", []) print(f"Fetched {len(rows)} leads from Supabase") return rows def rows_to_csv(rows: list[dict], path: Path) -> None: """Write leads to CSV.""" if not rows: print("No rows to write") return # Flatten top-level fields + phone from metadata fieldnames = ["name", "email", "phone", "company", "notes", "cta_type", "source_page", "created_at"] meta_fields = set() for row in rows: meta = row.get("metadata") or {} meta_fields.update(meta.keys()) all_fields = fieldnames + sorted(meta_fields) with open(path, "w", newline="", encoding="utf-8") as f: writer = csv.DictWriter(f, fieldnames=all_fields, extrasaction="ignore") writer.writeheader() for row in rows: meta = row.get("metadata") or {} flat = {k: v for k, v in row.items() if k != "metadata"} flat["phone"] = meta.get("phone", "") flat["event_id"] = meta.get("event_id", "") flat["registration_id"] = meta.get("registration_id", "") flat["lead_id"] = meta.get("lead_id", "") flat["guest_user_id"] = meta.get("guest_user_id", "") flat["registered_sessions"] = "; ".join(meta.get("registered_sessions", [])) if meta.get("registered_sessions") else "" for k, v in meta.items(): if k not in flat: flat[k] = v writer.writerow(flat) print(f"Written: {path} ({path.stat().st_size:,} bytes)") def rows_to_json(rows: list[dict], path: Path) -> None: """Write leads to JSON.""" with open(path, "w", encoding="utf-8") as f: json.dump({ "exported_at": datetime.utcnow().isoformat() + "Z", "total": len(rows), "rows": rows, }, f, indent=2, default=str) print(f"Written: {path} ({path.stat().st_size:,} bytes)") def convert_json_to_csv(json_path: Path, csv_path: Path) -> None: """Convert existing renatus_leads.json to CSV.""" with open(json_path) as f: data = json.load(f) rows = data if isinstance(data, list) else data.get("rows", []) if not rows: print("No rows found in JSON file") return rows_to_csv(rows, csv_path) def show_sample(rows: list[dict], n: int = 5) -> None: """Print a sample of leads.""" for row in rows[:n]: phone = (row.get("metadata") or {}).get("phone", "") sessions = (row.get("metadata") or {}).get("registered_sessions", []) print(f" {row.get('name', '?')} <{row.get('email', '?')}> | {phone} | {row.get('created_at', '')}") if sessions: print(f" sessions: {', '.join(sessions)}") if len(rows) > n: print(f" ... and {len(rows) - n} more") def main() -> int: args = parse_args() args.output_dir.mkdir(parents=True, exist_ok=True) # --convert-json if args.convert_json: json_path = args.output_dir / "renatus_leads.json" if not json_path.exists(): print(f"File not found: {json_path}", file=sys.stderr) return 1 csv_path = args.output_dir / "renatus_leads.csv" print(f"Converting {json_path} → {csv_path}") convert_json_to_csv(json_path, csv_path) return 0 # --export if args.export: if not args.ref or not args.token: print("ERROR: Missing --ref or --token. Set in config.json or set RENATUS_SUPABASE_REF / RENATUS_LEAD_TOKEN env vars.", file=sys.stderr) print(" Set SUPABASE_URL and LEAD_ADMIN_TOKEN env vars, or pass --ref and --token.", file=sys.stderr) return 1 print(f"Fetching up to {args.limit} leads from {args.ref}.supabase.co...") try: rows = fetch_leads(args.ref, args.token, args.limit) except RuntimeError as e: print(f"ERROR: {e}", file=sys.stderr) return 1 if args.dry_run: print("\nSample (first 5):") show_sample(rows) return 0 if args.format in ("csv", "both"): csv_path = args.output_dir / "renatus_leads.csv" rows_to_csv(rows, csv_path) if args.format in ("json", "both"): json_path = args.output_dir / "renatus_leads.json" rows_to_json(rows, json_path) print(f"\nTotal leads exported: {len(rows)}") print(f"Files: {args.output_dir / 'renatus_leads.csv'} + .json") return 0 # Default: show status of workspace leads files csv_path = args.output_dir / "renatus_leads.csv" json_path = args.output_dir / "renatus_leads.json" if csv_path.exists(): with open(csv_path) as f: count = sum(1 for _ in f) - 1 print(f"CSV: {csv_path} ({count:,} leads, {csv_path.stat().st_size:,} bytes)") else: print(f"CSV: not found (run with --export)") if json_path.exists(): with open(json_path) as f: data = json.load(f) total = len(data) if isinstance(data, list) else data.get("total", "?") print(f"JSON: {json_path} ({total:,} leads, {json_path.stat().st_size:,} bytes)") else: print(f"JSON: not found (run with --export)") print("\nUsage:") print(" --export Export from Supabase to CSV/JSON") print(" --convert-json Convert renatus_leads.json to CSV") print(" --dry-run Preview without saving") return 0 if __name__ == "__main__": raise SystemExit(main()) FILE:scripts/renatus_register_guest.py #!/usr/bin/env python3 """ renatus_register_guest.py Attach to a live authenticated browser session over CDP and register a guest for a Renatus event using the Back Office API flow recovered from Dropbox/registration_backoffice.myrenatus.com.har. Safety: - Dry-run by default (no write calls). - Use --execute for the real registration. Default session behavior: - If no --session-guid flags are provided and --all-sessions is not set, the script defaults to the FIRST event session only. This matches the captured HAR registration submit more closely than blindly registering every session. Examples: python3 scripts/renatus_register_guest.py \ --first-name Test --last-name Guest \ --email test@example.com --phone '(518) 555-1212' \ --event-id 0817966f-b9bb-448e-bbb8-b4160115bcc8 python3 scripts/renatus_register_guest.py \ --first-name Test --last-name Guest \ --email test@example.com --phone '(518) 555-1212' \ --event-id 0817966f-b9bb-448e-bbb8-b4160115bcc8 \ --all-sessions --execute """ from __future__ import annotations import argparse import json import sys import time from pathlib import Path from playwright.sync_api import sync_playwright DEFAULT_EVENT_ID = "0817966f-b9bb-448e-bbb8-b4160115bcc8" DEFAULT_CDP_URL = "http://127.0.0.1:9222" DEFAULT_SOURCE_ID = 8115 RENATUS_HOST = "https://backoffice.myrenatus.com" RENATUS_HOME_URL = f"{RENATUS_HOST}/Home/index#/index" JS_FLOW = r""" async (args) => { const { eventId, firstName, lastName, email, phone, sourceId, execute, allSessions, explicitSessionGuids, } = args; const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); function cookies() { const out = {}; const raw = document.cookie || ''; for (const part of raw.split(/;\s*/)) { if (!part) continue; const idx = part.indexOf('='); const k = decodeURIComponent(idx >= 0 ? part.slice(0, idx) : part); const v = decodeURIComponent(idx >= 0 ? part.slice(idx + 1) : ''); out[k] = v; } return out; } function storageAuth() { try { return JSON.parse(localStorage.getItem('auth') || '{}') || {}; } catch (_) { return {}; } } let currentAuth = storageAuth(); function xsrfToken() { const ls = localStorage.getItem('__RequestVerificationToken'); if (ls) return ls; const c = cookies(); for (const [k, v] of Object.entries(c)) { if (k.toLowerCase().includes('requestverificationtoken') || k.toLowerCase().includes('xsrf')) return v; } return null; } function accessToken() { return currentAuth?.access_token || ''; } async function api(method, url, body = null, contentType = 'application/json; charset=UTF-8') { const headers = { 'x-requested-with': 'XMLHttpRequest', }; const xsrf = xsrfToken(); const bearer = accessToken(); if (xsrf) headers['x-xsrf-token'] = xsrf; if (bearer && !url.endsWith('/Token')) headers['Authorization'] = `Bearer ${bearer}`; if (contentType) headers['content-type'] = contentType; const res = await fetch(url, { method, credentials: 'include', headers, body: body == null ? undefined : (typeof body === 'string' ? body : JSON.stringify(body)), }); const text = await res.text(); let parsed = null; try { parsed = JSON.parse(text); } catch (_) {} return { ok: res.ok, status: res.status, url: res.url, text, json: parsed, headers: Object.fromEntries(res.headers.entries()), }; } async function getEvent() { return api('GET', `${location.origin}/api/queryproxy/execute?url=/api/event/getsavedevent?&Value=${encodeURIComponent(eventId)}`, null, null); } async function maybeRefreshAuth() { const attempts = []; const refreshToken = currentAuth?.refresh_token || storageAuth()?.refresh_token || ''; if (!refreshToken) { return { refreshed: false, attempts, refreshTokensTried: 0, eventResp: await getEvent(), }; } const body = `refresh_token=${encodeURIComponent(refreshToken)}&grant_type=refresh_token`; const resp = await api('POST', `${location.origin}/Token`, body, 'application/x-www-form-urlencoded'); attempts.push({status: resp.status, ok: resp.ok, hasAccessToken: !!resp?.json?.access_token}); if (resp?.json?.access_token) { currentAuth = { ...currentAuth, ...resp.json }; try { localStorage.setItem('auth', JSON.stringify(currentAuth)); } catch (_) {} } await sleep(800); return { refreshed: !!resp?.json?.access_token, attempts, refreshTokensTried: 1, eventResp: await getEvent(), }; } let eventResp = await getEvent(); let refreshMeta = { refreshed: false, attempts: [], refreshTokensTried: 0, }; if (eventResp.status === 401) { await sleep(1500); eventResp = await getEvent(); } if (eventResp.status === 401) { refreshMeta = await maybeRefreshAuth(); eventResp = refreshMeta.eventResp; } if (eventResp.status === 401 || !eventResp.ok || !eventResp.json) { return { ok: false, stage: 'auth', message: 'Unable to access Renatus event API from the attached browser session.', eventStatus: eventResp.status, hasXsrf: !!xsrfToken(), hasAccessToken: !!accessToken(), hasRefreshToken: !!(currentAuth?.refresh_token), cookieNames: Object.keys(cookies()), localStorageKeys: Object.keys(localStorage || {}), refreshMeta, currentUrl: location.href, title: document.title, }; } const eventData = eventResp.json; const allEventSessions = (eventData.Sessions || []).map((s) => ({ sessionGuid: s.SessionGuid, name: s.Name, startDate: s.StartDate, endDate: s.EndDate, fee: s.Fee, inPersonFee: s.InPersonAttendingFee, mediumId: s.MediumId, capacity: s.Capacity, requireXtreamEducation: !!s.RequireXtreamEducation, requireActiveIMA: !!s.RequireActiveIMA, requireNoEducation: !!s.RequireNoEducation, locationName: s.LocationName, })); let selectedSessions = []; if (explicitSessionGuids && explicitSessionGuids.length) { selectedSessions = allEventSessions.filter((s) => explicitSessionGuids.includes(s.sessionGuid)); } else if (allSessions) { selectedSessions = allEventSessions.slice(); } else { selectedSessions = allEventSessions.slice(0, 1); } if (!selectedSessions.length) { return { ok: false, stage: 'session_selection', message: 'No matching event sessions selected.', allEventSessions, }; } if (!execute) { return { ok: true, stage: 'dry_run', eventId, eventName: eventData.Name, eventStartDate: eventData.EventStartDate, eventEndDate: eventData.EventEndDate, selectedSessions, allEventSessions, guest: { firstName, lastName, email, phone }, defaults: { sourceId, defaultMode: explicitSessionGuids?.length ? 'explicit-sessions' : (allSessions ? 'all-sessions' : 'first-session-only'), }, auth: { hasXsrf: !!xsrfToken(), refreshMeta, }, }; } const addCustomerPayload = { LeadId: '', OwnerId: '', CampaignId: '0', CampaignStatus: '', Phone: { CountryId: 'US', Number: phone }, Address: { AddressLine1: '', AddressLine2: '', City: '', State: '', PostalCode: '', Country: 'US', }, Email: email, FirstName: firstName, LastName: lastName, SourceId: sourceId, IsGuest: false, IsCustomer: true, MotivationText: '', OptIn: true, CanSms: true, ConsentDate: '', IsInternationalCustomer: false, }; const addResp = await api('POST', `${location.origin}/api/commandproxy/execute?url=/api/guestRegistration/addcustomer?`, addCustomerPayload); if (!addResp.ok || !addResp.json || !addResp.json.User) { return { ok: false, stage: 'addcustomer', message: 'guestRegistration/addcustomer failed', response: { status: addResp.status, text: addResp.text, }, }; } const addedUser = addResp.json.User; const ownerId = addedUser.OwnerId || addedUser.EnteredById || null; const leadId = addedUser.LeadId || null; if (!ownerId || !leadId) { return { ok: false, stage: 'addcustomer-parse', message: 'Could not extract ownerId/leadId from addcustomer response', response: addResp.json, }; } const customerResp = await api('POST', `${location.origin}/api/commandproxy/execute?url=/api/marketingLead/customerbyleadid?`, { Value: leadId, Key: ownerId, }); if (!customerResp.ok || !customerResp.json || !customerResp.json.CustomerUserId) { return { ok: false, stage: 'customerbyleadid', message: 'marketingLead/customerbyleadid failed', response: { status: customerResp.status, text: customerResp.text, }, }; } const guestUserId = customerResp.json.CustomerUserId; const feesResp = await api('POST', `${location.origin}/api/commandproxy/execute?url=/api/registration/fees?`, { EventId: eventId, Guests: [{ GuestLeadId: leadId, GuestUserId: guestUserId, Sessions: selectedSessions.map((s) => ({ SessionId: s.sessionGuid })), }], IsGuestParentRegistering: false, }); const feeGuest = feesResp?.json?.Guests?.[0]; if (!feesResp.ok || !feeGuest) { return { ok: false, stage: 'fees', message: 'registration/fees failed', response: { status: feesResp.status, text: feesResp.text, }, }; } const registrationSessions = (feeGuest.Sessions || []).map((s) => ({ SessionId: s.SessionId, Cost: Number(s.Fees || 0), PromoCodeApplied: '', QuestionAndAnswers: [], })); const totalCost = registrationSessions.reduce((sum, s) => sum + (Number(s.Cost) || 0), 0); const registerPayload = { EventId: eventId, Cost: totalCost, UserId: ownerId, GuestsList: [{ GuestId: leadId, GuestUserId: guestUserId, FirstName: firstName, LastName: lastName, IsGuest: false, SessionsList: registrationSessions, AttendInPersonChecked: false, }], PaymentsOptions: [], Added: new Date().toISOString(), NoOfTickets: 0, Seats: [], }; const registerResp = await api('POST', `${location.origin}/api/commandproxy/execute?url=/api/eventRegistration/registerForEvent?`, registerPayload); if (!registerResp.ok || !registerResp.json) { return { ok: false, stage: 'registerForEvent', message: 'eventRegistration/registerForEvent failed', response: { status: registerResp.status, text: registerResp.text, }, payloadPreview: registerPayload, }; } const reg = registerResp.json; return { ok: true, stage: 'registered', eventId, eventName: eventData.Name, selectedSessions, guest: { firstName, lastName, email, phone, leadId, guestUserId, ownerId, }, fees: { totalCost, sessions: registrationSessions, }, registration: { eventRegistrationContainerId: reg.EventRegistrationContainerId || null, guestRegistrationId: reg.GuestsList?.[0]?.RegistrationId || null, returnedGuestId: reg.GuestsList?.[0]?.GuestId || null, returnedGuestUserId: reg.GuestsList?.[0]?.GuestUserId || null, sessionIds: (reg.GuestsList?.[0]?.SessionsList || []).map((s) => s.SessionId), }, }; } """ def parse_args() -> argparse.Namespace: ap = argparse.ArgumentParser(description="Register a Renatus guest via a live authenticated browser session.") ap.add_argument("--first-name", required=True) ap.add_argument("--last-name", required=True) ap.add_argument("--email", required=True) ap.add_argument("--phone", required=True) ap.add_argument("--event-id", default=DEFAULT_EVENT_ID) ap.add_argument("--source-id", type=int, default=DEFAULT_SOURCE_ID) ap.add_argument("--cdp-url", default=DEFAULT_CDP_URL, help="Chrome/Brave CDP URL, e.g. http://127.0.0.1:9222") ap.add_argument("--session-guid", action="append", dest="session_guids", default=[], help="Repeat to select specific session GUID(s)") ap.add_argument("--all-sessions", action="store_true", help="Register all event sessions instead of default first session only") ap.add_argument("--execute", action="store_true", help="Actually perform registration writes (default: dry-run)") ap.add_argument("--json", action="store_true", help="Print raw JSON result") ap.add_argument("--timeout-seconds", type=int, default=45) return ap.parse_args() def build_flow_args(args: argparse.Namespace, execute: bool) -> dict: return { "eventId": args.event_id, "firstName": args.first_name, "lastName": args.last_name, "email": args.email, "phone": args.phone, "sourceId": args.source_id, "execute": bool(execute), "allSessions": bool(args.all_sessions), "explicitSessionGuids": args.session_guids, } def run_flow(page, args: argparse.Namespace, execute: bool) -> dict: target_url = RENATUS_HOME_URL page.goto(target_url, wait_until="domcontentloaded", timeout=args.timeout_seconds * 1000) try: page.wait_for_load_state("networkidle", timeout=5000) except Exception: pass time.sleep(2) return page.evaluate(JS_FLOW, build_flow_args(args, execute=execute)) def find_authenticated_page(browser, args: argparse.Namespace): diagnostics = [] if not browser.contexts: raise RuntimeError("No browser contexts found on attached CDP browser.") for context_index, context in enumerate(browser.contexts): page = context.new_page() try: result = run_flow(page, args, execute=False) diagnostics.append({ "contextIndex": context_index, "ok": bool(result.get("ok")), "stage": result.get("stage"), "eventStatus": result.get("eventStatus"), "currentUrl": result.get("currentUrl") or page.url, "title": result.get("title") or page.title(), }) if result.get("ok"): result["scanDiagnostics"] = diagnostics return page, result except Exception as e: diagnostics.append({ "contextIndex": context_index, "ok": False, "stage": "exception", "error": str(e), "currentUrl": page.url, }) try: page.close() except Exception: pass return None, { "ok": False, "stage": "auth_scan", "message": "No authenticated Renatus session found in any attached browser context.", "scanDiagnostics": diagnostics, } def human_print(result: dict) -> None: if not result.get("ok"): print("FAILED") print(f"stage: {result.get('stage')}") if result.get("message"): print(f"message: {result['message']}") if result.get("eventStatus") is not None: print(f"eventStatus: {result['eventStatus']}") if result.get("response"): print("response:") print(json.dumps(result["response"], indent=2)) if result.get("allEventSessions"): print("availableSessions:") for s in result["allEventSessions"]: print(f"- {s['sessionGuid']} | {s['name']}") return if result.get("stage") == "dry_run": print("DRY RUN") print(f"event: {result.get('eventName')} ({result.get('eventId')})") print(f"guest: {result['guest']['firstName']} {result['guest']['lastName']} <{result['guest']['email']}> {result['guest']['phone']}") print("selected sessions:") for s in result.get("selectedSessions", []): print(f"- {s['sessionGuid']} | {s['name']} | {s['startDate']}") if result.get("allEventSessions"): print("all event sessions:") for s in result["allEventSessions"]: reqs = [] if s.get("requireXtreamEducation"): reqs.append("RequireXtreamEducation") if s.get("requireActiveIMA"): reqs.append("RequireActiveIMA") req_str = f" | requirements={','.join(reqs)}" if reqs else "" print(f"- {s['sessionGuid']} | {s['name']} | {s['startDate']}{req_str}") return print("REGISTERED") print(f"event: {result.get('eventName')} ({result.get('eventId')})") print(f"guest lead id: {result['guest'].get('leadId')}") print(f"guest user id: {result['guest'].get('guestUserId')}") print(f"owner id: {result['guest'].get('ownerId')}") print(f"container id: {result['registration'].get('eventRegistrationContainerId')}") print(f"registration id: {result['registration'].get('guestRegistrationId')}") print("session ids:") for sid in result["registration"].get("sessionIds", []): print(f"- {sid}") def main() -> int: args = parse_args() if args.all_sessions and args.session_guids: print("Use either --all-sessions or --session-guid, not both.", file=sys.stderr) return 2 with sync_playwright() as p: browser = p.chromium.connect_over_cdp(args.cdp_url, timeout=args.timeout_seconds * 1000) page, result = find_authenticated_page(browser, args) if page is not None and args.execute: result = run_flow(page, args, execute=True) if args.json: print(json.dumps(result, indent=2)) else: human_print(result) try: if page is not None: page.close() except Exception: pass return 0 if result.get("ok") else 1 if __name__ == "__main__": raise SystemExit(main()) FILE:scripts/send_commercial_email_batches.py #!/usr/bin/env python3 """ send_commercial_email_batches.py Send commercial email to leads in batches with resume capability. Supports gws and gog CLI. Reads config from config.json (or env / defaults). Usage: # With config.json (auto-loaded from ../config.json or ./config.json) python3 scripts/send_commercial_email_batches.py --send # Override with CLI flags or env vars python3 scripts/send_commercial_email_batches.py --provider gog --account my@gmail.com --send # Dry run python3 scripts/send_commercial_email_batches.py --dry-run """ from __future__ import annotations import argparse import base64 import csv import json import os import subprocess import sys import time from datetime import datetime from pathlib import Path from shutil import which # ---------------------------------------------------------------------- # Config — import with graceful fallback # ---------------------------------------------------------------------- try: sys.path.insert(0, str(Path(__file__).parent)) from config_loader import load_config _cfg = load_config() except Exception: _cfg = {} WORKSPACE = Path(os.environ.get("WORKSPACE", "/home/umbrel/.openclaw/workspace")) DEFAULT_LOG = str(WORKSPACE / "logs/email_send_log.json") DEFAULT_CSV = str(WORKSPACE / "renatus_leads.csv") def _cfg_get(key: str, fallback: str = "") -> str: v = _cfg.get(key, "") if isinstance(_cfg, dict) else "" return v if v and v != "YOUR_" + key.partition("_")[2] else fallback def _default_provider() -> str: for exe in ("gws", "gog"): if which(exe): return exe return "gws" # ---------------------------------------------------------------------- # Helpers # ---------------------------------------------------------------------- def ensure_dir(path: str) -> None: Path(path).parent.mkdir(parents=True, exist_ok=True) def load_sent_log(log_path: str) -> set[str]: if not Path(log_path).exists(): return set() try: with open(log_path) as f: return set(json.load(f).get("sent", [])) except Exception: return set() def save_sent_log(log_path: str, sent_emails: set[str], stats: dict) -> None: data = { "sent": list(sent_emails), "stats": stats, "last_updated": datetime.now().isoformat(), } ensure_dir(log_path) with open(log_path, "w") as f: json.dump(data, f, indent=2) def load_template(template_path: str) -> str: p = Path(template_path) if p.exists(): return p.read_text(encoding="utf-8") # Try relative to this script local = Path(__file__).parent.parent / "assets" / "email" / "commercial-core-day1.html" if local.exists(): return local.read_text(encoding="utf-8") raise FileNotFoundError( f"Email template not found at {template_path}. " "Use --template or set RENATUS_TEMPLATE env var." ) def encode_unsubscribe_url(email: str, base_url: str = "https://YOUR_DOMAIN/unsubscribe.html") -> str: encoded = base64.urlsafe_b64encode(email.encode()).decode().rstrip("=") return f"{base_url}?e={encoded}" def personalize_email(html: str, email: str, base_url: str = "https://YOUR_DOMAIN/unsubscribe.html") -> str: unsub_url = encode_unsubscribe_url(email, base_url) html = html.replace("{{recipient_email}}", unsub_url) html = html.replace("{{unsubscribe_url}}", unsub_url) return html def get_leads(csv_path: str) -> list[dict]: leads = [] with open(csv_path, newline="", encoding="utf-8") as f: reader = csv.DictReader(f) for row in reader: if row.get("email") and "@" in row.get("email", ""): leads.append(row) return leads def send_gws(to: str, subject: str, html_body: str) -> tuple[bool, str]: cmd = ["gws", "gmail", "+send", "--to", to, "--subject", subject, "--body", html_body, "--html"] try: result = subprocess.run(cmd, capture_output=True, text=True, timeout=30) except subprocess.TimeoutExpired: return False, "Timeout (30s)" except FileNotFoundError: return False, "gws not found in PATH" except Exception as e: return False, str(e) if result.returncode == 0: return True, "" return False, (result.stderr or result.stdout or "Unknown error").strip() def send_gog(to: str, subject: str, html_body: str, account: str) -> tuple[bool, str]: cmd = ["gog", "gmail", "send", "--account", account, "--to", to, "--subject", subject, "--body-html", html_body] try: result = subprocess.run(cmd, capture_output=True, text=True, timeout=30) except subprocess.TimeoutExpired: return False, "Timeout (30s)" except FileNotFoundError: return False, "gog not found in PATH" except Exception as e: return False, str(e) if result.returncode == 0: return True, "" return False, (result.stderr or result.stdout or "Unknown error").strip() def send_email(provider: str, to: str, subject: str, html_body: str, account: str = "") -> tuple[bool, str]: if provider == "gog": if not account: return False, "gog requires --account" return send_gog(to, subject, html_body, account) return send_gws(to, subject, html_body) # ---------------------------------------------------------------------- # CLI # ---------------------------------------------------------------------- def parse_args() -> argparse.Namespace: ap = argparse.ArgumentParser( description="Send commercial email to leads in batches. " "Supports gws (default) and gog. Config loaded from config.json automatically." ) ap.add_argument("--csv", default=os.environ.get("RENATUS_LEADS_CSV", DEFAULT_CSV)) ap.add_argument("--template", default=os.environ.get("RENATUS_TEMPLATE", _cfg.get("template_path", "email/commercial-core-day1.html"))) ap.add_argument("--log", default=os.environ.get("EMAIL_SEND_LOG", DEFAULT_LOG)) ap.add_argument("--subject", default=os.environ.get("RENATUS_SUBJECT", _cfg.get("subject", "Free Real Estate Training"))) ap.add_argument("--unsub-url", dest="unsub_url", default=os.environ.get("RENATUS_UNSUB_URL", _cfg.get("unsub_url", "https://YOUR_DOMAIN/unsubscribe.html")), help="Base URL for unsubscribe link") ap.add_argument("--provider", choices=["gws", "gog", "auto"], default="auto") ap.add_argument("--account", default=os.environ.get("RENATUS_SENDER", _cfg.get("sender", "YOUR_SENDER@gmail.com")), help="Gmail account (for gog)") ap.add_argument("--batch-size", type=int, default=20) ap.add_argument("--start", type=int, default=0) ap.add_argument("--dry-run", action="store_true") ap.add_argument("--send", action="store_true", help="Actually send emails (without this flag, only dry-run preview runs)") ap.add_argument("--delay", type=float, default=2.0) ap.add_argument("--skip-sent", action="store_true", default=True) ap.add_argument("--no-skip-sent", dest="skip_sent", action="store_false") return ap.parse_args() def main() -> int: args = parse_args() if not args.send and not args.dry_run: print("No action. Use --dry-run to preview or --send to actually send.") return 1 if args.provider == "auto": provider = _default_provider() print(f"Auto-detected provider: {provider}") else: provider = args.provider if not which(provider): print(f"ERROR: {provider} not found. Install it or use --provider gws/gog.", file=sys.stderr) return 1 if not Path(args.csv).exists(): print(f"ERROR: CSV not found: {args.csv}", file=sys.stderr) print(" Run: python3 scripts/renatus_leads.py --export to download leads first.") return 1 try: template = load_template(args.template) except FileNotFoundError as e: print(f"ERROR: {e}", file=sys.stderr) return 1 leads = get_leads(args.csv) total_leads = len(leads) print(f"Provider: {provider}") print(f"Template: {args.template}") print(f"CSV: {args.csv} ({total_leads} leads)") sent_emails = load_sent_log(args.log) if args.skip_sent else set() if sent_emails: print(f"Already sent: {len(sent_emails)} (will skip)") batch = leads[args.start: args.start + args.batch_size] if not batch: print(f"No leads in batch starting at {args.start}") return 0 print(f"\nBatch: {args.start + 1}–{min(args.start + args.batch_size, total_leads)} of {total_leads}") print(f"Mode: {'DRY RUN' if args.dry_run else 'SEND'}") print("-" * 60) sent_count = failed_count = skipped_count = 0 for i, lead in enumerate(batch, args.start + 1): email = lead.get("email", "").strip() name = lead.get("name", "").strip() if not email or "@" not in email: print(f"[{i}/{total_leads}] SKIP (no valid email): {name}") skipped_count += 1 continue if email in sent_emails: print(f"[{i}/{total_leads}] ⏭ SKIP (already sent): {email}") skipped_count += 1 continue print(f"[{i}/{total_leads}] {'[DRY RUN] Would send to' if args.dry_run else 'Sending to'}: {name} <{email}>", end=" ") if args.dry_run: unsub_preview = encode_unsubscribe_url(email, args.unsub_url)[:60] + "..." print(f"\n subject: {args.subject}") print(f" unsub: {unsub_preview}") continue html = personalize_email(template, email, args.unsub_url) ok, err = send_email(provider, email, args.subject, html, args.account) if ok: print("✓") sent_emails.add(email) sent_count += 1 save_sent_log(args.log, sent_emails, { "total": total_leads, "sent": len(sent_emails), "last_batch_start": args.start, "provider": provider, "last_updated": datetime.now().isoformat(), }) else: print(f"✗ FAIL: {err[:80]}") failed_count += 1 if i < args.start + len(batch): time.sleep(args.delay) print("-" * 60) print(f"\nSent: {sent_count} Failed: {failed_count} Skipped: {skipped_count}") print(f"Total sent (all): {len(sent_emails)}/{total_leads}") next_start = args.start + args.batch_size if next_start < total_leads: print(f"\nNext: python3 scripts/send_commercial_email_batches.py --start {next_start} --batch-size {args.batch_size} --send") else: print("\n=== COMPLETE ===") return 0 if failed_count == 0 else 1 if __name__ == "__main__": raise SystemExit(main()) FILE:scripts/weekly_unsubscribe_sync.sh #!/bin/bash # # weekly_unsubscribe_sync.sh # # Automated weekly sync of unsubscribes from web form to Renatus. # Designed to run via cron (e.g., every Sunday at 2 AM). # # Requirements: # - Chrome/Brave running with --remote-debugging-port=9222 # - Logged into backoffice.myrenatus.com # - Python 3 + Playwright installed # set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" LOG_DIR="$SCRIPT_DIR/../logs" DATA_DIR="$SCRIPT_DIR/../data" DATE_STR=$(date +%Y%m%d_%H%M%S) LOG_FILE="$LOG_DIR/unsubscribe_sync_$DATE_STR.log" EMAILS_FILE="$DATA_DIR/unsubscribes_$DATE_STR.txt" REPORT_FILE="$DATA_DIR/unsubscribe_report_$DATE_STR.json" # Chrome CDP endpoint CDP_URL="${CDP_URL:-http://127.0.0.1:9222}" # Ensure directories exist mkdir -p "$LOG_DIR" "$DATA_DIR" log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE" } log "=== Weekly Unsubscribe Sync Started ===" log "Date: $DATE_STR" log "CDP URL: $CDP_URL" # Check if Chrome is accessible if ! curl -s "$CDP_URL/json/version" >/dev/null 2>&1; then log "ERROR: Cannot connect to Chrome at $CDP_URL" log "Make sure Chrome is running with: --remote-debugging-port=9222" exit 1 fi log "Chrome CDP endpoint is accessible" # Step 1: Extract unsubscribes from browser localStorage log "Step 1: Extracting unsubscribes from browser..." python3 << 'PYEOF' >"$EMAILS_FILE" 2>&1 import json import sys from playwright.sync_api import sync_playwright CDP_URL = "$CDP_URL" try: with sync_playwright() as p: browser = p.chromium.connect_over_cdp(CDP_URL, timeout=30000) # Try to get unsubscribes from first context emails = [] for context in browser.contexts: page = context.new_page() try: page.goto("https://YOUR_DOMAIN/unsubscribe.html", wait_until="domcontentloaded", timeout=10000) page.wait_for_timeout(1000) # Extract from localStorage result = page.evaluate(""" () => { try { return JSON.parse(localStorage.getItem('unsubscribed') || '[]'); } catch { return []; } } """) if isinstance(result, list): emails = result break except Exception as e: print(f"Error in context: {e}", file=sys.stderr) finally: try: page.close() except: pass # Also check all contexts for unsubscribed_leads key (edge function storage) for context in browser.contexts: for page in context.pages: try: result = page.evaluate(""" () => { const items = []; for (let i = 0; i < localStorage.length; i++) { const key = localStorage.key(i); if (key && key.includes('unsubscribe')) { items.push({key: key, value: localStorage.getItem(key)}); } } return items; } """) print(f"Found keys: {result}", file=sys.stderr) except: pass # Write emails one per line for email in emails: print(email) browser.close() except Exception as e: print(f"Failed to extract: {e}", file=sys.stderr) sys.exit(1) PYEOF EMAIL_COUNT=$(wc -l < "$EMAILS_FILE" | tr -d ' ') log "Found $EMAIL_COUNT unsubscribed email(s)" if [ "$EMAIL_COUNT" -eq 0 ]; then log "No unsubscribes to process. Exiting." exit 0 fi # Step 2: Check Renatus authentication log "Step 2: Verifying Renatus authentication..." python3 << 'PYEOF' import sys from playwright.sync_api import sync_playwright CDP_URL = "$CDP_URL" try: with sync_playwright() as p: browser = p.chromium.connect_over_cdp(CDP_URL, timeout=30000) for context in browser.contexts: page = context.new_page() try: page.goto("https://backoffice.myrenatus.com/Home/index", wait_until="domcontentloaded", timeout=15000) page.wait_for_timeout(2000) auth = page.evaluate(""" () => { const auth = localStorage.getItem('auth'); const xsrf = localStorage.getItem('__RequestVerificationToken'); return { hasAuth: !!auth, hasXsrf: !!xsrf }; } """) if auth.get('hasAuth') and auth.get('hasXsrf'): print("Auth OK") sys.exit(0) except: pass finally: try: page.close() except: pass print("No authenticated session found") sys.exit(1) except Exception as e: print(f"Error: {e}") sys.exit(1) PYEOF if [ $? -ne 0 ]; then log "ERROR: No authenticated Renatus session found" log "Make sure you're logged into backoffice.myrenatus.com" exit 1 fi log "Renatus authentication verified" # Step 3: Process unsubscribes log "Step 3: Processing unsubscribes in Renatus..." python3 "$SCRIPT_DIR/process_unsubscribes.py" \ --file "$EMAILS_FILE" \ --execute \ --cdp-url "$CDP_URL" \ 2>&1 | tee -a "$LOG_FILE" SYNC_STATUS=${PIPESTATUS[0]} # Step 4: Clear processed unsubscribes from localStorage if [ $SYNC_STATUS -eq 0 ]; then log "Step 4: Clearing processed unsubscribes from browser..." python3 << 'PYEOF' import sys from playwright.sync_api import sync_playwright CDP_URL = "$CDP_URL" EMAILS_FILE = "$EMAILS_FILE" try: # Read processed emails with open(EMAILS_FILE) as f: processed = set(line.strip() for line in f if line.strip()) with sync_playwright() as p: browser = p.chromium.connect_over_cdp(CDP_URL, timeout=30000) for context in browser.contexts: for page in context.pages: try: page.evaluate(""" (processed) => { try { const current = JSON.parse(localStorage.getItem('unsubscribed') || '[]'); const remaining = current.filter(e => !processed.includes(e)); localStorage.setItem('unsubscribed', JSON.stringify(remaining)); return { before: current.length, after: remaining.length }; } catch { return { error: true }; } } """, list(processed)) except: pass browser.close() print("Cleared processed unsubscribes") except Exception as e: print(f"Warning: Could not clear localStorage: {e}") sys.exit(0) # Non-fatal PYEOF fi # Step 5: Generate report log "Step 5: Generating report..." REPORT=$(cat << REPORTEOF { "sync_date": "$DATE_STR", "emails_processed": $EMAIL_COUNT, "status": "$([ $SYNC_STATUS -eq 0 ] && echo 'success' || echo 'failed')", "log_file": "$LOG_FILE", "emails_file": "$EMAILS_FILE" } REPORTEOF ) echo "$REPORT" > "$REPORT_FILE" log "=== Sync Complete ===" log "Report: $REPORT_FILE" log "Log: $LOG_FILE" # Optional: Send notification (uncomment if you want email/Slack alerts) # if [ $SYNC_STATUS -ne 0 ]; then # echo "Unsubscribe sync failed. Check $LOG_FILE" | mail -s "Unsubscribe Sync Failed" admin@example.com # fi exit $SYNC_STATUS

Renatus Icm

Comments (0)