@clawhub-captainsvbot-2041681aea
Autonomous GitHub Team — 41 AI agents that autonomously monitor a GitHub repository, detect bugs, create fixes, open PRs, and release to production. Triggers...
---
name: autonomous-github-team
description: >
Autonomous GitHub Team — 41 AI agents that autonomously monitor a GitHub repository, detect bugs,
create fixes, open PRs, and release to production. Triggers on: "run GitHub agents", "autonomous dev team",
"automated bug fixing", "AI code review", "GitHub workflow automation", "CI/CD automation".
⚠️ SECURITY: Requires a GitHub PAT with write access. Clones and executes third-party scripts.
See "Security Notes" below before installing.
metadata:
{
"openclaw":
{
"emoji": "🤖",
"primaryEnv": ["GH_TOKEN", "TARGET_REPO"],
"requires":
{
"bins": ["git", "bash", "curl", "python3"],
"env": ["GH_TOKEN", "TARGET_REPO"],
},
"install":
[
{
"id": "clone",
"kind": "clone",
"label": "Clone Autonomous GitHub Team repo",
"command": "git clone https://github.com/captainsvbot/AutonomousGitHubTeam.git /path/to/autonomous-github-team",
"pinnedRef": "v1.0.0",
"note": "Clone is pinned to a specific release tag. Always review the tagged commit before running.",
},
],
"permissions": ["github_repo_write", "github_pr_write", "github_issues_write"],
"risk_level": "high",
},
}
---
# 🤖 Autonomous GitHub Team Skill
> **⚠️ Security Warning — Read Before Installing**
>
> This skill clones and executes bash scripts from a remote repository. Before running:
> 1. **Audit the scripts first** — review every agent file in the `agents/` directory
> 2. **Use a least-privilege PAT** — dedicated token scoped to a single test repo, not your main account
> 3. **Never run on a production repo** until you've tested in an isolated fork
> 4. **Never commit tokens** — keep `config.env` private and out of version control
> 5. **Require human review** before merging any PRs the agents create
## What It Does
41 AI agents that autonomously monitor a GitHub repository, detect bugs, create fixes, open PRs, and release to production automatically.
## Required Environment Variables
```bash
GH_TOKEN # GitHub PAT — needs: contents:w, pr:w, issues:w
TARGET_REPO # The repository to operate on (format: owner/repo)
```
**Primary credential:** `GH_TOKEN` — treat this as a high-privilege secret.
## Setup
```bash
# 1. Clone the repo (pinned to v1.0.0 tag)
git clone --branch v1.0.0 https://github.com/captainsvbot/AutonomousGitHubTeam.git
cd autonomous-github-team
# 2. Configure — edit config.env
cp config.example.env config.env
nano config.env # set GH_TOKEN and TARGET_REPO
# 3. Audit the agents first (important!)
# Review agents/*.sh before running anything
# 4. Run
bash agents/orchestrator.sh
```
## Security Requirements for the GitHub Token
The skill needs a token that can:
- Read and write to repository contents
- Create and manage pull requests
- Read and write issues
**Create a dedicated token for this skill:**
1. GitHub → Settings → Developer Settings → Personal Access Tokens → Fine-grained tokens
2. Scope to **only the specific repository** you want the team to operate on
3. Grant only: `contents: read and write`, `pull requests: read and write`, `issues: read and write`
4. **Never** use your main account token with broad org access
## Available Agents
```bash
bash agents/orchestrator.sh # Full pipeline (detect → fix → release)
bash agents/security-agent.sh # Vulnerability scanning
bash agents/fixer-agent.sh # Apply fixes, open PRs
bash agents/git-guardian-agent.sh # Merge PRs (requires human review before main merges)
bash agents/backup-agent.sh # Backup repo via GitHub API
bash agents/rollback-agent.sh # Auto-revert broken commits
bash agents/team-bravo.sh # Activity monitor (read-only check)
```
## Gitflow
```
feature/fix → develop → main
```
Every run syncs `main → develop` first. The Git Guardian does not auto-merge to `main` — all merges to `main` require human approval.
## Before Running on a Production Repo
- [ ] Clone and review all `agents/*.sh` files
- [ ] Test in an isolated fork first
- [ ] Set up branch protection on `main` (require PR reviews)
- [ ] Use a dedicated, scope-limited PAT
- [ ] Keep `config.env` out of version control
## Security Checklist Before Publishing Changes
```bash
# Scan for accidentally committed secrets
grep -rni "gho_\|token\|secret\|api_key\|password" .
```
If you modify this repo, always scan before pushing publicly.
Generate tailored context windows with relevant code patterns, pitfalls, StackOverflow solutions, and local project context for AI coding tasks.
# Synthetic Context Generator (SCG)
Generate optimized context windows for AI coding tasks.
## Description
SCG takes a goal/task as input and returns a curated context window containing exactly what's needed:
- Relevant code patterns from knowledge base
- Common pitfalls and anti-patterns
- StackOverflow solutions
- Local project context
## Usage
```bash
# Generate context for a task
scg "write secure authentication middleware"
# Skip web search
scg "create React component" --no-web
# Custom token limit
scg "optimize database query" --max-tokens 2000
# Index a local project
scg index ./my-project
# Learn from a codebase
scg learn ./src
```
## Options
- `--context` - Include local project context (default: true)
- `--max-tokens` - Maximum tokens in output (default: 4000)
- `--no-web` - Skip StackOverflow search
- `--verbose` - Show debug info
## Examples
```bash
# Security-focused
scg "secure SQL query"
# → SQL injection patterns, parameterized queries, secrets handling
# React development
scg "create React component with hooks"
# → Custom hooks, useEffect best practices, testing patterns
# API development
scg "build REST API with Express"
# → Rate limiting, Helmet, JWT, error handling
```
## Triggers
- "get context for..."
- "best practices for..."
- "how to write..."
- "implement..."
- "create..."
FILE:README.md
# The Synthetic Context Generator ⚡
> Feed it a goal, get the perfect context window — nothing more, nothing less.
## The Problem
Your AI coding assistant needs context. But dumping entire repos or files creates context window bloat. You get irrelevant noise, hit token limits, and the AI misses what actually matters.
## The Solution
SCG is an intelligent context broker. You give it a goal, it finds exactly what you need:
```
$ scg "write secure SQL query"
⚠️ Common Pitfalls
- DON'T: Store passwords as plain text
- DON'T: Use string concatenation for SQL
✅ Recommended Patterns
- Use parameterized queries
- bcrypt for password hashing
```
## Installation
```bash
git clone https://github.com/captainsvbot/The-Synthetic-Context-Generator.git
cd The-Synthetic-Context-Generator
npm install
npm link # Make 'scg' available globally
```
## Usage
```bash
# Basic usage
scg "write secure authentication middleware"
# With project context
scg "fix CORS vulnerability" --context ./myproject
# Limit tokens
scg "optimize database queries" --max-tokens 2000
# Skip web search
scg "basic Express setup" --no-web
# Index a project for local search
scg index ./myproject
# Add to knowledge base
scg learn ./my-patterns.json
```
## Features
- **Goal Parser** — Extracts intent, language, security level from natural language
- **Local Search** — Indexes your projects for fuzzy search
- **Knowledge Base** — Curated patterns (security pitfalls, best practices)
- **Web Search** — Pulls relevant StackOverflow answers
- **Token Control** — Respects your context window limits
## Architecture
```
Goal: "write secure code"
↓
[Goal Parser] → intent: write, security
↓
[Parallel Search]
- Local files (fuzzy search)
- Knowledge base (curated)
- StackOverflow (API)
↓
[Context Composer] → formats & truncates
↓
Perfect Context Window ⚡
```
## Tech Stack
- Node.js CLI
- Fuse.js for fuzzy search
- tiktoken for token counting
- StackExchange API
## Roadmap
- [ ] Cursor/Rovio plugin for auto-inject
- [ ] Learn from user's codebase patterns
- [ ] Feedback loop (track what context helped)
- [ ] Vector embeddings for semantic search
- [ ] GitHub issues/PRs search
---
Built by CaptainSV ⚓
FILE:SPEC.md
# The Synthetic Context Generator (SCG)
**Goal:** Feed it a goal/task → get the perfect context window with exactly what's needed, nothing more.
## Core Concept
AI coding assistants suffer from context overflow. You dump entire repos or files when you only need a specific function. SCG acts as an intelligent context broker:
```
Goal: "write secure SQL query"
→ Search docs + examples + past vulnerabilities + StackOverflow
→ Inject only: the vulnerable patterns to avoid, secure alternatives, ORM best practices
```
## Architecture
### Components
1. **Goal Parser** — Extract intent, language, framework, security level from natural language
2. **Context Source Index** — Local docs, codebases, error logs, learned failures
3. **External Search** — StackOverflow, GitHub issues, framework docs (via API or scrape)
4. **Relevance Scorer** — Rank and weight findings by recency, success rate, similarity
5. **Context Composer** — Assemble final context window (truncate by token budget)
### Data Sources
| Source | Priority | Update Frequency |
|--------|----------|-------------------|
| Local project docs | High | On-demand |
| Code examples repo | High | Indexed at start |
| Past failures log | High | Every failure |
| StackOverflow | Medium | Cached results |
| Framework docs | Medium | Weekly refresh |
### CLI Interface
```bash
scg "write secure authentication middleware"
scg --goal "fix CORS vulnerability" --context ~/myproject --max-tokens 4000
scg --learn-from ./error-logs.json
```
### Output Format
```markdown
## Relevant Context (1,240 tokens)
### 📚 Documentation
- [Express.js Security](https://expressjs.com/en/advanced/best-practice-security.md)
- helmet.js configuration options
### ⚠️ Common Pitfalls
- DON'T: Store passwords as plain text (see: authservice.js:42)
- DON'T: Use eval() for dynamic permissions
### ✅ Recommended Patterns
- bcrypt.compare() for password verification
- JWT with RS256 for session tokens
### 🔧 StackOverflow Solutions
- [Best practice for JWT refresh tokens](https://stackoverflow.com/...) #847 votes
```
## Tech Stack
- **Runtime:** Node.js CLI
- **Search:** Local fuse.js for fuzzy + vector similarity (via embedding)
- **External:** StackExchange API, GitHub API
- **Storage:** SQLite for failure log + embeddings cache
- **Token counting:** tiktoken
## MVP Scope
1. Parse goal string → extract intent keywords
2. Search local files for matching patterns
3. Search a small "knowledge base" of examples
4. Output formatted context
5. `scg --learn` to add new examples to KB
## Future Ideas
- Cursor/Rovio plugin to auto-inject context
- Learn from user's codebase patterns
- Track what context actually helped (feedback loop)
FILE:package-lock.json
{
"name": "the-synthetic-context-generator",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "the-synthetic-context-generator",
"version": "1.0.0",
"license": "MIT",
"dependencies": {
"commander": "^11.1.0",
"dotenv": "^16.3.1",
"fuse.js": "^7.0.0",
"glob": "^10.3.10",
"tiktoken": "^1.0.14"
},
"bin": {
"scg": "src/cli.js"
}
},
"node_modules/@isaacs/cliui": {
"version": "8.0.2",
"resolved": "https://registry.npmjs.org/@isaacs/cliui/-/cliui-8.0.2.tgz",
"integrity": "sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==",
"license": "ISC",
"dependencies": {
"string-width": "^5.1.2",
"string-width-cjs": "npm:string-width@^4.2.0",
"strip-ansi": "^7.0.1",
"strip-ansi-cjs": "npm:strip-ansi@^6.0.1",
"wrap-ansi": "^8.1.0",
"wrap-ansi-cjs": "npm:wrap-ansi@^7.0.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/@pkgjs/parseargs": {
"version": "0.11.0",
"resolved": "https://registry.npmjs.org/@pkgjs/parseargs/-/parseargs-0.11.0.tgz",
"integrity": "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg==",
"license": "MIT",
"optional": true,
"engines": {
"node": ">=14"
}
},
"node_modules/ansi-regex": {
"version": "6.2.2",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-6.2.2.tgz",
"integrity": "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg==",
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/chalk/ansi-regex?sponsor=1"
}
},
"node_modules/ansi-styles": {
"version": "6.2.3",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-6.2.3.tgz",
"integrity": "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg==",
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/balanced-match": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
"license": "MIT"
},
"node_modules/brace-expansion": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
"license": "MIT",
"dependencies": {
"color-name": "~1.1.4"
},
"engines": {
"node": ">=7.0.0"
}
},
"node_modules/color-name": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"license": "MIT"
},
"node_modules/commander": {
"version": "11.1.0",
"resolved": "https://registry.npmjs.org/commander/-/commander-11.1.0.tgz",
"integrity": "sha512-yPVavfyCcRhmorC7rWlkHn15b4wDVgVmBA7kV4QVBsF7kv/9TKJAbAXVTxvTnwP8HHKjRCJDClKbciiYS7p0DQ==",
"license": "MIT",
"engines": {
"node": ">=16"
}
},
"node_modules/cross-spawn": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
"integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==",
"license": "MIT",
"dependencies": {
"path-key": "^3.1.0",
"shebang-command": "^2.0.0",
"which": "^2.0.1"
},
"engines": {
"node": ">= 8"
}
},
"node_modules/dotenv": {
"version": "16.6.1",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",
"integrity": "sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://dotenvx.com"
}
},
"node_modules/eastasianwidth": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/eastasianwidth/-/eastasianwidth-0.2.0.tgz",
"integrity": "sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA==",
"license": "MIT"
},
"node_modules/emoji-regex": {
"version": "9.2.2",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-9.2.2.tgz",
"integrity": "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg==",
"license": "MIT"
},
"node_modules/foreground-child": {
"version": "3.3.1",
"resolved": "https://registry.npmjs.org/foreground-child/-/foreground-child-3.3.1.tgz",
"integrity": "sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==",
"license": "ISC",
"dependencies": {
"cross-spawn": "^7.0.6",
"signal-exit": "^4.0.1"
},
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/fuse.js": {
"version": "7.1.0",
"resolved": "https://registry.npmjs.org/fuse.js/-/fuse.js-7.1.0.tgz",
"integrity": "sha512-trLf4SzuuUxfusZADLINj+dE8clK1frKdmqiJNb1Es75fmI5oY6X2mxLVUciLLjxqw/xr72Dhy+lER6dGd02FQ==",
"license": "Apache-2.0",
"engines": {
"node": ">=10"
}
},
"node_modules/glob": {
"version": "10.5.0",
"resolved": "https://registry.npmjs.org/glob/-/glob-10.5.0.tgz",
"integrity": "sha512-DfXN8DfhJ7NH3Oe7cFmu3NCu1wKbkReJ8TorzSAFbSKrlNaQSKfIzqYqVY8zlbs2NLBbWpRiU52GX2PbaBVNkg==",
"deprecated": "Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting [email protected]",
"license": "ISC",
"dependencies": {
"foreground-child": "^3.1.0",
"jackspeak": "^3.1.2",
"minimatch": "^9.0.4",
"minipass": "^7.1.2",
"package-json-from-dist": "^1.0.0",
"path-scurry": "^1.11.1"
},
"bin": {
"glob": "dist/esm/bin.mjs"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/is-fullwidth-code-point": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/isexe": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz",
"integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==",
"license": "ISC"
},
"node_modules/jackspeak": {
"version": "3.4.3",
"resolved": "https://registry.npmjs.org/jackspeak/-/jackspeak-3.4.3.tgz",
"integrity": "sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/cliui": "^8.0.2"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
},
"optionalDependencies": {
"@pkgjs/parseargs": "^0.11.0"
}
},
"node_modules/lru-cache": {
"version": "10.4.3",
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-10.4.3.tgz",
"integrity": "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ==",
"license": "ISC"
},
"node_modules/minimatch": {
"version": "9.0.9",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-9.0.9.tgz",
"integrity": "sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg==",
"license": "ISC",
"dependencies": {
"brace-expansion": "^2.0.2"
},
"engines": {
"node": ">=16 || 14 >=14.17"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/minipass": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=16 || 14 >=14.17"
}
},
"node_modules/package-json-from-dist": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/package-json-from-dist/-/package-json-from-dist-1.0.1.tgz",
"integrity": "sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw==",
"license": "BlueOak-1.0.0"
},
"node_modules/path-key": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz",
"integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/path-scurry": {
"version": "1.11.1",
"resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-1.11.1.tgz",
"integrity": "sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==",
"license": "BlueOak-1.0.0",
"dependencies": {
"lru-cache": "^10.2.0",
"minipass": "^5.0.0 || ^6.0.2 || ^7.0.0"
},
"engines": {
"node": ">=16 || 14 >=14.18"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/shebang-command": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
"integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==",
"license": "MIT",
"dependencies": {
"shebang-regex": "^3.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/shebang-regex": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz",
"integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/signal-exit": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-4.1.0.tgz",
"integrity": "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==",
"license": "ISC",
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/string-width": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-5.1.2.tgz",
"integrity": "sha512-HnLOCR3vjcY8beoNLtcjZ5/nxn2afmME6lhrDrebokqMap+XbeW8n9TXpPDOqdGK5qcI3oT0GKTW6wC7EMiVqA==",
"license": "MIT",
"dependencies": {
"eastasianwidth": "^0.2.0",
"emoji-regex": "^9.2.2",
"strip-ansi": "^7.0.1"
},
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/string-width-cjs": {
"name": "string-width",
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/string-width-cjs/node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/string-width-cjs/node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"license": "MIT"
},
"node_modules/string-width-cjs/node_modules/strip-ansi": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-ansi": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-7.2.0.tgz",
"integrity": "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w==",
"license": "MIT",
"dependencies": {
"ansi-regex": "^6.2.2"
},
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/chalk/strip-ansi?sponsor=1"
}
},
"node_modules/strip-ansi-cjs": {
"name": "strip-ansi",
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-ansi-cjs/node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/tiktoken": {
"version": "1.0.22",
"resolved": "https://registry.npmjs.org/tiktoken/-/tiktoken-1.0.22.tgz",
"integrity": "sha512-PKvy1rVF1RibfF3JlXBSP0Jrcw2uq3yXdgcEXtKTYn3QJ/cBRBHDnrJ5jHky+MENZ6DIPwNUGWpkVx+7joCpNA==",
"license": "MIT"
},
"node_modules/which": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
"integrity": "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==",
"license": "ISC",
"dependencies": {
"isexe": "^2.0.0"
},
"bin": {
"node-which": "bin/node-which"
},
"engines": {
"node": ">= 8"
}
},
"node_modules/wrap-ansi": {
"version": "8.1.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-8.1.0.tgz",
"integrity": "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ==",
"license": "MIT",
"dependencies": {
"ansi-styles": "^6.1.0",
"string-width": "^5.0.1",
"strip-ansi": "^7.0.1"
},
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/wrap-ansi-cjs": {
"name": "wrap-ansi",
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
"integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.0.0",
"string-width": "^4.1.0",
"strip-ansi": "^6.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/wrap-ansi-cjs/node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/wrap-ansi-cjs/node_modules/ansi-styles": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
"integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
"license": "MIT",
"dependencies": {
"color-convert": "^2.0.1"
},
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/wrap-ansi-cjs/node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"license": "MIT"
},
"node_modules/wrap-ansi-cjs/node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/wrap-ansi-cjs/node_modules/strip-ansi": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
}
}
}
FILE:package.json
{
"name": "the-synthetic-context-generator",
"version": "1.0.0",
"description": "Feed it a goal, get the perfect context window",
"type": "module",
"bin": {
"scg": "./src/cli.js"
},
"scripts": {
"start": "node src/cli.js",
"test": "echo \"No tests yet\" && exit 0"
},
"dependencies": {
"commander": "^11.1.0",
"fuse.js": "^7.0.0",
"tiktoken": "^1.0.14",
"glob": "^10.3.10",
"dotenv": "^16.3.1"
},
"keywords": ["ai", "context", "coding", "developer-tools"],
"license": "MIT"
}
FILE:src/cli.js
#!/usr/bin/env node
import { Command } from 'commander';
import { parseGoal } from './sources/goalParser.js';
import { searchLocal, indexProject } from './sources/localSearch.js';
import { searchKnowledgeBase } from './sources/knowledgeBase.js';
import { searchWeb } from './sources/webSearch.js';
import { composeContext } from './sources/contextComposer.js';
import { tokenCount } from './utils/tokenCounter.js';
const program = new Command();
program
.name('scg')
.description('The Synthetic Context Generator - Feed it a goal, get the perfect context')
.version('1.0.0');
program
.argument('[goal]', 'The goal or task you want context for')
.option('-c, --context <path>', 'Path to project/context files')
.option('-m, --max-tokens <number>', 'Max tokens in output', '4000')
.option('--no-web', 'Skip web searches')
.option('--verbose', 'Show verbose output')
.action(async (goal, options) => {
if (!goal) {
console.log('Usage: scg "write secure SQL query"');
console.log(' scg --goal "fix auth bug" --context ./myproject');
process.exit(0);
}
console.log(`🎯 Goal: goal\n`);
// Parse goal
const parsed = parseGoal(goal);
if (options.verbose) console.log('📊 Parsed:', parsed);
// Search local files if context path provided
let localResults = [];
if (options.context) {
localResults = await searchLocal(parsed, options.context);
}
// Search knowledge base
const kbResults = await searchKnowledgeBase(parsed);
// Search web if enabled
let webResults = [];
if (options.web && options.web !== false) {
webResults = await searchWeb(parsed);
}
// Compose final context
const maxTokens = parseInt(options.maxTokens || '4000');
const context = composeContext({
goal,
parsed,
local: localResults,
kb: kbResults,
web: webResults,
maxTokens
});
console.log(context);
const tokens = await tokenCount(context);
console.log(`\n📏 Context size: tokens tokens`);
});
// Index command
program
.command('index <path>')
.description('Index a project for local search')
.action(async (path) => {
console.log(`📂 Indexing path...`);
await indexProject(path);
console.log('✅ Indexed successfully!');
});
// Learn command
program
.command('learn <path>')
.description('Add files to knowledge base')
.action(async (path) => {
console.log(`📚 Learning from path...`);
// TODO: Implement learning
console.log('✅ Learned!');
});
program.parse();
FILE:src/sources/contextComposer.js
/**
* Context Composer - Assemble final context window
*/
import { truncateToTokens } from '../utils/tokenCounter.js';
export function composeContext({ goal, parsed, local, kb, web, maxTokens }) {
const sections = [];
// Header
sections.push(`## 🎯 Context for: "goal"\n`);
sections.push(`*Generated by SCG | Intents: parsed.intents.join(', ') || 'general' | Langs: parsed.languages.join(', ') || 'any'*\n`);
// Security warning if high security
if (parsed.securityLevel === 'high') {
sections.push(`⚠️ **SECURITY-CRITICAL**: Exercise extra caution. Validate all inputs, follow OWASP guidelines.\n`);
}
// 📚 Knowledge Base (curated patterns)
const pitfalls = kb.filter(i => i.category === 'common-pitfalls');
const patterns = kb.filter(i => i.category === 'recommended-patterns');
if (pitfalls.length) {
sections.push(`### ⚠️ Common Pitfalls`);
for (const p of pitfalls.slice(0, 4)) {
sections.push(`\n**p.title**`);
sections.push('```' + (p.languages?.includes('python') ? 'python' : 'javascript') + '\n' + p.content + '\n```');
}
sections.push('');
}
if (patterns.length) {
sections.push(`### ✅ Recommended Patterns`);
for (const p of patterns.slice(0, 4)) {
sections.push(`\n**p.title**`);
sections.push('```' + (p.languages?.includes('python') ? 'python' : 'javascript') + '\n' + p.content + '\n```');
}
sections.push('');
}
// 📂 Local Results
if (local.length) {
sections.push(`### 📂 From Your Project`);
for (const l of local.slice(0, 3)) {
const relevance = l.relevance === 'high' ? '🟢' : l.relevance === 'medium' ? '🟡' : '🔴';
sections.push(`\nrelevance **l.source:l.line**`);
sections.push('```');
sections.push(l.content.slice(0, 500));
sections.push('```');
}
sections.push('');
}
// 🌐 Web Results (StackOverflow)
if (web.length) {
sections.push(`### 🌐 StackOverflow Solutions`);
for (const w of web.slice(0, 3)) {
const votes = w.score > 50 ? '🔥' : w.score > 10 ? '⭐' : '📗';
const answered = w.isAnswered ? '✅' : '❓';
sections.push(`\nvotes answered [w.title](w.url)`);
sections.push(` Score: w.score | Answers: w.answerCount | Tags: w.tags?.slice(0, 3).join(', ')`);
}
sections.push('');
}
// Footer
sections.push(`\n---\n*Context truncated to ~maxTokens tokens*`);
let context = sections.join('\n');
// Truncate if needed
const estimatedTokens = Math.ceil(context.length / 4);
if (estimatedTokens > maxTokens) {
context = truncateToTokens(context, maxTokens);
}
return context;
}
FILE:src/sources/goalParser.js
/**
* Goal Parser - Extract intent and keywords from natural language goals
*/
export function parseGoal(goal) {
const lower = goal.toLowerCase();
// Intent detection
const intents = [];
if (lower.includes('write') || lower.includes('create') || lower.includes('implement') || lower.includes('build') || lower.includes('make') || lower.includes('add') || lower.includes('setup')) {
intents.push('write');
}
if (lower.includes('fix') || lower.includes('bug') || lower.includes('error') || lower.includes('repair') || lower.includes('resolve')) {
intents.push('fix');
}
if (lower.includes('secure') || lower.includes('vulnerability') || lower.includes('sanitize')) {
intents.push('security');
}
if (lower.includes('optimize') || lower.includes('performance') || lower.includes('speed')) {
intents.push('optimize');
}
if (lower.includes('test') || lower.includes('spec')) {
intents.push('test');
}
if (lower.includes('refactor') || lower.includes('cleanup')) {
intents.push('refactor');
}
if (lower.includes('debug') || lower.includes('troubleshoot')) {
intents.push('debug');
}
// Default to 'write' if nothing detected
if (intents.length === 0) {
intents.push('write');
}
// Language/framework detection
const languages = [];
const langPatterns = {
'javascript': /\b(js|javascript|node|nodejs|express|jwt|react|vue|angular)\b/,
'typescript': /\b(ts|typescript)\b/,
'python': /\b(python|py|django|flask|fastapi)\b/,
'rust': /\b(rust|rs)\b/,
'go': /\b(go|golang)\b/,
'java': /\b(java|spring)\b/,
'c#': /\b(c#|csharp|\.net|asp\.net)\b/,
'sql': /\b(sql|mysql|postgres|postgresql|mongodb|redis)\b/,
'html': /\b(html|css|frontend)\b/,
'bash': /\b(bash|shell|sh|script|dockerfile|docker)\b/,
'docker': /\b(docker|dockerfile|container)\b/,
};
for (const [lang, pattern] of Object.entries(langPatterns)) {
if (pattern.test(lower)) {
languages.push(lang);
}
}
// Security level
let securityLevel = 'normal';
if (lower.includes('secure') || lower.includes('vulnerability') || lower.includes('exploit')) {
securityLevel = 'high';
} else if (lower.includes('public') || lower.includes('api')) {
securityLevel = 'medium';
}
// Extract keywords (simple extraction)
const stopWords = ['write', 'create', 'implement', 'fix', 'error', 'with', 'the', 'this', 'that', 'from', 'have', 'some'];
const keywords = goal
.split(/\s+/)
.filter(w => w.length > 2 && !stopWords.includes(w.toLowerCase()))
.slice(0, 10);
// Always include intents as keywords for pattern matching
const allKeywords = [...new Set([...intents, ...keywords])];
return {
original: goal,
intents,
languages,
securityLevel,
keywords: allKeywords,
timestamp: new Date().toISOString()
};
}
FILE:src/sources/knowledgeBase.js
/**
* Knowledge Base - Curated examples and patterns
*/
// Built-in knowledge base
const knowledgeBase = [
// Security patterns
{
intent: 'security',
languages: ['javascript', 'typescript'],
category: 'common-pitfalls',
title: 'DON\'T: Plain text passwords',
content: `// ❌ NEVER store passwords as plain text
const user = { password: inputPassword }; // Vulnerable!
// ✅ ALWAYS hash passwords
import bcrypt from 'bcrypt';
const saltRounds = 12;
const hashedPassword = await bcrypt.hash(inputPassword, saltRounds);
const isValid = await bcrypt.compare(inputPassword, hashedPassword);`,
tags: ['auth', 'password', 'security', 'bcrypt'],
votes: 950
},
{
intent: 'security',
languages: ['javascript', 'typescript'],
category: 'common-pitfalls',
title: 'DON\'T: SQL injection via string concatenation',
content: `// ❌ NEVER build SQL with string concatenation
const query = "SELECT * FROM users WHERE id = " + userId; // SQL Injection!
// ✅ Use parameterized queries
const query = "SELECT * FROM users WHERE id = ?";
db.query(query, [userId]);`,
tags: ['sql', 'injection', 'security', 'database'],
votes: 920
},
{
intent: 'security',
languages: ['javascript', 'typescript'],
category: 'common-pitfalls',
title: 'DON\'T: Use eval() for dynamic code',
content: `// ❌ NEVER use eval() or new Function()
const perms = eval(userInput); // Code injection!
// ✅ Use safe alternatives
const actions = {
read: () => doRead(),
write: () => doWrite()
};
actions[userInput]?.();`,
tags: ['eval', 'injection', 'security'],
votes: 880
},
{
intent: 'security',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ JWT best practices',
content: `// ✅ JWT Security Checklist:
// 1. Use RS256 (asymmetric) not HS256
// 2. Set short expiration (15min access, 7d refresh)
// 3. Store refresh token in httpOnly cookie
// 4. Validate issuer (iss) and audience (aud)
// 5. Blacklist tokens on logout
import jwt from 'jsonwebtoken';
const token = jwt.sign(payload, privateKey, {
algorithm: 'RS256',
expiresIn: '15m',
issuer: 'your-app'
});`,
tags: ['jwt', 'auth', 'token', 'security'],
votes: 850
},
{
intent: 'security',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Helmet.js for Express',
content: `// ✅ Use Helmet to set secure HTTP headers
import helmet from 'helmet';
app.use(helmet());
// Fine-tune specific headers
app.use(helmet.contentSecurityPolicy({
directives: {
defaultSrc: ["'self'"],
scriptSrc: ["'self'", "'unsafe-inline'"],
styleSrc: ["'self'", "'unsafe-inline'"],
imgSrc: ["'self'", "data:", "https:"],
}
}));`,
tags: ['express', 'helmet', 'headers', 'security'],
votes: 720
},
// General JavaScript patterns
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Async/await error handling',
content: `// ✅ Proper async error handling
async function fetchData() {
try {
const response = await fetch(url);
if (!response.ok) {
throw new Error(\`HTTP \response.status\`);
}
return await response.json();
} catch (error) {
console.error('Fetch failed:', error);
throw error; // Re-throw or handle
}
}`,
tags: ['async', 'error-handling', 'promise'],
votes: 810
},
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Environment config pattern',
content: `// ✅ Use environment variables with defaults
import dotenv from 'dotenv';
dotenv.config();
export const config = {
db: {
host: process.env.DB_HOST || 'localhost',
port: parseInt(process.env.DB_PORT || '5432'),
name: process.env.DB_NAME || 'app_dev',
},
jwt: {
secret: process.env.JWT_SECRET, // Required - fail if missing
expiresIn: process.env.JWT_EXPIRES_IN || '15m',
}
};
// Validate required vars at startup
if (!process.env.JWT_SECRET) {
throw new Error('JWT_SECRET is required');
}`,
tags: ['config', 'environment', 'dotenv'],
votes: 780
},
// Node.js/Express patterns
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Express route handler structure',
content: `// ✅ Clean Express route handler
import { Router } from 'express';
const router = Router();
// Controller functions separated from routes
const userController = {
async getUser(req, res, next) {
try {
const user = await User.findById(req.params.id);
if (!user) {
return res.status(404).json({ error: 'User not found' });
}
res.json(user);
} catch (error) {
next(error); // Pass to error middleware
}
}
};
router.get('/:id', userController.getUser);
export default router;`,
tags: ['express', 'router', 'structure', 'clean-code'],
votes: 750
},
{
intent: 'fix',
languages: ['javascript', 'typescript'],
category: 'common-pitfalls',
title: 'DON\'T: Mutation in React state',
content: `// ❌ NEVER mutate state directly
setUser({ ...user });
user.name = newName; // Won't trigger re-render!
// ✅ Use immutable patterns
setUser(prev => ({
...prev,
name: newName
}));
// Or with Immer
setUser(draft => {
draft.name = newName;
});`,
tags: ['react', 'state', 'mutation', 'immutable'],
votes: 840
},
{
intent: 'optimize',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Debounce and throttle utilities',
content: `// ✅ Debounce - wait for idle
function debounce(fn, delay) {
let timeoutId;
return (...args) => {
clearTimeout(timeoutId);
timeoutId = setTimeout(() => fn(...args), delay);
};
}
// ✅ Throttle - limit frequency
function throttle(fn, limit) {
let inThrottle;
return (...args) => {
if (!inThrottle) {
fn(...args);
inThrottle = true;
setTimeout(() => inThrottle = false, limit);
}
};
}
// Usage
const handleSearch = debounce(async (query) => {
const results = await search(query);
setResults(results);
}, 300);`,
tags: ['performance', 'debounce', 'throttle', 'optimize'],
votes: 820
},
// Python patterns
{
intent: 'write',
languages: ['python'],
category: 'recommended-patterns',
title: '✅ Python type hints',
content: `from typing import Optional, List
def get_user(user_id: int) -> Optional[dict]:
"""Get user by ID with type hints."""
user = db.query("SELECT * FROM users WHERE id = ?", user_id)
return user if user else None
def process_items(items: List[str]) -> List[str]:
"""Process a list of items."""
return [item.strip().lower() for item in items]`,
types: ['python', 'typing', 'type-hints'],
votes: 700
},
{
intent: 'security',
languages: ['python'],
category: 'recommended-patterns',
title: '✅ Python SQL injection prevention',
content: `# ✅ Use parameterized queries in Python
cursor.execute(
"SELECT * FROM users WHERE id = %s", # Note: %s not f-string!
(user_id,)
)
# ✅ Or use SQLAlchemy ORM
user = session.query(User).filter(User.id == user_id).first()
# ❌ NEVER do this:
cursor.execute(f"SELECT * FROM users WHERE id = {user_id}")`,
tags: ['python', 'sql', 'injection', 'sqlalchemy'],
votes: 850
},
// API design
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ RESTful API response format',
content: `// ✅ Consistent API response format
function successResponse(data, meta = {}) {
return {
success: true,
data,
meta: { timestamp: new Date().toISOString(), ...meta }
};
}
function errorResponse(message, code = 'ERROR', status = 400) {
return {
success: false,
error: { message, code, status }
};
}
// Usage
app.get('/api/users', (req, res) => {
const users = User.findAll();
res.json(successResponse(users, { total: users.length }));
});`,
tags: ['api', 'rest', 'response-format'],
votes: 680
},
// Docker patterns
{
intent: 'write',
languages: ['bash'],
category: 'recommended-patterns',
title: '✅ Docker multi-stage build for Node.js',
content: `# ✅ Multi-stage build for smaller images
# Stage 1: Build
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
# Stage 2: Production
FROM node:20-alpine AS production
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./
USER node
EXPOSE 3000
CMD ["node", "dist/index.js"]`,
tags: ['docker', 'node', 'container', 'build'],
votes: 800
},
{
intent: 'write',
languages: ['bash'],
category: 'common-pitfalls',
title: "DON'T: Run containers as root",
content: `# ❌ NEVER run as root in production
FROM node:20
WORKDIR /app
COPY . .
RUN npm install
CMD ["node", "index.js"]
# ✅ Always create and use non-root user
FROM node:20-alpine
WORKDIR /app
COPY --chown=node:node . .
USER node
CMD ["node", "index.js"]`,
tags: ['docker', 'security', 'root'],
votes: 750
},
// Git patterns
{
intent: 'write',
languages: ['bash'],
category: 'recommended-patterns',
title: '✅ Conventional commits',
content: `# ✅ Use conventional commits for clear changelog
feat: add user authentication
fix: resolve CORS issue in API
docs: update README with installation
style: format code with prettier
refactor: simplify error handling
test: add unit tests for auth controller
chore: update dependencies
# ✅ Use git hooks to enforce
npx husky add .husky/commit-msg 'npx commitlint --edit "$1"'`,
tags: ['git', 'commit', 'conventional'],
votes: 700
},
// Testing patterns
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Jest unit test structure',
content: `// ✅ Clean Jest test structure
describe('UserService', () => {
let userService;
let mockDb;
beforeEach(() => {
mockDb = { findById: jest.fn() };
userService = new UserService(mockDb);
});
describe('getUser', () => {
it('should return user when found', async () => {
const mockUser = { id: 1, name: 'John' };
mockDb.findById.mockResolvedValue(mockUser);
const result = await userService.getUser(1);
expect(result).toEqual(mockUser);
expect(mockDb.findById).toHaveBeenCalledWith(1);
});
it('should throw NotFoundError when user missing', async () => {
mockDb.findById.mockResolvedValue(null);
await expect(userService.getUser(999)).rejects.toThrow(NotFoundError);
});
});
});`,
tags: ['jest', 'test', 'unit-test', 'tdd'],
votes: 820
},
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ React Testing Library best practices',
content: `// ✅ Test behavior, not implementation
import { render, screen, fireEvent } from '@testing-library/react';
test('should login user with valid credentials', async () => {
render(<Login />);
// Query by accessible name (label, placeholder, aria-label)
const emailInput = screen.getByLabelText(/email/i);
const passwordInput = screen.getByLabelText(/password/i);
const submitButton = screen.getByRole('button', { name: /submit/i });
fireEvent.change(emailInput, { target: { value: '[email protected]' } });
fireEvent.change(passwordInput, { target: { value: 'password123' } });
fireEvent.click(submitButton);
// Assert the outcome, not the implementation
await screen.findByText(/welcome/i);
});`,
tags: ['react', 'testing-library', 'jest', 'test'],
votes: 780
},
// TypeScript patterns
{
intent: 'write',
languages: ['typescript'],
category: 'recommended-patterns',
title: '✅ TypeScript strict null handling',
content: `// ✅ Use strict null checks
interface User {
id: number;
name: string;
email?: string; // Optional
}
function getEmail(user: User): string {
// ❌ BAD: user.email could be undefined!
return user.email.toLowerCase();
// ✅ GOOD: Handle undefined
return user.email?.toLowerCase() ?? 'no email';
}
// ✅ Use type guards
function isUser(obj: unknown): obj is User {
return typeof obj === 'object' && obj !== null && 'id' in obj;
}
function process(obj: unknown) {
if (isUser(obj)) {
console.log(obj.name); // TypeScript knows it's User
}
}`,
tags: ['typescript', 'null', 'undefined', 'type-guard'],
votes: 750
},
{
intent: 'write',
languages: ['typescript'],
category: 'recommended-patterns',
title: '✅ TypeScript discriminated unions',
content: `// ✅ Use discriminated unions for type-safe state
type RequestState<T> =
| { status: 'idle' }
| { status: 'loading' }
| { status: 'success'; data: T }
| { status: 'error'; error: Error };
function handleState(state: RequestState<User>) {
switch (state.status) {
case 'idle':
return 'Start loading';
case 'loading':
return 'Loading...';
case 'success':
return state.data.name; // TypeScript knows data exists
case 'error':
return state.error.message; // TypeScript knows error exists
}
}
// ✅ Exhaustiveness check
function exhaustive(state: RequestState<User>) {
// TypeScript errors if we miss a case
const _exhaustive: never = state;
}`,
tags: ['typescript', 'union', 'state', 'pattern'],
votes: 720
},
// Error handling
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Custom error classes',
content: `// ✅ Create custom errors for better error handling
class AppError extends Error {
constructor(
message: string,
public statusCode: number = 500,
public code: string = 'INTERNAL_ERROR'
) {
super(message);
this.name = this.constructor.name;
Error.captureStackTrace(this, this.constructor);
}
}
class NotFoundError extends AppError {
constructor(resource: string) {
super(\`\resource not found\`, 404, 'NOT_FOUND');
}
}
class ValidationError extends AppError {
constructor(message: string) {
super(message, 400, 'VALIDATION_ERROR');
}
}
// Usage
throw new NotFoundError('User');`,
tags: ['error', 'exception', 'class', 'handling'],
votes: 710
},
// Redis patterns
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Redis caching with TTL',
content: `// ✅ Cache with expiration
const CACHE_TTL = 300; // 5 minutes
async function getCachedUser(userId: number) {
const cacheKey = \`user:\userId\`;
// Try cache first
const cached = await redis.get(cacheKey);
if (cached) {
return JSON.parse(cached);
}
// Fetch from DB
const user = await db.findUser(userId);
// Store in cache with TTL
if (user) {
await redis.setex(cacheKey, CACHE_TTL, JSON.stringify(user));
}
return user;
}
// ✅ Invalidate cache on update
async function updateUser(userId: number, data: UserData) {
const user = await db.updateUser(userId, data);
await redis.del(\`user:\userId\`); // Invalidate
return user;
}`,
tags: ['redis', 'cache', 'performance'],
votes: 680
},
// Rate limiting
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Express rate limiting',
content: `// ✅ Rate limiting in Express
import rateLimit from 'express-rate-limit';
// General API limit
const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per window
message: { error: 'Too many requests, please try again later' },
standardHeaders: true,
legacyHeaders: false,
});
// Strict limit for auth endpoints
const authLimiter = rateLimit({
windowMs: 15 * 60 * 1000,
max: 5, // Only 5 attempts
skipSuccessfulRequests: true,
});
app.use('/api/', apiLimiter);
app.use('/api/auth/login', authLimiter);`,
tags: ['express', 'rate-limit', 'security', 'ddos'],
votes: 740
},
// Input validation
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Zod schema validation',
content: `// ✅ Use Zod for runtime validation
import { z } from 'zod';
const UserSchema = z.object({
email: z.string().email(),
password: z.string().min(8).max(100),
age: z.number().int().min(13).max(120).optional(),
role: z.enum(['user', 'admin', 'moderator']),
});
type User = z.infer<typeof UserSchema>;
// Validate incoming request
function validateUser(data: unknown): User {
return UserSchema.parse(data);
}
// Express middleware with Zod
import { zodMiddleware } from './middleware';
app.post('/users', zodMiddleware(UserSchema), createUser);`,
tags: ['validation', 'zod', 'schema', 'runtime'],
votes: 760
},
// Environment variables
{
intent: 'write',
languages: ['javascript', 'typescript', 'python'],
category: 'common-pitfalls',
title: "DON'T: Secret keys in source code",
content: `// ❌ NEVER hardcode secrets
const apiKey = 'sk_live_123456789'; // Exposed in git!
const dbPassword = 'mysecretpass';
// ✅ Use environment variables
const apiKey = process.env.API_KEY;
const dbPassword = process.env.DB_PASSWORD;
if (!apiKey) {
throw new Error('API_KEY is required');
}
// ✅ Use .env file (add to .gitignore)
// .env:
// API_KEY=sk_live_123456789
// DB_PASSWORD=secret
// ✅ Validate at startup
const config = {
apiKey: z.string().parse(process.env.API_KEY),
port: z.number().default(3000).parse(process.env.PORT),
};`,
tags: ['security', 'env', 'secrets', 'configuration'],
votes: 890
},
// Logging
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Structured logging with Pino',
content: `// ✅ Use Pino for structured JSON logging
import pino from 'pino';
const logger = pino({
level: process.env.LOG_LEVEL || 'info',
formatters: {
level: (label) => ({ level: label }),
},
});
// ✅ Log with context
logger.info({ userId: user.id, action: 'login' }, 'User logged in');
logger.error({ err: error, userId: user.id }, 'Login failed');
// ✅ Child loggers for request context
const childLogger = logger.child({ requestId: req.id });
childLogger.info('Processing request');
// ✅ Avoid string interpolation
// ❌ BAD: logger.info('User ' + user.id + ' logged in');
// ✅ GOOD: logger.info({ userId: user.id }, 'User logged in');`,
tags: ['logging', 'pino', 'structured', 'monitoring'],
votes: 650
},
// React patterns
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ React custom hooks',
content: `// ✅ Extract logic into custom hooks
function useUser(userId: number) {
const [user, setUser] = useState<User | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<Error | null>(null);
useEffect(() => {
setLoading(true);
fetchUser(userId)
.then(setUser)
.catch(setError)
.finally(() => setLoading(false));
}, [userId]);
return { user, loading, error };
}
// Usage in component
function UserProfile({ userId }: { userId: number }) {
const { user, loading, error } = useUser(userId);
if (loading) return <Spinner />;
if (error) return <Error error={error} />;
return <div>{user.name}</div>;
}`,
tags: ['react', 'hooks', 'custom', 'reuse'],
votes: 800
},
// File handling
{
intent: 'write',
languages: ['python'],
category: 'common-pitfalls',
title: "DON'T: Leave files open",
content: `# ❌ BAD: May leave file open on error
f = open('file.txt', 'r')
data = f.read()
f.close()
# ✅ GOOD: Use context manager
with open('file.txt', 'r') as f:
data = f.read()
# File automatically closed
# ✅ BEST: Use pathlib
from pathlib import Path
content = Path('file.txt').read_text()
# Write with encoding
Path('output.txt').write_text(data, encoding='utf-8')`,
tags: ['python', 'file', 'io', 'context-manager'],
votes: 720
},
// SQL/ORM patterns
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ Prisma query patterns',
content: `// ✅ Prisma - always include select/where
const user = await prisma.user.findUnique({
where: { id: userId },
select: { id: true, email: true, name: true } // Only what you need
});
// ✅ Pagination
const page = 1;
const limit = 10;
const users = await prisma.user.findMany({
skip: (page - 1) * limit,
take: limit,
orderBy: { createdAt: 'desc' }
});
// ✅ Transaction for related writes
await prisma.$transaction([
prisma.order.create({ data: orderData }),
prisma.inventory.update({ where: { id: itemId }, data: { stock: { decrement: 1 } } })
]);`,
tags: ['prisma', 'orm', 'database', 'query'],
votes: 700
},
// WebSocket
{
intent: 'write',
languages: ['javascript', 'typescript'],
category: 'recommended-patterns',
title: '✅ WebSocket with reconnection',
content: `// ✅ WebSocket with auto-reconnection
class WSClient {
constructor(url) {
this.url = url;
this.connect();
}
connect() {
this.ws = new WebSocket(this.url);
this.ws.onopen = () => console.log('Connected');
this.ws.onclose = () => this.reconnect();
this.ws.onerror = (err) => console.error('WS Error:', err);
}
reconnect() {
setTimeout(() => {
console.log('Reconnecting...');
this.connect();
}, 1000);
}
send(data) {
if (this.ws.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(data));
}
}
}`,
tags: ['websocket', 'reconnection', 'realtime'],
votes: 650
}
];
let customKB = [];
// Security-related keywords that should boost security patterns
const SECURITY_KEYWORDS = ['auth', 'jwt', 'token', 'password', 'login', 'security', 'secure', 'credential', 'session', 'oauth', 'access', 'permission', 'role'];
/**
* Search the knowledge base
*/
export function searchKnowledgeBase(parsed) {
const results = [];
// Check for security keywords in the parsed query
const hasSecurityKeyword = parsed.keywords.some(k =>
SECURITY_KEYWORDS.some(sk => k.toLowerCase().includes(sk))
);
const intentMatch = (item) => {
if (!parsed.intents.length) return true;
return parsed.intents.some(i => item.intent === i || item.content.toLowerCase().includes(i));
};
const langMatch = (item) => {
if (!parsed.languages.length) return true;
return parsed.languages.some(l => item.languages?.includes(l));
};
const keywordMatch = (item) => {
if (!parsed.keywords.length) return true;
return parsed.keywords.some(k =>
item.title.toLowerCase().includes(k) ||
item.content.toLowerCase().includes(k) ||
item.tags?.some(t => t.includes(k))
);
};
const allItems = [...knowledgeBase, ...customKB];
for (const item of allItems) {
let score = 0;
// Intent match (highest weight)
if (parsed.intents.some(i => item.intent === i)) score += 5;
else if (intentMatch(item)) score += 1;
// Language match - stronger weight for language-specific patterns
if (parsed.languages.length && langMatch(item)) score += 5;
// Penalize items with language restrictions that don't match
else if (item.languages?.length && !langMatch(item)) score -= 3;
// Keyword match in title/content/tags
const km = keywordMatch(item);
if (km) score += 3;
// Security items get bonus for security keywords in query
if (hasSecurityKeyword && (item.category === 'common-pitfalls' || item.tags?.some(t => SECURITY_KEYWORDS.includes(t)))) {
score += 5;
}
// Security items get bonus for high security level
if (parsed.securityLevel === 'high' && item.category === 'common-pitfalls') {
score += 4;
}
if (score > 0) {
results.push({
...item,
score,
relevance: score > 8 ? 'high' : score > 4 ? 'medium' : 'low'
});
}
}
// Sort by score, then by votes
results.sort((a, b) => b.score - a.score || (b.votes || 0) - (a.votes || 0));
return results.slice(0, 8);
}
/**
* Add items to custom knowledge base
*/
export function addToKnowledgeBase(items) {
customKB.push(...items);
console.log(`📚 Added items.length items to knowledge base`);
}
FILE:src/sources/localSearch.js
/**
* Local Search - Search local files and project context
*/
import fs from 'fs';
import path from 'path';
import { glob } from 'glob';
import Fuse from 'fuse.js';
let fileIndex = [];
let fuse = null;
export async function indexProject(projectPath) {
const patterns = [
'**/*.js',
'**/*.ts',
'**/*.jsx',
'**/*.tsx',
'**/*.py',
'**/*.rs',
'**/*.go',
'**/*.java',
'**/*.md',
'**/*.json'
];
fileIndex = [];
for (const pattern of patterns) {
const files = await glob(pattern, {
cwd: projectPath,
ignore: ['node_modules/**', '.git/**', 'dist/**', 'build/**', '__pycache__/**']
});
for (const file of files) {
const fullPath = path.join(projectPath, file);
try {
const content = fs.readFileSync(fullPath, 'utf-8');
const lines = content.split('\n');
// Index in chunks for better search
const chunkSize = 50;
for (let i = 0; i < lines.length; i += chunkSize) {
const chunk = lines.slice(i, i + chunkSize).join('\n');
fileIndex.push({
file,
path: fullPath,
chunk: i,
content: chunk,
preview: chunk.slice(0, 200)
});
}
} catch (e) {
// Skip binary or unreadable files
}
}
}
// Setup fuzzy search
fuse = new Fuse(fileIndex, {
keys: ['content', 'file'],
threshold: 0.4,
includeScore: true,
minMatchCharLength: 3
});
console.log(`📂 Indexed fileIndex.length chunks from projectPath`);
}
export function searchLocal(parsed, projectPath) {
if (!fuse && projectPath) {
// Auto-index if not done
indexProject(projectPath);
}
if (!fuse) return [];
const query = [
...parsed.intents,
...parsed.languages,
...parsed.keywords
].join(' ');
const results = fuse.search(query, { limit: 10 });
return results.map(r => ({
type: 'local',
source: r.item.file,
line: r.item.chunk * 50,
content: r.item.content,
score: 1 - r.item.score,
relevance: r.item.score < 0.3 ? 'high' : r.item.score < 0.6 ? 'medium' : 'low'
}));
}
FILE:src/sources/webSearch.js
/**
* Web Search - StackOverflow and GitHub search
*/
// StackExchange API (no key needed for basic search)
const STACKOVERFLOW_API = 'https://api.stackexchange.com/2.3/search/advanced';
export async function searchWeb(parsed) {
const results = [];
// Build query from intent + keywords
const query = [
...parsed.intents,
...parsed.languages.slice(0, 2),
...parsed.keywords.slice(0, 3)
].join(' ');
if (!query.trim()) return results;
try {
// Search StackOverflow
const soParams = new URLSearchParams({
order: 'desc',
sort: 'relevance',
q: query,
site: 'stackoverflow',
answers: 'true',
filter: '!nNPvSNVZJS'
});
const soResponse = await fetch(`STACKOVERFLOW_API?soParams`);
const soData = await soResponse.json();
if (soData.items) {
for (const item of soData.items.slice(0, 5)) {
results.push({
type: 'stackoverflow',
title: item.title,
url: item.link,
score: item.score,
answerCount: item.answer_count,
tags: item.tags,
isAnswered: item.is_answered,
relevance: item.score > 10 ? 'high' : item.score > 3 ? 'medium' : 'low'
});
}
}
} catch (error) {
console.error('StackOverflow search failed:', error.message);
}
// Could add GitHub search here too
return results;
}
FILE:src/utils/tokenCounter.js
/**
* Token Counter - Estimate token counts
*/
import { encoding_for_model } from 'tiktoken';
let encoder = null;
async function getEncoder() {
if (!encoder) {
try {
encoder = await encoding_for_model('gpt-4');
} catch {
// Fallback to cl100k_base
encoder = await import('tiktoken').then(m => m.get_encoding('cl100k_base'));
}
}
return encoder;
}
export async function tokenCount(text) {
try {
const enc = await getEncoder();
return enc.encode(text).length;
} catch {
// Fallback: rough estimate
return Math.ceil(text.length / 4);
}
}
export async function truncateToTokens(text, maxTokens) {
try {
const enc = await getEncoder();
const tokens = enc.encode(text);
if (tokens.length <= maxTokens) return text;
return enc.decode(tokens.slice(0, maxTokens));
} catch {
// Fallback
return text.slice(0, maxTokens * 4);
}
}