yvg/pounce
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

docs(memory-bank): update architecture/auth/perf context

Browse Source
This commit is contained in:
yves.gugger
2025-12-12 12:07:20 +01:00
parent 5d23d34a8a
commit d08ca33fe3
4 changed files with 80 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
memory-bank/activeContext.md
View File

@ -8,7 +8,7 @@ Pounce Terminal fully functional with complete monitoring & notification system.
- [x] Database models (User, Domain, DomainCheck, Subscription, TLDPrice, DomainHealthCache) - [x] Database models (User, Domain, DomainCheck, Subscription, TLDPrice, DomainHealthCache)
- [x] Domain checker service (WHOIS + RDAP + DNS) - [x] Domain checker service (WHOIS + RDAP + DNS)
- [x] Domain health checker (DNS, HTTP, SSL layers) - [x] Domain health checker (DNS, HTTP, SSL layers)
- [x] Authentication system (JWT + OAuth) - [x] Authentication system (HttpOnly cookies + OAuth)
- [x] API endpoints for domain management - [x] API endpoints for domain management
- [x] Tiered scheduler for domain checks (Scout=daily, Trader=hourly, Tycoon=10min) - [x] Tiered scheduler for domain checks (Scout=daily, Trader=hourly, Tycoon=10min)
- [x] Next.js frontend with dark terminal theme - [x] Next.js frontend with dark terminal theme
@ -18,8 +18,26 @@ Pounce Terminal fully functional with complete monitoring & notification system.
- [x] **Watchlist with automatic monitoring & alerts** - [x] **Watchlist with automatic monitoring & alerts**
- [x] **Health check overlays with complete DNS/HTTP/SSL details** - [x] **Health check overlays with complete DNS/HTTP/SSL details**
- [x] **Instant alert toggle (no refresh needed)** - [x] **Instant alert toggle (no refresh needed)**
- [x] **Performance Phase 0–2 applied (scheduler split, DB/index fixes, cached health, dashboard summary, metrics, job queue scaffolding)**
## Recent Changes (Dec 2024) ## Recent Changes (Dec 2025)
### Security hardening
- **HttpOnly cookie auth** (no JWT in URLs / no token in `localStorage`)
- **OAuth redirect hardening** (state + redirect validation)
- **Blog HTML sanitization** on backend
- **Secrets removed from repo history** + `.gitignore` hardened
### Performance & architecture phases (0 → 2)
- **Scheduler split**: API runs with `ENABLE_SCHEDULER=false`, scheduler runs as separate process/container
- **Market feed**: bounded DB queries + pagination (no full table loads)
- **Health**: bulk cached endpoint (`/domains/health-cache`) + cache-first per-domain health
- **Radar**: single-call dashboard payload (`/dashboard/summary`) → fewer frontend round-trips
- **DB migrations**: idempotent indexes + optional columns for existing DBs
- **Auction scoring**: persisted `pounce_score` populated by scraper
- **Admin**: removed N+1 patterns in user listing/export
- **Observability**: Prometheus metrics (`/metrics`) + optional DB query timing
- **Job queue**: Redis + ARQ worker scaffolding + admin scraping enqueue
### Watchlist & Monitoring ### Watchlist & Monitoring
1. **Automatic domain checks**: Runs based on subscription tier 1. **Automatic domain checks**: Runs based on subscription tier
@ -46,9 +64,9 @@ Pounce Terminal fully functional with complete monitoring & notification system.
## Next Steps ## Next Steps
1. **Configure SMTP on server** - Required for email alerts to work 1. **Configure SMTP on server** - Required for email alerts to work
2. **Test email delivery** - Verify alerts are sent correctly 2. **Run production stack with scheduler + worker** (Docker Compose includes `scheduler`, `worker`, `redis`)
3. **Consider SMS alerts** - Would require Twilio integration 3. **Monitor `/metrics`** and set alerts (p95 latency, DB query time, job failures)
4. **Monitor scheduler health** - Check logs for job execution 4. **Run load test** (`loadtest/k6/api-smoke.js`) after each deployment
## Server Deployment Checklist ## Server Deployment Checklist
- [ ] Set `SMTP_*` environment variables (see `env.example`) - [ ] Set `SMTP_*` environment variables (see `env.example`)
@ -69,4 +87,4 @@ Pounce Terminal fully functional with complete monitoring & notification system.
- Email alerts require SMTP configuration - Email alerts require SMTP configuration
- Some TLDs (.ch, .de) don't publish expiration dates publicly - Some TLDs (.ch, .de) don't publish expiration dates publicly
- SSL checks may fail on local dev (certificate chain issues) - SSL checks may fail on local dev (certificate chain issues)
- Scheduler starts automatically with uvicorn - Scheduler should not run in the API process in production (avoid duplicate jobs with multiple API workers)

35
memory-bank/progress.md
View File

@ -1,30 +1,25 @@
# DomainWatch - Progress # Pounce - Progress
## What Works ## What Works
- ✅ Full backend API - ✅ FastAPI backend + Next.js frontend (Terminal + public pages)
- ✅ User registration and login - ✅ Cookie-based auth (HttpOnly) + OAuth
- ✅ JWT authentication - ✅ Market feed + auctions + listings + TLD intel
- ✅ Public domain availability check - ✅ Watchlist + cached health checks (bulk + per-domain cache-first)
- ✅ Domain watchlist (add/remove/refresh) - ✅ Separate scheduler process (avoid duplicate jobs with multiple API workers)
- ✅ Subscription tiers and limits - ✅ Redis-backed job queue scaffolding (ARQ worker) + admin scraping enqueue
- ✅ Daily scheduled domain checks - ✅ Prometheus metrics endpoint (`/metrics`)
- ✅ Frontend with responsive design - ✅ Load test scaffolding (`loadtest/k6`)
- ✅ Dashboard with domain list
- ✅ Pricing page
## What's Left ## What's Left
- ⏳ Email notifications - ⏳ SMTP configuration + deliverability verification
- ⏳ Payment integration - ⏳ Production observability rollout (dashboards/alerts around `/metrics`)
- ⏳ Domain check history view - ⏳ Optional: migrate periodic background jobs from APScheduler → queue/worker for scaling
- ⏳ Password reset functionality
- ⏳ User settings page
- ⏳ Admin dashboard
## Current Issues ## Current Issues
- None known - awaiting first test run - None known - awaiting first test run
## Performance Notes ## Performance Notes
- WHOIS queries: ~1-3 seconds per domain - WHOIS queries are inherently slow (external I/O): expect ~1–3s/domain
- DNS queries: ~0.1-0.5 seconds per domain - DB hotspots were reduced (market feed bounded queries, N+1 removed in price tracking/admin)
- Scheduler configured for 6:00 AM daily checks - Use `/metrics` + `loadtest/k6` to track p95 after deployments

52
memory-bank/systemPatterns.md
View File

@ -1,18 +1,24 @@
# DomainWatch - System Patterns # Pounce - System Patterns
## Architecture ## Architecture
``` ```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌────────────────────┐
│ Next.js App │────▶│ FastAPI Backend │ │ Next.js App │◀────▶│ FastAPI API │
│ (Port 3000) │◀────│ (Port 8000) │ │ (Port 3000) │ │ (Port 8000) │
└─────────────────┘ └────────┬────────┘ └─────────────────┘ └──────────┬─────────┘
│ │
┌────────────┼────────────┐ ┌───────────────┼────────────────┐
│ │ │ │ │ │
┌─────▼─────┐ ┌────▼────┐ ┌────▼────┐ ┌─────▼─────┐ ┌────▼────┐ ┌─────▼─────┐
│ SQLite/ │ │ WHOIS │ │ DNS │ │ Postgres │ │ Redis │ │ External │
│ Postgres │ │ Lookups │ │ Lookups │ │ (primary) │ │ queue + │ │ I/O (DNS, │
└───────────┘ └─────────┘ └─────────┘ └───────────┘ │ limiter │ │ WHOIS, HTTP│
└─────────┘ └───────────┘
Separate processes (recommended):
- API: `ENABLE_SCHEDULER=false`
- Scheduler: `python backend/run_scheduler.py`
- Worker: `arq app.jobs.worker.WorkerSettings`
``` ```
## Design Patterns ## Design Patterns
@ -32,10 +38,10 @@
## Authentication Flow ## Authentication Flow
``` ```
1. User registers → Creates user + free subscription 1. User registers → Creates user + free subscription
2. User logs in → Receives JWT token 2. User logs in → Backend sets HttpOnly auth cookie (JWT inside cookie)
3. Token stored in localStorage 3. Frontend calls API with `credentials: 'include'`
4. API requests include Bearer token 4. Backend validates cookie → Returns user data
5. Backend validates token → Returns user data 5. OAuth uses `state` + validated redirects, then sets cookie (no JWT in URL)
``` ```
## Domain Checking Strategy ## Domain Checking Strategy
@ -49,16 +55,12 @@
## Scheduler Pattern ## Scheduler Pattern
``` ```
APScheduler (AsyncIO mode) APScheduler (AsyncIO mode) in separate scheduler process
│ │
└── CronTrigger (daily at 06:00) ├── Domain checks (tier-based frequency)
│ ├── TLD price scrape + change detection
└── check_all_domains() ├── Auction scrape + cleanup
│ └── Health cache refresh (writes DomainHealthCache used by UI)
├── Fetch all domains
├── Check each with 0.5s delay
├── Update statuses
└── Log newly available domains
``` ```
## Database Models ## Database Models

21
memory-bank/techContext.md
View File

@ -1,4 +1,4 @@
# DomainWatch - Technical Context # Pounce - Technical Context
## Tech Stack ## Tech Stack
@ -6,8 +6,10 @@
- **Framework:** FastAPI (Python 3.11+) - **Framework:** FastAPI (Python 3.11+)
- **Database:** SQLite (development) / PostgreSQL (production) - **Database:** SQLite (development) / PostgreSQL (production)
- **ORM:** SQLAlchemy 2.0 with async support - **ORM:** SQLAlchemy 2.0 with async support
- **Authentication:** JWT with python-jose, bcrypt for password hashing - **Authentication:** HttpOnly cookie auth (JWT in cookie) + OAuth (Google/GitHub)
- **Scheduling:** APScheduler for background jobs - **Scheduling:** APScheduler (recommended as separate scheduler process)
- **Job Queue (optional):** Redis + ARQ worker for non-blocking admin/long jobs
- **Observability:** Prometheus metrics endpoint (`/metrics`)
### Frontend ### Frontend
- **Framework:** Next.js 14 (App Router) - **Framework:** Next.js 14 (App Router)
@ -30,6 +32,8 @@ hushen_test/
│ │ ├── models/ # Database models │ │ ├── models/ # Database models
│ │ ├── schemas/ # Pydantic schemas │ │ ├── schemas/ # Pydantic schemas
│ │ ├── services/ # Business logic │ │ ├── services/ # Business logic
│ │ ├── jobs/ # ARQ worker + tasks (optional)
│ │ ├── observability/ # Metrics (Prometheus)
│ │ ├── config.py # Settings │ │ ├── config.py # Settings
│ │ ├── database.py # DB configuration │ │ ├── database.py # DB configuration
│ │ ├── main.py # FastAPI app │ │ ├── main.py # FastAPI app
@ -45,7 +49,7 @@ hushen_test/
└── memory-bank/ # Project documentation └── memory-bank/ # Project documentation
``` ```
## API Endpoints ## API Endpoints (high-level)
### Public ### Public
- `POST /api/v1/check/` - Check domain availability - `POST /api/v1/check/` - Check domain availability
@ -53,14 +57,16 @@ hushen_test/
### Authenticated ### Authenticated
- `POST /api/v1/auth/register` - Register user - `POST /api/v1/auth/register` - Register user
- `POST /api/v1/auth/login` - Get JWT token - `POST /api/v1/auth/login` - Sets HttpOnly cookie session
- `GET /api/v1/auth/me` - Current user info - `GET /api/v1/auth/me` - Current user info
- `GET /api/v1/domains/` - List monitored domains - `GET /api/v1/domains/` - List monitored domains
- `GET /api/v1/domains/health-cache` - Bulk cached health reports for watchlist UI
- `POST /api/v1/domains/` - Add domain to watchlist - `POST /api/v1/domains/` - Add domain to watchlist
- `DELETE /api/v1/domains/{id}` - Remove domain - `DELETE /api/v1/domains/{id}` - Remove domain
- `POST /api/v1/domains/{id}/refresh` - Manual refresh - `POST /api/v1/domains/{id}/refresh` - Manual refresh
- `GET /api/v1/subscription/` - User subscription info - `GET /api/v1/subscription/` - User subscription info
- `GET /api/v1/subscription/tiers` - Available plans - `GET /api/v1/subscription/tiers` - Available plans
- `GET /api/v1/dashboard/summary` - Single-call payload for `/terminal/radar`
## Development ## Development
@ -81,8 +87,9 @@ npm run dev
``` ```
## Production Deployment ## Production Deployment
- Backend: uvicorn with gunicorn - Backend: run API + scheduler (separate) + optional worker
- Frontend: next build && next start - Frontend: Next.js (`output: 'standalone'` for Docker)
- Database: PostgreSQL recommended - Database: PostgreSQL recommended
- Redis: recommended (rate limiting storage + job queue)
- Reverse proxy: nginx recommended - Reverse proxy: nginx recommended