yvg/pounce
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

docs(memory-bank): update architecture/auth/perf context

Browse Source
This commit is contained in:
yves.gugger
2025-12-12 12:07:20 +01:00
parent 5d23d34a8a
commit d08ca33fe3
4 changed files with 80 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
memory-bank/activeContext.md
View File

@ -8,7 +8,7 @@ Pounce Terminal fully functional with complete monitoring & notification system.
- [x] Database models (User, Domain, DomainCheck, Subscription, TLDPrice, DomainHealthCache)
- [x] Domain checker service (WHOIS + RDAP + DNS)
- [x] Domain health checker (DNS, HTTP, SSL layers)
- [x] Authentication system (JWT + OAuth)
- [x] Authentication system (HttpOnly cookies + OAuth)
- [x] API endpoints for domain management
- [x] Tiered scheduler for domain checks (Scout=daily, Trader=hourly, Tycoon=10min)
- [x] Next.js frontend with dark terminal theme
@ -18,8 +18,26 @@ Pounce Terminal fully functional with complete monitoring & notification system.
- [x] **Watchlist with automatic monitoring & alerts**
- [x] **Health check overlays with complete DNS/HTTP/SSL details**
- [x] **Instant alert toggle (no refresh needed)**
- [x] **Performance Phase 0–2 applied (scheduler split, DB/index fixes, cached health, dashboard summary, metrics, job queue scaffolding)**
## Recent Changes (Dec 2024)
## Recent Changes (Dec 2025)
### Security hardening
- **HttpOnly cookie auth** (no JWT in URLs / no token in `localStorage`)
- **OAuth redirect hardening** (state + redirect validation)
- **Blog HTML sanitization** on backend
- **Secrets removed from repo history** + `.gitignore` hardened
### Performance & architecture phases (0 → 2)
- **Scheduler split**: API runs with `ENABLE_SCHEDULER=false`, scheduler runs as separate process/container
- **Market feed**: bounded DB queries + pagination (no full table loads)
- **Health**: bulk cached endpoint (`/domains/health-cache`) + cache-first per-domain health
- **Radar**: single-call dashboard payload (`/dashboard/summary`) → fewer frontend round-trips
- **DB migrations**: idempotent indexes + optional columns for existing DBs
- **Auction scoring**: persisted `pounce_score` populated by scraper
- **Admin**: removed N+1 patterns in user listing/export
- **Observability**: Prometheus metrics (`/metrics`) + optional DB query timing
- **Job queue**: Redis + ARQ worker scaffolding + admin scraping enqueue
### Watchlist & Monitoring
1. **Automatic domain checks**: Runs based on subscription tier
@ -46,9 +64,9 @@ Pounce Terminal fully functional with complete monitoring & notification system.
## Next Steps
1. **Configure SMTP on server** - Required for email alerts to work
2. **Test email delivery** - Verify alerts are sent correctly
3. **Consider SMS alerts** - Would require Twilio integration
4. **Monitor scheduler health** - Check logs for job execution
2. **Run production stack with scheduler + worker** (Docker Compose includes `scheduler`, `worker`, `redis`)
3. **Monitor `/metrics`** and set alerts (p95 latency, DB query time, job failures)
4. **Run load test** (`loadtest/k6/api-smoke.js`) after each deployment
## Server Deployment Checklist
- [ ] Set `SMTP_*` environment variables (see `env.example`)
@ -69,4 +87,4 @@ Pounce Terminal fully functional with complete monitoring & notification system.
- Email alerts require SMTP configuration
- Some TLDs (.ch, .de) don't publish expiration dates publicly
- SSL checks may fail on local dev (certificate chain issues)
- Scheduler starts automatically with uvicorn
- Scheduler should not run in the API process in production (avoid duplicate jobs with multiple API workers)

35
memory-bank/progress.md
View File

@ -1,30 +1,25 @@
# DomainWatch - Progress
# Pounce - Progress
## What Works
- ✅ Full backend API
- ✅ User registration and login
- ✅ JWT authentication
- ✅ Public domain availability check
- ✅ Domain watchlist (add/remove/refresh)
- ✅ Subscription tiers and limits
- ✅ Daily scheduled domain checks
- ✅ Frontend with responsive design
- ✅ Dashboard with domain list
- ✅ Pricing page
- ✅ FastAPI backend + Next.js frontend (Terminal + public pages)
- ✅ Cookie-based auth (HttpOnly) + OAuth
- ✅ Market feed + auctions + listings + TLD intel
- ✅ Watchlist + cached health checks (bulk + per-domain cache-first)
- ✅ Separate scheduler process (avoid duplicate jobs with multiple API workers)
- ✅ Redis-backed job queue scaffolding (ARQ worker) + admin scraping enqueue
- ✅ Prometheus metrics endpoint (`/metrics`)
- ✅ Load test scaffolding (`loadtest/k6`)
## What's Left
- ⏳ Email notifications
- ⏳ Payment integration
- ⏳ Domain check history view
- ⏳ Password reset functionality
- ⏳ User settings page
- ⏳ Admin dashboard
- ⏳ SMTP configuration + deliverability verification
- ⏳ Production observability rollout (dashboards/alerts around `/metrics`)
- ⏳ Optional: migrate periodic background jobs from APScheduler → queue/worker for scaling
## Current Issues
- None known - awaiting first test run
## Performance Notes
- WHOIS queries: ~1-3 seconds per domain
- DNS queries: ~0.1-0.5 seconds per domain
- Scheduler configured for 6:00 AM daily checks
- WHOIS queries are inherently slow (external I/O): expect ~1–3s/domain
- DB hotspots were reduced (market feed bounded queries, N+1 removed in price tracking/admin)
- Use `/metrics` + `loadtest/k6` to track p95 after deployments

48
memory-bank/systemPatterns.md
View File

@ -1,18 +1,24 @@
# DomainWatch - System Patterns
# Pounce - System Patterns
## Architecture
```
┌─────────────────┐ ┌─────────────────┐
│ Next.js App │────▶│ FastAPI Backend │
│ (Port 3000) │◀────│ (Port 8000) │
└─────────────────┘ └────────┬────────┘
┌─────────────────┐ ┌────────────────────┐
│ Next.js App │◀────▶│ FastAPI API │
│ (Port 3000) │ │ (Port 8000) │
└─────────────────┘ └──────────┬─────────┘
│
┌────────────┼────────────┐
┌───────────────┼────────────────┐
│ │ │
┌─────▼─────┐ ┌────▼────┐ ┌────▼────┐
│ SQLite/ │ │ WHOIS │ │ DNS │
│ Postgres │ │ Lookups │ │ Lookups │
└───────────┘ └─────────┘ └─────────┘
┌─────▼─────┐ ┌────▼────┐ ┌─────▼─────┐
│ Postgres │ │ Redis │ │ External │
│ (primary) │ │ queue + │ │ I/O (DNS, │
└───────────┘ │ limiter │ │ WHOIS, HTTP│
└─────────┘ └───────────┘
Separate processes (recommended):
- API: `ENABLE_SCHEDULER=false`
- Scheduler: `python backend/run_scheduler.py`
- Worker: `arq app.jobs.worker.WorkerSettings`
```
## Design Patterns
@ -32,10 +38,10 @@
## Authentication Flow
```
1. User registers → Creates user + free subscription
2. User logs in → Receives JWT token
3. Token stored in localStorage
4. API requests include Bearer token
5. Backend validates token → Returns user data
2. User logs in → Backend sets HttpOnly auth cookie (JWT inside cookie)
3. Frontend calls API with `credentials: 'include'`
4. Backend validates cookie → Returns user data
5. OAuth uses `state` + validated redirects, then sets cookie (no JWT in URL)
```
## Domain Checking Strategy
@ -49,16 +55,12 @@
## Scheduler Pattern
```
APScheduler (AsyncIO mode)
APScheduler (AsyncIO mode) in separate scheduler process
│
└── CronTrigger (daily at 06:00)
│
└── check_all_domains()
│
├── Fetch all domains
├── Check each with 0.5s delay
├── Update statuses
└── Log newly available domains
├── Domain checks (tier-based frequency)
├── TLD price scrape + change detection
├── Auction scrape + cleanup
└── Health cache refresh (writes DomainHealthCache used by UI)
```
## Database Models

21
memory-bank/techContext.md
View File

@ -1,4 +1,4 @@
# DomainWatch - Technical Context
# Pounce - Technical Context
## Tech Stack
@ -6,8 +6,10 @@
- **Framework:** FastAPI (Python 3.11+)
- **Database:** SQLite (development) / PostgreSQL (production)
- **ORM:** SQLAlchemy 2.0 with async support
- **Authentication:** JWT with python-jose, bcrypt for password hashing
- **Scheduling:** APScheduler for background jobs
- **Authentication:** HttpOnly cookie auth (JWT in cookie) + OAuth (Google/GitHub)
- **Scheduling:** APScheduler (recommended as separate scheduler process)
- **Job Queue (optional):** Redis + ARQ worker for non-blocking admin/long jobs
- **Observability:** Prometheus metrics endpoint (`/metrics`)
### Frontend
- **Framework:** Next.js 14 (App Router)
@ -30,6 +32,8 @@ hushen_test/
│ │ ├── models/ # Database models
│ │ ├── schemas/ # Pydantic schemas
│ │ ├── services/ # Business logic
│ │ ├── jobs/ # ARQ worker + tasks (optional)
│ │ ├── observability/ # Metrics (Prometheus)
│ │ ├── config.py # Settings
│ │ ├── database.py # DB configuration
│ │ ├── main.py # FastAPI app
@ -45,7 +49,7 @@ hushen_test/
└── memory-bank/ # Project documentation
```
## API Endpoints
## API Endpoints (high-level)
### Public
- `POST /api/v1/check/` - Check domain availability
@ -53,14 +57,16 @@ hushen_test/
### Authenticated
- `POST /api/v1/auth/register` - Register user
- `POST /api/v1/auth/login` - Get JWT token
- `POST /api/v1/auth/login` - Sets HttpOnly cookie session
- `GET /api/v1/auth/me` - Current user info
- `GET /api/v1/domains/` - List monitored domains
- `GET /api/v1/domains/health-cache` - Bulk cached health reports for watchlist UI
- `POST /api/v1/domains/` - Add domain to watchlist
- `DELETE /api/v1/domains/{id}` - Remove domain
- `POST /api/v1/domains/{id}/refresh` - Manual refresh
- `GET /api/v1/subscription/` - User subscription info
- `GET /api/v1/subscription/tiers` - Available plans
- `GET /api/v1/dashboard/summary` - Single-call payload for `/terminal/radar`
## Development
@ -81,8 +87,9 @@ npm run dev
```
## Production Deployment
- Backend: uvicorn with gunicorn
- Frontend: next build && next start
- Backend: run API + scheduler (separate) + optional worker
- Frontend: Next.js (`output: 'standalone'` for Docker)
- Database: PostgreSQL recommended
- Redis: recommended (rate limiting storage + job queue)
- Reverse proxy: nginx recommended