Major changes: - Add TLD price scraper with Porkbun API (886+ TLDs, no API key needed) - Fix .ch domain checker using rdap.nic.ch custom RDAP - Integrate database for TLD price history tracking - Add admin endpoints for manual scrape and stats - Extend scheduler with daily TLD price scrape job (03:00 UTC) - Update API to use DB data with static fallback - Update README with complete documentation New files: - backend/app/services/tld_scraper/ (scraper package) - TLD_TRACKING_PLAN.md (implementation plan) API changes: - POST /admin/scrape-tld-prices - trigger manual scrape - GET /admin/tld-prices/stats - database statistics - GET /tld-prices/overview now uses DB data
15 KiB
pounce — Domain Availability Monitoring
A professional full-stack application for monitoring domain name availability with TLD price tracking and intelligence.
Features
Core Functionality
- Domain Availability Monitoring — Track any domain and get notified when it becomes available
- TLD Price Intelligence — Compare prices across 886+ TLDs from Porkbun API
- Automated Price Scraping — Daily cronjob scrapes real TLD prices from public APIs
- Expiration Tracking — Monitor domain expiration dates and plan ahead
- Real-time Checks — RDAP, WHOIS, and DNS-based availability verification
- Swiss Domain Support — Special RDAP integration for .ch/.li domains via nic.ch
- Historical Data — Track price trends and availability history over time (12 months)
User Experience
- Modern UI — Clean, minimalist dark-mode design with smooth animations
- Responsive — Fully optimized for desktop, tablet, and mobile
- Authentication — Secure JWT-based auth with subscription tiers
- Dashboard — Personal watchlist with status indicators and actions
Tech Stack
Backend
| Technology | Purpose |
|---|---|
| Python 3.12+ | Runtime |
| FastAPI | Async web framework |
| SQLAlchemy 2.0 | Async ORM |
| SQLite/PostgreSQL | Database |
| APScheduler | Background job scheduling |
| python-whois | WHOIS domain lookups |
| whodap | RDAP domain lookups |
| httpx | Async HTTP client (for custom RDAP) |
| BeautifulSoup4 | Web scraping (backup) |
| Pydantic | Data validation |
| JWT (python-jose) | Authentication |
Frontend
| Technology | Purpose |
|---|---|
| Next.js 14 | React framework (App Router) |
| TypeScript | Type safety |
| Tailwind CSS | Styling |
| Zustand | State management |
| Lucide React | Icons (outlined style) |
| clsx | Conditional classes |
Project Structure
pounce/
├── backend/
│ ├── app/
│ │ ├── api/ # API endpoints
│ │ │ ├── auth.py # Authentication (register, login, me)
│ │ │ ├── check.py # Domain availability check
│ │ │ ├── domains.py # Domain watchlist CRUD
│ │ │ ├── subscription.py # User subscription management
│ │ │ ├── tld_prices.py # TLD pricing data
│ │ │ └── admin.py # Admin endpoints
│ │ ├── models/ # SQLAlchemy models
│ │ │ ├── user.py # User model
│ │ │ ├── domain.py # Domain watchlist model
│ │ │ ├── subscription.py # Subscription tiers
│ │ │ └── tld_price.py # TLD pricing model
│ │ ├── schemas/ # Pydantic schemas
│ │ ├── services/ # Business logic
│ │ │ ├── auth.py # Auth service (JWT, hashing)
│ │ │ ├── domain_checker.py # RDAP/WHOIS/DNS checks
│ │ │ └── tld_scraper/ # TLD price scraping
│ │ │ ├── __init__.py
│ │ │ ├── base.py # Base scraper class
│ │ │ ├── porkbun.py # Porkbun API scraper
│ │ │ ├── tld_list.py # TLD-List scraper (placeholder)
│ │ │ └── aggregator.py # Combines sources, saves to DB
│ │ ├── config.py # Environment settings
│ │ ├── database.py # Database connection
│ │ ├── main.py # FastAPI app entry
│ │ └── scheduler.py # Background jobs
│ ├── requirements.txt
│ ├── Dockerfile
│ └── env.example
├── frontend/
│ ├── src/
│ │ ├── app/ # Next.js App Router pages
│ │ │ ├── page.tsx # Homepage with domain checker
│ │ │ ├── layout.tsx # Root layout
│ │ │ ├── dashboard/ # User dashboard
│ │ │ ├── login/ # Login page
│ │ │ ├── register/ # Registration page
│ │ │ ├── pricing/ # Pricing plans
│ │ │ └── tld-pricing/ # TLD price intelligence
│ │ │ ├── page.tsx # TLD overview
│ │ │ └── [tld]/ # TLD detail page
│ │ ├── components/ # React components
│ │ │ ├── Header.tsx # Navigation header
│ │ │ ├── Footer.tsx # Site footer
│ │ │ └── DomainChecker.tsx # Domain search component
│ │ └── lib/ # Utilities
│ │ ├── api.ts # API client
│ │ └── store.ts # Zustand store
│ ├── tailwind.config.ts # Tailwind configuration
│ ├── package.json
│ ├── Dockerfile
│ └── env.example
├── docker-compose.yml # Docker deployment
├── DEPLOYMENT.md # Deployment guide
└── README.md
Pages Overview
| Route | Description | Auth Required |
|---|---|---|
/ |
Homepage with domain checker, features, pricing preview | No |
/login |
User login | No |
/register |
User registration | No |
/dashboard |
Personal domain watchlist | Yes |
/pricing |
Subscription plans with FAQ | No |
/tld-pricing |
TLD price overview with trends | No* |
/tld-pricing/[tld] |
TLD detail with registrar comparison | Yes |
*Unauthenticated users see limited data with shimmer effects
Quick Start
Prerequisites
- Python 3.12+
- Node.js 18+
- npm or yarn
1. Clone Repository
git clone https://git.6bit.ch/yvg/pounce.git
cd pounce
2. Backend Setup
cd backend
# Create virtual environment
python3 -m venv venv
source venv/bin/activate # Linux/macOS
# .\venv\Scripts\activate # Windows
# Install dependencies
pip install -r requirements.txt
# Create environment file
cp env.example .env
# Generate secret key and add to .env
python -c "import secrets; print(f'SECRET_KEY={secrets.token_hex(32)}')"
Edit .env:
DATABASE_URL=sqlite+aiosqlite:///./domainwatch.db
SECRET_KEY=your-generated-secret-key
CORS_ORIGINS=http://localhost:3000,http://127.0.0.1:3000
Start backend:
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
3. Frontend Setup
cd frontend
# Install dependencies
npm install
# Create environment file
echo "NEXT_PUBLIC_API_URL=http://localhost:8000" > .env.local
# Start development server
npm run dev
4. Access Application
- Frontend: http://localhost:3000
- Backend API: http://localhost:8000
- API Docs: http://localhost:8000/docs
API Endpoints
Authentication
| Method | Endpoint | Description |
|---|---|---|
| POST | /api/v1/auth/register |
Register new user |
| POST | /api/v1/auth/login |
Login (returns JWT) |
| GET | /api/v1/auth/me |
Get current user |
Domain Check
| Method | Endpoint | Description |
|---|---|---|
| POST | /api/v1/check |
Check domain availability |
Domain Watchlist
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/domains |
List watched domains |
| POST | /api/v1/domains |
Add domain to watchlist |
| DELETE | /api/v1/domains/{id} |
Remove domain |
| POST | /api/v1/domains/{id}/refresh |
Refresh domain status |
| GET | /api/v1/domains/{id}/history |
Get check history |
TLD Prices
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/tld-prices/overview |
Get all TLDs overview |
| GET | /api/v1/tld-prices/trending |
Get trending TLDs |
| GET | /api/v1/tld-prices/{tld} |
Get TLD details |
| GET | /api/v1/tld-prices/{tld}/history |
Get price history |
| GET | /api/v1/tld-prices/{tld}/compare |
Compare registrar prices |
Subscription
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/subscription |
Get current subscription |
| POST | /api/v1/subscription/upgrade |
Upgrade plan |
Admin
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/admin/users |
List all users |
| POST | /api/v1/admin/upgrade-user |
Upgrade user subscription |
| POST | /api/v1/admin/scrape-tld-prices |
Manually trigger TLD price scrape |
| GET | /api/v1/admin/tld-prices/stats |
Get TLD price database stats |
Subscription Tiers
| Feature | Starter (Free) | Professional ($4.99/mo) | Enterprise ($9.99/mo) |
|---|---|---|---|
| Domains | 3 | 25 | 100 |
| Check Frequency | Daily | Daily | Hourly |
| Notifications | Priority Email | Priority Email | |
| WHOIS Data | Basic | Full | Full |
| Check History | — | 30 days | Unlimited |
| Expiration Tracking | — | ✓ | ✓ |
| API Access | — | — | ✓ |
Environment Variables
Backend (.env)
| Variable | Description | Default |
|---|---|---|
DATABASE_URL |
Database connection string | sqlite+aiosqlite:///./domainwatch.db |
SECRET_KEY |
JWT signing key (min 32 chars) | Required |
CORS_ORIGINS |
Allowed origins (comma-separated) | http://localhost:3000 |
ACCESS_TOKEN_EXPIRE_MINUTES |
JWT token lifetime | 10080 (7 days) |
SCHEDULER_CHECK_INTERVAL_HOURS |
Domain check interval | 24 |
SMTP_HOST |
Email server host | Optional |
SMTP_PORT |
Email server port | 587 |
SMTP_USER |
Email username | Optional |
SMTP_PASSWORD |
Email password | Optional |
Frontend (.env.local)
| Variable | Description | Default |
|---|---|---|
NEXT_PUBLIC_API_URL |
Backend API URL | http://localhost:8000 |
Production Deployment
Docker (Recommended)
# Clone repository
git clone https://git.6bit.ch/yvg/pounce.git
cd pounce
# Set environment variables
export SECRET_KEY=$(python3 -c "import secrets; print(secrets.token_hex(32))")
# Build and start
docker-compose up -d --build
Manual Deployment
See DEPLOYMENT.md for detailed instructions including:
- Systemd service setup
- PM2 for Node.js
- Nginx reverse proxy configuration
- SSL/HTTPS with Let's Encrypt
- PostgreSQL database setup
Domain Checking Methods
pounce uses multiple methods to verify domain availability:
- Custom RDAP — For TLDs with own RDAP servers (e.g.,
.ch,.liviardap.nic.ch) - RDAP via whodap — Modern protocol with detailed data, supported by major registries
- WHOIS (Fallback) — Traditional protocol for domains without RDAP support
- DNS (Quick check) — Fast initial availability check via DNS queries
The system automatically falls back to the next method if one fails.
Supported TLDs with Custom RDAP
| TLD | RDAP Server | Notes |
|---|---|---|
.ch |
rdap.nic.ch |
Swiss domains - WHOIS blocked |
.li |
rdap.nic.ch |
Liechtenstein - same registry |
Standard RDAP TLDs
.com, .net, .org, .info, .biz, .io, .co, .ai, .app, .dev, .xyz, .me, and many more.
TLD Price Scraping
pounce automatically scrapes TLD prices from public sources for price intelligence and trend tracking.
Data Source
Primary: Porkbun Public API (no API key required!)
POST https://api.porkbun.com/api/json/v3/pricing/get
- ✅ 886+ TLDs with pricing
- ✅ Registration, Renewal, Transfer prices
- ✅ No authentication needed
- ✅ Updated daily via scheduler
Scheduler Jobs
| Job | Schedule | Description |
|---|---|---|
| Domain Check | Configurable (default: daily) | Checks all watched domains |
| TLD Price Scrape | Daily at 03:00 UTC | Scrapes current TLD prices |
Manual Scrape
Trigger a manual TLD price scrape:
curl -X POST http://localhost:8000/api/v1/admin/scrape-tld-prices
Response:
{
"message": "TLD price scrape completed",
"status": "success",
"tlds_scraped": 886,
"prices_saved": 886,
"sources_succeeded": 1
}
Database Stats
Get current TLD price data statistics:
curl http://localhost:8000/api/v1/admin/tld-prices/stats
Response:
{
"total_records": 886,
"unique_tlds": 886,
"unique_registrars": 1,
"latest_record": "2025-12-08T08:04:54",
"data_range_days": 0
}
Price Data Model
TLDPrice:
- tld: str # e.g., "com", "io", "ai"
- registrar: str # e.g., "porkbun"
- registration_price: float
- renewal_price: float
- transfer_price: float
- currency: str # "USD"
- recorded_at: datetime # For historical tracking
Future Data Sources
The scraper architecture supports multiple sources:
- Porkbun API ✅ (active)
- TLD-List.com (blocked - placeholder)
- Namecheap Affiliate (requires signup)
- Cloudflare (requires account)
Design System
Colors
- Background: Dark (
#0a0a0a) - Foreground: Light (
#fafafa) - Accent: Teal (
#00d4aa) - Warning: Orange (
#f97316) - Success: Green (same as accent)
Typography
- Display: Playfair Display (serif)
- Body: Inter (sans-serif)
- Mono: JetBrains Mono (code)
Components
- All icons use outlined style (Lucide React)
- Minimalist, award-winning aesthetic
- Smooth animations and transitions
- Responsive design (mobile-first)
Development
Backend
cd backend
source venv/bin/activate
# Run with auto-reload
uvicorn app.main:app --reload
# API docs available at:
# http://localhost:8000/docs (Swagger)
# http://localhost:8000/redoc (ReDoc)
Frontend
cd frontend
# Development server
npm run dev
# Lint
npm run lint
# Build for production
npm run build
# Start production server
npm start
Troubleshooting
Backend won't start
- Check Python version:
python3 --version(needs 3.12+) - Ensure virtual environment is activated
- Check
.envfile exists and has validSECRET_KEY - Check for syntax errors in config files
Frontend can't connect to backend
- Ensure backend is running on port 8000
- Check
NEXT_PUBLIC_API_URLin.env.local - Verify CORS origins in backend
.env - Check browser console for CORS errors
Domain checks failing
- Some TLDs may have rate limits
- Certain ccTLDs require special handling (see below)
- Check backend logs for specific errors
Swiss/Liechtenstein domains (.ch, .li):
- WHOIS is blocked for automated requests
- pounce uses the official
rdap.nic.chAPI instead - This is automatic - no configuration needed
Other ccTLDs:
- Some country code TLDs don't support RDAP
- pounce falls back to DNS checking
- Results may be less detailed (no registrar info)
Database errors
- Delete
domainwatch.dbto reset (dev only) - Check database URL format in
.env - Ensure write permissions on database directory
License
MIT License
Support
For issues and feature requests, please open a GitHub issue or contact support@pounce.dev