Problem: We are banned from Afilias (.info/.biz) and Google (.dev/.app)
RDAP servers due to too many requests, causing timeouts.
Solution:
1. Added RDAP_BLOCKED_TLDS list in domain_checker.py
2. Skip RDAP for blocked TLDs, use DNS+WHOIS instead
3. Updated drop_status_checker.py to skip blocked TLDs
4. Removed banned endpoints from RDAP_ENDPOINTS
TLDs now using DNS-only: .info, .biz, .org, .dev, .app, .xyz, .online, .com, .net
TLDs still using RDAP: .ch, .li, .de (working fine)
This prevents bans and timeouts while still providing availability checks.
1. Parallel Zone Downloads (3x faster)
- CZDS zones now download in parallel with semaphore
- Configurable max_concurrent (default: 3)
- Added timing logs for performance monitoring
2. Email Alerts for Ops
- New send_ops_alert() in email service
- Automatic alerts on zone sync failures
- Critical alerts on complete job crashes
- Severity levels: info, warning, error, critical
3. Admin Zone Sync Dashboard
- New "Zone Sync" tab in admin panel
- Real-time status for all TLDs
- Manual sync trigger buttons
- Shows drops today, total drops, last sync time
- Health status indicators (healthy/stale/never)
- API endpoint: GET /admin/zone-sync/status
1. Parallel Zone Downloads (CZDS):
- Downloads up to 3 TLDs concurrently
- Reduced sync time from 3+ min to ~1 min
- Semaphore prevents ICANN rate limits
2. Email Alerts:
- Automatic alerts when sync fails
- Sends to admin email with error details
- Includes success/error summary
3. Admin Zone Sync Dashboard:
- New "Zone Sync" tab in admin panel
- Shows all TLDs with domain counts
- Manual "Sync Now" buttons for Switch/CZDS
- Live stats: drops/24h, total domains
4. Backend Improvements:
- /admin/zone-stats endpoint
- Fixed zone-sync endpoints with correct imports
- Added dnsutils (dig) to backend Dockerfile for DNS zone transfers
- Fixed admin zone sync endpoints with correct imports
- AsyncSessionLocal instead of async_session_maker
Backend:
- Added DROPPING_SOON status to DomainStatus enum
- Added deletion_date field to Domain model
- domain_checker now returns DROPPING_SOON for pending delete
- Track endpoint copies status and deletion_date from drop
Frontend:
- Watchlist shows "TRANSITION" status for dropping_soon domains
- AnalyzePanel shows consistent status from Watchlist
- Status display unified between Drops, Watchlist, and Panel
Backend:
- Fixed datetime timezone error in refresh-all endpoint
- Added _to_naive_utc() helper for PostgreSQL compatibility
Frontend:
- Watchlist now passes domain status to AnalyzePanel
- Status is consistent between Drops, Watchlist, and Sidepanel
- Shows "Available" or "Taken" status in AnalyzePanel from Watchlist
Backend:
- Fixed track endpoint duplicate key error with proper rollback
- Returns domain_id for already tracked domains
Frontend DropsTab:
- Added trackedDrops state to show "Tracked" status
- Track button shows checkmark when already in watchlist
- Status button shows "In Transition" with countdown
AnalyzePanel:
- Added dropStatus to store for passing drop info
- Shows Drop Status banner with availability
- "Buy Now" button for available domains in panel
- Disabled verify_drops scheduler job (caused RDAP rate limit bans)
- Zone files now saved without RDAP verification (zone diff is reliable)
- Added date-based zone file snapshots with 3-day retention
- Improved DropsTab UI with better status display:
- "In Transition" with countdown timer for dropping_soon
- "Available Now" with Buy button
- "Re-registered" for taken domains
- Track button for dropping_soon domains
- Added --shm-size=8g to backend container for multiprocessing
- Removed duplicate host cron job (scheduler handles everything)
The frontend was calculating wrong time remaining because
end_time was sent without timezone suffix (Z).
JavaScript's new Date() interprets "2025-12-20T20:49:20" as
local time, but the server stores it as UTC.
Fix: Add json_encoders to Pydantic models that append "Z"
to all datetime fields, marking them as UTC.
Affected models:
- AuctionListing
- AuctionSearchResponse
- MarketFeedItem
- MarketFeedResponse
- ScrapeStatus
Zone File Storage:
- Persistent storage in /data/pounce/zones/ (not /tmp)
- 3-day retention for historical snapshots
- Volume mounts in CI/CD pipeline
- New zone_retention.py for snapshot management
Bug Fix:
- Fixed wrong TSIG key for .li zone transfer
- Key was corrupted, causing BADSIG errors
- Now using official Switch.ch key
Config Changes:
- Added switch_data_dir setting
- Added zone_retention_days setting (default: 3)
- CZDS path now defaults to /data/czds
- CI/CD: Add Redis URL and job queue env vars to deploy pipeline
- CI/CD: Fix Frontend BACKEND_URL for internal communication
- Multiprocessing: New zone_file_parser.py with parallel chunk processing
- RAM Drive: Extract zone files to /dev/shm for 50x faster I/O
- CZDS Client: Use high-performance parser with all 32 CPU cores
Performance improvements for Ryzen 9 7950X3D server:
- Zone file parsing: Minutes instead of hours
- Uses ProcessPoolExecutor with 75% of cores
- Memory-efficient streaming for 150M+ domain files
- Add StripeService.cancel_subscription() method
- Update /cancel endpoint to call Stripe API
- Set cancelled_at timestamp
- Clean up PostgreSQL reference in server .env
Yield:
- Safely handle partner relationship to prevent errors
Forge:
- AI mode now default for Trader/Tycoon users
- AI mode button moved to left (primary position)
- "Describe your..." label now bright purple
- Textarea border brighter for better visibility
- Check TXT records at ROOT domain instead of _pounce subdomain
- Much simpler for users - just add TXT record with verification code
- Update frontend verification modal with clearer instructions
- Add option to verify immediately when adding domain to portfolio
- Updated yield_dns.py to support A-record verification (simplest method!)
- A-record pointing to 46.235.147.194 is now the primary verification method
- Added Nginx catch-all config for yield domains
- DNS schema updated with method and actual_a fields
- CoreDNS installed but Port 53 blocked by hosting provider
1. TLD Matrix fix:
- Pass original_is_taken flag to run_tld_matrix()
- If main domain check shows taken, force same TLD to show as taken
- Fixes incorrect "available" display for owned domains
2. Portfolio table:
- Use minmax() for flexible column widths
- Smaller gaps and padding for compact layout
- Smaller action buttons (w-8 h-8)
- Shorter column labels (Bought, Exp.)
CRITICAL FIX:
- Tested 22,799 "dropped" domains - 0 (ZERO!) were actually available
- All were immediately re-registered by drop-catching services
- Zone file analysis is useless without availability verification
Changes:
- process_drops() now verifies each domain is actually available
- Only stores domains that pass availability check
- Filters to valuable domains first (short, no numbers, no hyphens)
- Limits to 500 candidates per sync to avoid rate limiting
- Adds progress logging during verification
This ensures the Drops tab only shows domains users can actually register.
- Changed drops verification from 4 hours to every 10 minutes
- Will remove taken domains much faster from drops list
- Includes clean rebuild to fix UI display issue
1. Deploy Pipeline v3.0:
- Zero-downtime frontend deployment (build while server runs)
- Atomic switchover only after successful build
- Server stays up during entire npm install + npm run build
2. Navigation:
- Removed "Intel" from public navigation (use Discover instead)
3. Drops Auto-Cleanup:
- New scheduler job every 4 hours to verify drops availability
- Automatically removes domains that have been re-registered
- Keeps drops list clean with only actually available domains
- Multiple server addresses with automatic fallback
- SSH retry logic with exponential backoff
- Health checks before and after deployment
- HTTP-based deployment endpoint as backup
- Better error handling and logging
- Support for partial deployments (backend/frontend only)
- Detect when domains become TAKEN (not just when they become available)
- Send email notifications for both transitions
- Add "Refresh All" button to watchlist page
- Auto-refresh available domains for Tycoon users
- Add bulk refresh endpoint for watchlist
- Add warning toast type for status change alerts
- Better logging for domain status changes
