Learn cvar and command names from the server for autocomplete

Send cvarlist and cmdlist on connect and merge the answer into the
autocomplete candidates. Number the server menu from 0, keep suggesting
sibling commands while the command name is typed, mask the password in
the startup line, ignore venv/, fix README claims.

.gitignore

@@ -1,4 +1,5 @@
 __pycache__/
+venv/
 venv3/
 *test*
 *.json

README.md

@@ -40,7 +40,7 @@ qlpycon --host tcp://10.13.12.93:28960 --password secret   # connect directly
 qlpycon --list                                     # list configured servers
 ```
 
-Without arguments, qlpycon shows the servers from `[servers]` and connects to the one you select (Up/Down or j/k, Enter or the number key; q quits). If no servers are configured it connects to `[connection] host`.
+Without arguments, qlpycon shows the servers from `[servers]` numbered from 0 and connects to the one you select (Up/Down or j/k, Enter or the number key 0-9; q quits). If no servers are configured it connects to `[connection] host`.
 
 **Keys in the console:**
 - `Enter`: send the command
@@ -51,12 +51,12 @@ Without arguments, qlpycon shows the servers from `[servers]` and connects to th
 - `Ctrl-C` twice: quit
 
 **Options:**
-- `--host URI` — ZMQ RCON endpoint
-- `--password PASS` — RCON password (or set `QLPYCON_PASSWORD` env var)
-- `--list` — list configured servers and exit
-- `-v` / `-vv` — verbose (INFO) or debug (DEBUG) logging
-- `--json FILE` — log all JSON events to file
-- `--unknown-log FILE` — log unparsed events (default: unknown_events.log)
+- `--host URI`: ZMQ RCON endpoint
+- `--password PASS`: RCON password (or set `QLPYCON_PASSWORD` env var)
+- `--list`: list configured servers and exit
+- `-v` / `-vv`: verbose (INFO) or debug (DEBUG) logging
+- `--json FILE`: log all JSON events to file
+- `--unknown-log FILE`: log unparsed events (default: unknown_events.log)
 
 ## Features
 
@@ -64,25 +64,28 @@ Without arguments, qlpycon shows the servers from `[servers]` and connects to th
 - Team-aware colorized output with Quake color code support
 - Powerup pickup and carrier kill notifications
 - Server info panel (map, gametype, scores, players)
-- Tab autocomplete for cvars and commands with fuzzy matching
-- Argument suggestions for 25+ commands (bot names, maps, gametypes)
+- Tab autocomplete for cvars and commands with fuzzy matching; on connect the
+  server's own `cvarlist` and `cmdlist` output is read so every cvar and
+  command the server knows is suggested, not only the built-in list
+- Argument suggestions for 22 commands (bot names, maps, gametypes)
 - Command history (↑/↓)
 - Output scrollback (PgUp/PgDn) that survives terminal resizes
 
 ## Architecture
 
 ```
-main.py        — entry point, arg parsing, signal handling
-qlpycon.conf   — user configuration
+main.py        - entry point, arg parsing, signal handling
+qlpycon.conf   - user configuration
 lib/
-  constants.py — weapons, teams, colors, limits
-  settings.py  — config file loader
-  state.py     — game state (server info, players, teams)
-  network.py   — ZMQ connections (RCON DEALER, stats SUB)
-  parser.py    — JSON event parsing
-  formatter.py — message formatting and colorization
-  ui.py        — curses interface (info / output / input panels)
-  cvars.py     — cvar/command database and autocomplete
+  constants.py - weapons, teams, colors, limits
+  settings.py  - config file loader
+  state.py     - game state (server info, players, teams)
+  network.py   - ZMQ connections (RCON DEALER, stats SUB)
+  parser.py    - JSON event parsing
+  formatter.py - message formatting and colorization
+  ui.py        - curses interface (info / output / input panels)
+  cvars.py     - built-in cvar/command database and autocomplete
+  namelist.py  - learns cvar/command names from the server (cvarlist, cmdlist)
 ```
 
 ## License
@@ -175,7 +178,7 @@ $ addbot Sarge
   ↑ Shows skill levels (1-5)
 
 $ addbot Sarge 4
-red blue free spectator any
+red blue free spectator r
   ↑ Shows team values
 
 $ map
@@ -187,29 +190,35 @@ bloodrun
   ↑ Fuzzy matches 'blood' → 'bloodrun'
 
 $ g_gametype
-0 1 2 3 4 FFA Duel TDM CA CTF
-  ↑ Shows numeric and string gametype values
+ffa duel race tdm ca
+  ↑ Shows gametype values (string names first, then the numbers 0-11)
 
 $ callvote
-map kick shuffle teamsize
+map map_restart nextmap gametype kick
   ↑ Shows vote types
 ```
 
-The system knows valid values for **25+ commands** including:
+The system knows valid values for **22 commands** including:
 - **32 bot names**: Sarge, Ranger, Visor, Xaero, Anarki, etc.
-- **40+ maps**: bloodrun, campgrounds, toxicity, aerowalk, etc.
-- **12 game types**: FFA, Duel, TDM, CA, CTF, etc. (numeric and string forms)
-- **Vote types**: map, kick, shuffle, teamsize, g_gametype, etc.
-- **Team values**: red, blue, free, spectator, any
+- **38 maps**: bloodrun, campgrounds, toxicity, aerowalk, etc.
+- **12 game types**: ffa, duel, tdm, ca, ctf, etc. (string and numeric forms)
+- **Vote types**: map, kick, shuffle, teamsize, gametype, etc.
+- **Team values**: red, blue, free, spectator (and r, b, f, s)
 - **Skill levels**: 1-5 for bots
-- **Boolean values**: 0, 1, true, false, enabled, disabled
+- **Boolean values**: 0, 1
 - **Common settings**: timelimits, fraglimits, sv_fps values, etc.
 
 Arguments with freetext (like player names or custom messages) fall back to showing the signature with highlighting.
 
 ## Supported Commands
 
-**76 cvars and commands included:**
+Right after connecting, qlpycon sends `cvarlist` and `cmdlist` to the server
+and adds every name from the answer to the autocomplete candidates (the
+listing itself is hidden from the output). The built-in list below is the
+fallback before that answer arrives and the source of signatures and argument
+values.
+
+**Built-in cvars and commands:**
 
 ### Commands with Signatures
 
@@ -275,7 +284,9 @@ autocomplete('stat')
 
 ### Add Your Own Cvars
 
-Edit `cvars.py` and add to the appropriate list:
+Names the server reports via `cvarlist`/`cmdlist` are picked up automatically.
+To add a signature or argument values, or a name the server does not report,
+edit `cvars.py` and add to the appropriate list:
 
 ```python
 # Custom cvars

lib/cvars.py

@@ -40,6 +40,14 @@ GAME_CVARS = [
     'g_quadHog',
     'g_training',
     'g_instagib',
+    'g_password',
+    'g_needpass',
+    'g_overtime',
+    'g_startingHealth',
+    'g_startingArmor',
+    'g_infiniteAmmo',
+    'g_loadout',
+    'sv_privatePassword',
 ]
 
 # Map and rotation
@@ -92,6 +100,8 @@ QLX_CVARS = [
     'qlx_serverBrandName',
     'qlx_owner',
     'qlx_redditAuth',
+    'qlx_plugins',
+    'qlx_pluginsPath',
 ]
 
 # Bot cvars
@@ -151,9 +161,21 @@ COMMANDS = [
     'scores',
     # Info commands
     'serverinfo',
+    'systeminfo',
     'players',
     'maplist',
     'configstrings',
+    'cvarlist',
+    'cmdlist',
+    'dumpuser',
+    # Console
+    'exec',
+    'set',
+    'seta',
+    'sets',
+    'echo',
+    'clientkick',
+    'cp',
 ]
 
 # Bot names for addbot command (complete list)
@@ -271,6 +293,9 @@ COMMAND_SIGNATURES = {
     'sv_maxclients': '<1-64>',
     'sv_hostname': '<name>',
     'sv_fps': '<20|30|40|60|125>',
+    'g_password': '<password>',
+    'g_needpass': '<0|1>',
+    'sv_privatePassword': '<password>',
 
     # Network
     'net_port': '<port number>',
@@ -527,12 +552,18 @@ def fuzzy_match(query, candidates, max_results=5):
     return [match for match, score in matches[:max_results]]
 
 
-def autocomplete(partial, max_results=5):
+def autocomplete(partial, learned_names=(), max_results=5):
     """
-    Autocomplete a partial cvar/command
-    Returns list of suggestions
+    Autocomplete a partial cvar/command name.
+    Candidates are the built-in lists plus the names learned from the server
+    (see lib/namelist.py); a learned name that is already built in is skipped.
     """
-    return fuzzy_match(partial, ALL_CVARS, max_results)
+    candidates = list(ALL_CVARS)
+    builtin_lower = {name.lower() for name in ALL_CVARS}
+    for name in sorted(learned_names):
+        if name.lower() not in builtin_lower:
+            candidates.append(name)
+    return fuzzy_match(partial, candidates, max_results)
 
 
 def parse_signature(signature):

lib/namelist.py

@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+"""
+Cvar and command names learned from the server, for autocomplete.
+
+After connecting, main.py sends `cvarlist` and `cmdlist`. The server answers
+with one line per name and a count line at the end:
+    cvarlist:  S     A sv_hostname "My Server"   ...   1234 total cvars
+    cmdlist:   addbot                             ...   321 commands
+NameCapture collects the names from those lines and hides the listing from
+the output window while it is running.
+"""
+
+import logging
+import re
+import time
+
+logger = logging.getLogger('namelist')
+
+# Capture stops when no listing output arrived for this many seconds.
+# Guards against the count line looking different than expected, which would
+# otherwise hide all later output forever. Measured from the last message, not
+# from the request: a large cvarlist (5000+ messages) takes seconds to process.
+CAPTURE_TIMEOUT = 5.0
+
+# Cvar line: the name is the word directly before the quoted value.
+# Example: 'S     A sv_hostname "My Server"' -> sv_hostname
+CVARLIST_LINE = re.compile(r'([A-Za-z_]\w*)\s+"')
+
+# Command line: one bare word.
+# Example: 'addbot' -> addbot
+CMDLIST_LINE = re.compile(r'^\s*([A-Za-z_]\w*)\s*$')
+
+# Count line that ends a listing.
+# Examples: '1234 total cvars', '321 commands'
+END_LINE = re.compile(r'^\s*\d+\s+(total cvars|commands)\s*$')
+
+
+def unwrap(message):
+    """
+    Turn one RCON message into plain text with real newlines.
+    Example: 'print "addbot\\n"' -> 'addbot\n'
+    """
+    text = message
+    if text.startswith('print "'):
+        text = text[7:]
+        if text.endswith('"'):
+            text = text[:-1]
+    return text.replace('\\n', '\n')
+
+
+class NameCapture:
+    """Collects cvar and command names from the server's cvarlist and cmdlist output"""
+
+    def __init__(self):
+        self.names = set()
+        self.listings_pending = 0  # count lines still expected; 0 = not capturing
+        self.deadline = 0
+        self.partial_line = ''
+
+    def request(self, rcon):
+        """Send cvarlist and cmdlist; their output is captured until both count lines arrived"""
+        self.listings_pending = 2
+        self.deadline = time.time() + CAPTURE_TIMEOUT
+        self.partial_line = ''
+        rcon.send_command(b'cvarlist')
+        rcon.send_command(b'cmdlist')
+        logger.info('Requested cvarlist and cmdlist for autocomplete')
+
+    def handle(self, message):
+        """
+        Feed one RCON message.
+        Returns True when the message belongs to a running listing (hide it from the output).
+        """
+        if self.listings_pending == 0:
+            return False
+        if time.time() > self.deadline:
+            logger.warning(f'No listing output for {CAPTURE_TIMEOUT:.0f}s, '
+                           f'stopping capture with {len(self.names)} names')
+            self.listings_pending = 0
+            return False
+        self.deadline = time.time() + CAPTURE_TIMEOUT
+
+        # A line can arrive in several messages (the server sends each flag column of
+        # a cvarlist row as its own 1-byte message), so keep the unfinished tail
+        text = self.partial_line + unwrap(message)
+        lines = text.split('\n')
+        self.partial_line = lines.pop()
+        for line in lines:
+            self._learn_line(line)
+
+        # The last count line may come without a trailing newline
+        if END_LINE.match(self.partial_line):
+            self._learn_line(self.partial_line)
+            self.partial_line = ''
+        return True
+
+    def _learn_line(self, line):
+        if END_LINE.match(line):
+            self.listings_pending -= 1
+            if self.listings_pending == 0:
+                logger.info(f'Learned {len(self.names)} cvar and command names from the server')
+            return
+
+        match = CVARLIST_LINE.search(line) or CMDLIST_LINE.match(line)
+        if match:
+            self.names.add(match.group(1))

lib/ui.py

@@ -167,21 +167,20 @@ def suggestion_list_hint(suggestions, arg_type=None):
     return f'{shown}{more}'
 
 
-def compute_autocomplete(words, ends_with_space, player_names_provider):
+def compute_autocomplete(words, ends_with_space, player_names_provider, learned_names_provider=None):
     """
     Work out what to suggest for the current input.
+    player_names_provider / learned_names_provider: callables returning the
+    current player names and the cvar/command names learned from the server.
     Returns (suggestions, original_word, hint_parts):
     - suggestions: values cycled with Tab
     - original_word: the word Tab replaces ('' means Tab appends a new word)
     - hint_parts: [(text, attributes)] shown below the input line
     """
     first_word = words[0].lower()
+    typing_command_name = len(words) == 1 and not ends_with_space
 
-    if first_word in COMMAND_ARGUMENTS:
-        if len(words) == 1 and not ends_with_space:
-            # Just the command: show its signature with the first argument highlighted
-            return [], '', signature_hint(first_word, 0)
-
+    if first_word in COMMAND_ARGUMENTS and not typing_command_name:
         if ends_with_space:
             arg_position = len(words) - 1
             current_value = ''
@@ -199,26 +198,34 @@ def compute_autocomplete(words, ends_with_space, player_names_provider):
         # No value suggestions (free text, no players known): show the signature
         return [], '', signature_hint(first_word, arg_position)
 
-    if COMMAND_SIGNATURES.get(first_word):
+    if COMMAND_SIGNATURES.get(first_word) and not typing_command_name:
         # Command with a signature but no argument definitions
         return [], '', signature_hint(first_word, 0)
 
-    # Not a recognized command: suggest commands matching the current word
+    # Still typing the command name, or an unknown command: suggest names matching
+    # the current word. A complete name like `map` stays here on purpose because it
+    # can be the prefix of others (map_restart, maplist); its signature is shown too.
     current_word = words[-1]
     if len(current_word) < 2:
         return [], '', []
-    suggestions = autocomplete(current_word, max_results=5)
-    if not suggestions:
-        return [], current_word, []
-    return suggestions, current_word, [(' '.join(suggestions), curses.A_DIM)]
+    learned_names = learned_names_provider() if learned_names_provider else ()
+    suggestions = autocomplete(current_word, learned_names, max_results=5)
+    hint_parts = []
+    if suggestions:
+        hint_parts.append((' '.join(suggestions), curses.A_DIM))
+    if typing_command_name and COMMAND_SIGNATURES.get(first_word):
+        hint_parts.append(('  ', 0))
+        hint_parts.extend(signature_hint(first_word, 0))
+    return suggestions, current_word, hint_parts
 
 
 class InputLine:
     """The command input line: text, cursor, history and autocomplete state"""
 
-    def __init__(self, max_history, player_names_provider=None):
+    def __init__(self, max_history, player_names_provider=None, learned_names_provider=None):
         self.max_history = max_history
         self.player_names_provider = player_names_provider
+        self.learned_names_provider = learned_names_provider
         self.text = ''
         self.cursor = 0
         self.history = []
@@ -310,7 +317,7 @@ class InputLine:
             self._clear_suggestions()
             return
         self.suggestions, self.original_word, self.hint_parts = compute_autocomplete(
-            words, self.text.endswith(' '), self.player_names_provider)
+            words, self.text.endswith(' '), self.player_names_provider, self.learned_names_provider)
         self.suggestion_index = -1
 
     def _history_previous(self):
@@ -384,7 +391,7 @@ def select_server(screen, servers):
     """
     Full-screen menu listing the configured servers.
     servers: dict name -> host. Returns the chosen name, or None when the user quits.
-    Keys: Up/Down or j/k move, Enter connects, 1-9 connect directly, q or Esc quit.
+    Keys: Up/Down or j/k move, Enter connects, 0-9 connect directly, q or Esc quit.
     """
     names = list(servers)
     selected = 0
@@ -405,8 +412,8 @@ def select_server(screen, servers):
             selected = (selected + 1) % len(names)
         elif key in (curses.KEY_ENTER, 10, 13):
             return names[selected]
-        elif ord('1') <= key <= ord('9') and key - ord('1') < len(names):
-            return names[key - ord('1')]
+        elif ord('0') <= key <= ord('9') and key - ord('0') < len(names):
+            return names[key - ord('0')]
         elif key in (ord('q'), 27):  # 27 = Escape
             return None
         # Any other key (including KEY_RESIZE) just redraws
@@ -416,8 +423,7 @@ def _draw_server_menu(screen, servers, names, selected):
     screen.erase()
     safe_addstr(screen, 0, 2, 'Quake Live PyCon - select a server', curses.A_BOLD)
     for index, name in enumerate(names):
-        number = index + 1
-        line = f'{number:>2}. {name:<12} {servers[name]}'
+        line = f'{index:>2}. {name:<12} {servers[name]}'
         attributes = curses.A_REVERSE if index == selected else 0
         safe_addstr(screen, 2 + index, 2, line, attributes)
     safe_addstr(screen, 3 + len(names), 2,
@@ -428,7 +434,8 @@ def _draw_server_menu(screen, servers, names, selected):
 class UIManager:
     """Manages curses windows and display"""
 
-    def __init__(self, screen, title, max_history=MAX_COMMAND_HISTORY, player_names_provider=None):
+    def __init__(self, screen, title, max_history=MAX_COMMAND_HISTORY,
+                 player_names_provider=None, learned_names_provider=None):
         self.screen = screen
         self.title = title
         self.info_window = None
@@ -445,7 +452,7 @@ class UIManager:
         # Visual rows scrolled up from the newest output; 0 = follow new output
         self.scroll_offset = 0
 
-        self.input_line = InputLine(max_history, player_names_provider)
+        self.input_line = InputLine(max_history, player_names_provider, learned_names_provider)
         self.game_state = None  # last state shown in the info window
 
         self._init_curses()

main.py

@@ -22,6 +22,7 @@ from lib.parser import EventParser
 from lib.formatter import format_message, format_chat_message, format_powerup_message, strip_color_codes
 from lib.ui import UIManager, select_server
 from lib.settings import ConfigLoader
+from lib.namelist import NameCapture
 
 # Pre-compiled regex patterns
 CVAR_RESPONSE_PATTERN = re.compile(r'"([^"]+)"\s+is:"([^"]*)"')
@@ -131,7 +132,7 @@ def handle_stats_connection(message):
             password_str = match.group(2)
             password_str = strip_color_codes(password_str)
             stats_password = password_str.strip()
-            logger.info(f'Got stats password: {stats_password}')
+            logger.info('Got stats password')
 
     return stats_port, stats_password
 
@@ -340,15 +341,17 @@ def main_loop(screen, args):
 
     # Initialize components
     game_state = GameState()
+    name_capture = NameCapture()  # cvar/command names learned from the server for autocomplete
     ui = UIManager(screen, args.title, args.max_history,
-                   player_names_provider=lambda: game_state.player_tracker.get_player_names())
+                   player_names_provider=lambda: game_state.player_tracker.get_player_names(),
+                   learned_names_provider=lambda: name_capture.names)
 
     # Setup logging to output window
     log_handler = ui.setup_logging()
     logger.addHandler(log_handler)
 
     # Attach UI log handler to lib module loggers
-    for lib_name in ('network', 'parser', 'state'):
+    for lib_name in ('network', 'parser', 'state', 'namelist'):
         lib_logger = logging.getLogger(lib_name)
         if log_handler not in lib_logger.handlers:
             lib_logger.addHandler(log_handler)
@@ -359,7 +362,8 @@ def main_loop(screen, args):
     ui.print_message(f"zmq python bindings {zmq.__version__}, libzmq version {zmq.zmq_version()}\n")
 
     # Initialize network connections
-    ui.print_message(f"Connecting with host={args.host} password={args.password}\n")
+    password_display = '***' if args.password else '(none)'
+    ui.print_message(f"Connecting with host={args.host} password={password_display}\n")
     rcon = RconConnection(args.host, args.password, args.identity)
     rcon.connect()
 
@@ -408,6 +412,7 @@ def main_loop(screen, args):
                 ui.print_message("Requesting connection info...\n")
                 rcon.send_command(b'zmq_stats_password')
                 rcon.send_command(b'net_port')
+                name_capture.request(rcon)
             elif monitor_event and monitor_event[0] == zmq.EVENT_DISCONNECTED:
                 timestamp = time.strftime('%H:%M:%S')
                 ui.print_message(f"^3[^7{timestamp}^3] ^8^1Disconnected from server - waiting for reconnect...^0^7\n")
@@ -516,6 +521,12 @@ def main_loop(screen, args):
                             ui.print_message(f"^1[^7{timestamp}^1] Error: Stats connection failed: {e}^7\n")
                             logger.error(f'Stats connection failed: {e}')
 
+                    # Hide the cvarlist/cmdlist output while learning names from it.
+                    # Runs after the cvar handling above so the stats credentials and
+                    # server info responses are never swallowed.
+                    if name_capture.handle(message):
+                        continue
+
                     # Try to parse as game event
                     parsed_event = event_parser.parse_event(message)
                     if parsed_event: