feat: Config-driven opt-in authentication registry with multi-platform support#2393
feat: Config-driven opt-in authentication registry with multi-platform support#2393
Conversation
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/da7ecfd0-e1c9-48dc-b692-27be0879e976 Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
…through auth registry - Add authentication/http.py with open_url() that tries each configured provider in registry order, falling through on 401/403 to the next, and finally to unauthenticated - Add build_request() for one-shot request construction - Add configured_providers() to registry __init__ - Remove api_base_url() from AuthProvider ABC (unused) - Remove hosts attribute from providers (no host matching) - Replace _github_http.py usage in ExtensionCatalog and PresetCatalog - Wire IntegrationCatalog and WorkflowCatalog through open_url (were unauthenticated) - Wire _fetch_latest_release_tag() through open_url - Wire all inline --from-url downloads through open_url - Fix unused stub variable flagged by code-quality bot - 49 auth tests (positive + negative), 1805 total tests passing
There was a problem hiding this comment.
Pull request overview
Adds an authentication provider registry to support multiple platforms (GitHub + Azure DevOps) and refactors outbound HTTP calls to use shared authenticated helpers.
Changes:
- Introduces
specify_cli.authenticationregistry +AuthProviderbase, withGitHubAuthandAzureDevOpsAuthimplementations. - Adds
authentication.httphelpers (build_request,open_url) and refactors catalog/workflow fetching and release-tag checks to use them. - Updates/expands tests to cover registry behavior, provider token resolution, and HTTP delegation.
Show a summary per file
| File | Description |
|---|---|
| tests/test_presets.py | Updates auth-related request tests for new auth helper behavior. |
| tests/test_extensions.py | Updates auth-related request tests for new auth helper behavior. |
| tests/test_authentication.py | Adds comprehensive tests for registry, providers, and HTTP helper behavior. |
| tests/integrations/test_integration_catalog.py | Adjusts urlopen monkeypatching to target new auth HTTP module. |
| src/specify_cli/workflows/catalog.py | Switches catalog fetching to use authentication.http.open_url. |
| src/specify_cli/presets.py | Routes request/open helpers through new auth HTTP module. |
| src/specify_cli/integrations/catalog.py | Switches catalog fetching to use authentication.http.open_url. |
| src/specify_cli/extensions.py | Routes request/open helpers through new auth HTTP module. |
| src/specify_cli/authentication/http.py | Implements shared authenticated request building + retry/fallthrough behavior. |
| src/specify_cli/authentication/github.py | Adds GitHub token resolution + Bearer header construction. |
| src/specify_cli/authentication/base.py | Adds AuthProvider ABC with get_token, auth_headers, is_configured. |
| src/specify_cli/authentication/azure_devops.py | Adds Azure DevOps PAT/token resolution + Basic auth header construction. |
| src/specify_cli/authentication/init.py | Implements global provider registry + builtin registration. |
| src/specify_cli/init.py | Refactors _fetch_latest_release_tag() and URL-download paths to use open_url. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 14/14 changed files
- Comments generated: 7
…d extra_headers to open_url - Fix _open_url() docstrings in extensions.py and presets.py that incorrectly claimed redirect stripping behavior - Add extra_headers parameter to open_url() so callers can pass additional headers (e.g. Accept) that persist across retries - Restore Accept: application/vnd.github+json header in _fetch_latest_release_tag() via extra_headers
Security-first redesign: no credentials are sent unless the user explicitly creates ~/.specify/auth.json mapping hosts to providers. - Add authentication/config.py: loads and validates auth.json with host-to-provider mappings, supports token/token_env/azure-ad/azure-cli - Refactor AuthProvider ABC: auth_headers(token, scheme) + resolve_token(entry) - Refactor GitHubAuth: bearer scheme only, token from config entry - Refactor AzureDevOpsAuth: 4 schemes (basic-pat, bearer, azure-cli, azure-ad) with dynamic token acquisition for azure-cli and azure-ad - Rewrite authentication/http.py: host matching, redirect stripping, provider fallthrough on 401/403, unauthenticated fallback - Add docs/reference/authentication.md with full reference and template - 1823 tests passing (67 auth-specific)
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 17/17 changed files
- Comments generated: 13
| | Field | Required | Description | | ||
| |---|---|---| | ||
| | `hosts` | Yes | Array of hostnames this entry applies to. Supports wildcards (e.g. `*.visualstudio.com`). | | ||
| | `provider` | Yes | Built-in provider key: `github` or `azure-devops`. | | ||
| | `auth` | Yes | Auth scheme (see below). | | ||
| | `token` | No | Token value (inline). Use `token_env` instead when possible. | | ||
| | `token_env` | No | Environment variable name to read the token from. | | ||
|
|
There was a problem hiding this comment.
The tables use leading double pipes (||) which creates an empty first column and is likely unintended / may break markdown rendering and linting. Consider converting these to standard markdown table syntax with single leading/trailing |.
| auth = entry_raw.get("auth", "") | ||
| if not isinstance(auth, str) or not auth: | ||
| raise ValueError(f"providers[{i}]: 'auth' must be a non-empty string") | ||
| if auth not in _VALID_AUTH_SCHEMES: | ||
| raise ValueError( | ||
| f"providers[{i}]: unsupported auth scheme {auth!r}; " | ||
| f"valid: {sorted(_VALID_AUTH_SCHEMES)}" | ||
| ) |
There was a problem hiding this comment.
auth is validated against a global set, but the config doesn't validate that the selected provider actually supports the requested scheme (e.g., provider="github" + auth="basic-pat" would pass here but later raise at request time). Consider validating provider+scheme compatibility during load (using the provider registry) and raising a clear ValueError that points to the offending entry.
| import os | ||
| import stat | ||
| from dataclasses import dataclass, field | ||
| from fnmatch import fnmatch | ||
| from pathlib import Path | ||
| from typing import Any |
There was a problem hiding this comment.
There are several unused imports here (os, field, Any). They add noise and can fail strict linting; please remove them.
| import os | |
| import stat | |
| from dataclasses import dataclass, field | |
| from fnmatch import fnmatch | |
| from pathlib import Path | |
| from typing import Any | |
| import stat | |
| from dataclasses import dataclass | |
| from fnmatch import fnmatch | |
| from pathlib import Path |
| def build_request(url: str, extra_headers: dict[str, str] | None = None) -> urllib.request.Request: | ||
| """Build a :class:`~urllib.request.Request`, attaching auth when config matches. | ||
|
|
||
| Uses the first matching entry from ``auth.json`` whose token resolves. | ||
| Returns a plain request when no entry matches or the file doesn't exist. | ||
| """ | ||
| headers: dict[str, str] = {} | ||
| entries = find_entries_for_url(url, _load_config()) | ||
| for entry in entries: | ||
| provider = get_provider(entry.provider) | ||
| if provider is None: | ||
| continue | ||
| token = provider.resolve_token(entry) | ||
| if token: | ||
| headers.update(provider.auth_headers(token, entry.auth)) | ||
| break | ||
| if extra_headers: | ||
| headers.update(extra_headers) | ||
| return urllib.request.Request(url, headers=headers) | ||
|
|
||
|
|
||
| def open_url(url: str, timeout: int = 10, extra_headers: dict[str, str] | None = None): | ||
| """Open *url* with config-driven auth, redirect stripping, and fallthrough. | ||
|
|
||
| 1. Find ``auth.json`` entries whose hosts match the URL. | ||
| 2. For each entry, resolve the token and try the request. | ||
| 3. On 401/403 move to the next matching entry. | ||
| 4. After all entries exhausted (or none matched), try unauthenticated. | ||
| 5. Non-auth errors (404, 500, network) raise immediately. | ||
|
|
||
| *extra_headers* (e.g. ``Accept``) are merged into every attempt. | ||
| """ | ||
| entries = find_entries_for_url(url, _load_config()) | ||
|
|
||
| def _make_req(auth_headers: dict[str, str]) -> urllib.request.Request: | ||
| merged = {**auth_headers} | ||
| if extra_headers: | ||
| merged.update(extra_headers) | ||
| return urllib.request.Request(url, headers=merged) |
There was a problem hiding this comment.
extra_headers are merged after provider-generated auth headers in both build_request() and open_url(), which allows callers to override Authorization (accidentally or intentionally) and potentially bypass the configured provider behavior. Consider merging extra_headers first and then applying provider auth headers last, or explicitly rejecting/ignoring an Authorization key in extra_headers.
| tried = 0 | ||
| for entry in entries: | ||
| provider = get_provider(entry.provider) | ||
| if provider is None: | ||
| continue | ||
| token = provider.resolve_token(entry) | ||
| if not token: | ||
| continue | ||
| tried += 1 | ||
|
|
There was a problem hiding this comment.
tried is incremented but never read. This looks like leftover instrumentation; please remove it (or use it) to avoid dead code.
…heme validation, security hardening - Remove unused imports (os, field, Any) in config.py - Normalize hosts during load (strip + lowercase) - Validate token/token_env are non-empty strings during load - Validate provider+scheme compatibility during load - Fix extra_headers order: auth headers applied last, cannot be overridden - Remove unused 'tried' variable in http.py - Warn (once) on malformed auth.json instead of silent fallback - URL-encode OAuth2 client credentials body in azure_devops.py - Update 403 message to mention auth.json configuration - Fix registry leak in test_register_duplicate (try/finally) - Fix import style consistency in test_authentication.py - Add azure-cli and azure-ad token acquisition tests (mock subprocess/urlopen) - Add autouse fixture to isolate upgrade tests from real auth.json - 1829 tests passing
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 17/17 changed files
- Comments generated: 4
…ization from extra_headers - Reject unknown provider keys during auth.json load with clear error message - Validate azure-ad tenant_id/client_id/client_secret_env as non-empty strings - Strip Authorization from extra_headers in both build_request and open_url to prevent accidental or intentional bypass of provider-configured auth - Add tests for unknown provider and incompatible scheme validation - 1831 tests passing
| try: | ||
| return opener.open(req, timeout=timeout) | ||
| except urllib.error.HTTPError as exc: | ||
| if exc.code in (401, 403): |
| _VALID_AUTH_SCHEMES = frozenset({ | ||
| "bearer", | ||
| "basic-pat", | ||
| "azure-cli", | ||
| "azure-ad", | ||
| }) | ||
|
|
||
|
|
| payload = _json.loads(result.stdout) | ||
| token = payload.get("accessToken", "").strip() | ||
| return token or None | ||
| except (OSError, _json.JSONDecodeError, KeyError): |
| cfg.chmod(stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP | stat.S_IROTH) | ||
| with pytest.warns(UserWarning, match="readable by group"): | ||
| load_auth_config(cfg) |
| try: | ||
| with urllib.request.urlopen(entry.url, timeout=10) as resp: | ||
| from specify_cli.authentication.http import open_url | ||
|
|
||
| with open_url(entry.url, timeout=10) as resp: |
|
@copilot apply changes based on the comments in this thread |
…meoutExpired, skip POSIX test on Windows, remove unused import Agent-Logs-Url: https://github.com/github/spec-kit/sessions/a1e29737-dd6e-4287-96c1-509e0c96fb21 Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
Applied in commit
|
| return None, ( | ||
| f"rate limited (configure {_auth_config_path()} with a GitHub token)" |
| # Warn (but don't fail) if the file is world-readable. | ||
| try: | ||
| mode = config_path.stat().st_mode | ||
| if mode & (stat.S_IRGRP | stat.S_IROTH): | ||
| import warnings | ||
|
|
||
| warnings.warn( | ||
| f"{config_path} is readable by group/others. " | ||
| "Consider restricting with: chmod 600 " | ||
| f"{config_path}", | ||
| UserWarning, |
|
@copilot apply changes based on the comments in this thread |
…X permission check on Windows Agent-Logs-Url: https://github.com/github/spec-kit/sessions/4636bcdb-87ae-45d6-9545-a40e4effd617 Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
Applied in commit
|
| hosts = entry_raw.get("hosts") | ||
| if not isinstance(hosts, list) or not hosts: | ||
| raise ValueError(f"providers[{i}]: 'hosts' must be a non-empty array") | ||
| if not all(isinstance(h, str) and h.strip() for h in hosts): | ||
| raise ValueError(f"providers[{i}]: each host must be a non-empty string") | ||
| # Normalize hosts: strip whitespace and lowercase | ||
| hosts = [h.strip().lower() for h in hosts] | ||
|
|
| def _load_config() -> list[AuthConfigEntry]: | ||
| """Load auth config, using override if set (for testing).""" | ||
| if _config_override is not None: | ||
| return _config_override | ||
| try: | ||
| return load_auth_config() | ||
| except (ValueError, OSError) as exc: | ||
| import warnings | ||
| config_path = _default_config_path() | ||
| warnings.warn( | ||
| f"Failed to load {config_path}: {exc}. " | ||
| "All requests will be unauthenticated.", | ||
| UserWarning, | ||
| stacklevel=2, | ||
| ) | ||
| return [] |
|
@copilot apply changes based on the comments in this thread |
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/889b58a7-7f8c-47e2-8056-931ebcc671cc Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
… type Agent-Logs-Url: https://github.com/github/spec-kit/sessions/889b58a7-7f8c-47e2-8056-931ebcc671cc Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/889b58a7-7f8c-47e2-8056-931ebcc671cc Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
Applied in commits
|
Security-first authentication for catalog systems
No credentials are sent unless the user explicitly creates
~/.specify/auth.jsonmapping hosts to providers. This is a full opt-in model — without the config file, all HTTP requests are unauthenticated.~/.specify/auth.json{ "providers": [ { "hosts": ["github.com", "api.github.com", "raw.githubusercontent.com", "codeload.github.com"], "provider": "github", "auth": "bearer", "token_env": "GH_TOKEN" } ] }See
docs/reference/authentication.mdfor full reference and template.New:
src/specify_cli/authentication/How it works
open_url()matches URL hostname againsthostsinauth.jsontoken,token_env, or dynamic acquisition)Authorizationheader (Bearer, Basic, etc.)Authorizationwhen leaving the entry's declared hostsProviders and auth schemes
GitHub (
github)bearerAzure DevOps (
azure-devops)basic-patAuthorization: Basic base64(:<PAT>)bearerazure-cliaz account get-access-tokenazure-adWiring
All HTTP request paths now go through
authentication.http.open_url():ExtensionCatalog._open_url()/_make_request()PresetCatalog._open_url()/_make_request()IntegrationCatalog._fetch_single_catalog()WorkflowCatalog._fetch_single_catalog()_fetch_latest_release_tag()(withAccept: application/vnd.github+json)--from-urldownload paths in CLI commandsSecurity hardening
hostsentries inauth.jsonare now validated to only allow exact hostnames (e.g.github.com) or the constrained*.suffixwildcard form (e.g.*.visualstudio.com). Dangerous forms like*github.com— whichfnmatchwould match againstgithub.com.evil.com, leaking credentials — are rejected with aValueErrorat config-load time._load_config()reads and validatesauth.jsonat most once per process. Subsequent calls return the cached result, eliminating redundant I/O on every HTTP request. A warning about a malformed config file fires at most once.HTTPErrorresponses are explicitly closed before retrying the next auth entry on 401/403, preventing connection leaks.supported_auth_schemescheck is derived from the registered provider rather than a separate static set, eliminating the risk of the two drifting out of sync.Tests
1823+ total tests passing, including auth-specific tests covering:
*.suffixacceptance)