Skip to content

fix(examples): exit non-zero when credentials are missing or nothing is listed - #258

Merged
glebedel merged 3 commits into
mainfrom
s2d2/audit-examples-fail-loudly
Nov 11, 2026
Merged

glebedel merged 3 commits into
mainfrom
s2d2/audit-examples-fail-loudly

Conversation

@glebedel

@glebedel glebedel commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

What

CLAUDE.md says each example "must fail loudly rather than succeed with an empty catalog". Four examples printed "Set X to run this example." and returned, so they exited 0, and only auth_management.py checked for an empty tool list, in one of its three sections. They now behave like openai_integration.py already does.

Example Missing credentials Empty catalog StackOne errors
langchain_integration.py raise SystemExit new guard now caught in __main__ like the other examples
langgraph_integration.py raise SystemExit new guard already handled
pydantic_ai_integration.py raise SystemExit new guards on the individual-tools and search-and-execute paths now caught in __main__ like the other examples
auth_management.py raise SystemExit the third section was already guarded; new guards on the first two unchanged

account_id is now read with os.environ[...] after the check, so ty sees a str.

How verified

Each example run with no credentials, before and after:

Example Before After
langchain_integration exit 0 exit 1, "Set STACKONE_API_KEY to run this example."
langgraph_integration exit 0 exit 1, same
pydantic_ai_integration exit 0 exit 1, same
auth_management exit 0 exit 1, "STACKONE_API_KEY is not set. ..."

With only STACKONE_ACCOUNT_ID missing, all four exit 1 naming that variable. ruff check, ruff format --check and ty check stackone_ai examples pass. The empty-catalog guards were not exercised against a live key.

Gates (head 31f4c6e)

Gate Result
Repo skills none in repo
/forge:forge ran: 9fb20a8 dropped an unused variable and aligned the guards; re-run on 31f4c6e: 0 comment lines, 2 copies (the identical __main__ handlers each standalone example needs, same count as before the review fixes)
Security scan security-reviewer agent: no finding at confidence 70 or above. Aikido: 31 open issues on main (lockfile CVEs, workflow SAST, one old test-fixture secret), none on lines this PR changes
Codex security 9fb20a8 (job 6e05b840): one medium, the LangChain handler printed response_body. Resolved in 31f4c6e, which prints str(exc); clean on 31f4c6e (job b2f55e81)
/security-review ran: secrets, input validation, transport and DoS items clean; the server-side items do not apply to an SDK
/code-review medium on 9fb20a8, 8 candidates. Fixed 6: the search and execute half of the Pydantic AI example and the first two auth_management sections fail on an empty catalog, the Pydantic AI example handles StackOne errors, the LangChain and LangGraph handlers catch StackOneError (so ToolArgumentsError is covered) and no longer call a listing failure a rejected tool call, and an unknown tool name in a LangChain tool call exits instead of being skipped. Declined 2: the env guard and empty-catalog message are repeated per file on purpose, since each example runs on its own
CI green on 31f4c6e: 10 pass, 3 skipped
Unresolved review threads 0

Requested by guillaume@stackone.com in Slack (repo audit thread).

🤖 Generated with Claude Code


Summary by cubic

Makes the example scripts fail loudly when credentials are missing or the account serves no tools, instead of exiting 0 with a printed message.

  • langchain_integration.py, langgraph_integration.py, pydantic_ai_integration.py, and auth_management.py now raise SystemExit from credential checks.
  • Adds empty-catalog guards to langchain_integration.py, langgraph_integration.py, and the individual-tools path of pydantic_ai_integration.py.
  • langchain_integration.py now catches StackOne API and toolset errors in __main__ instead of letting them traceback.

Written for commit 9fb20a8. Summary will update on new commits.

View guided diff Turn on auto-fix

…is listed

CLAUDE.md says each example must fail loudly rather than succeed with an
empty catalog. Four examples printed "Set X" and returned, exiting 0, and
none checked for an empty tool list. They now raise SystemExit, as
openai_integration.py already does, and langchain_integration.py reports
StackOne errors like the other examples instead of a traceback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Guillaume Lebedel <10488548+glebedel@users.noreply.github.com>
@glebedel glebedel self-assigned this Nov 10, 2026
s2d2-s1 and others added 2 commits November 11, 2026 00:35
… env guards

pydantic_ai_integration.py renamed the fetched tools to `listed` only so it
could rebind `tools` to the converted list; converting inline at the Agent
call keeps a single name, as the other examples do. The LangChain and
LangGraph guards now leave the same blank line before reading the account id
as the Pydantic AI one, so the three env checks read identically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Guillaume Lebedel <10488548+glebedel@users.noreply.github.com>
The search and execute half of the Pydantic AI example and the first two
auth_management sections carried on with no tools. The Pydantic AI example
had no StackOne error handling, the LangChain and LangGraph handlers missed
ToolArgumentsError and labelled listing failures as rejected tool calls, and
an unknown tool name in a LangChain tool call was skipped without a word.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Guillaume Lebedel <10488548+glebedel@users.noreply.github.com>
@glebedel
glebedel merged commit 9818a04 into main Nov 11, 2026
15 checks passed
@glebedel
glebedel deleted the s2d2/audit-examples-fail-loudly branch November 11, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants