Skip to content

fix(ENG-902): send a cleared optional field as null on account edit - #250

Open
jpmiguel wants to merge 6 commits into
mainfrom
joao/eng-902-cleared-optional-field-values-are-retained-on
Open

jpmiguel wants to merge 6 commits into
mainfrom
joao/eng-902-cleared-optional-field-values-are-retained-on

Conversation

@jpmiguel

Copy link
Copy Markdown
Contributor

Summary

Fixes ENG-902: clearing an optional field when editing an account didn't stick. The relink PATCH dropped every empty field, the API merged the body onto the stored credentials, and the old value came back on the next edit (reproduced with Plaid and Tableau).

The API half is StackOneHQ/unified-cloud-api#9777, which removes a key the Hub sends as null.

This PR moves payload building out of handleConnect into a pure utils/credentialsPayload.ts:

  • On an edit (prefillAccount set), a non-secret field that the form showed with a value and the user emptied is sent as null.
  • "Showed with a value" comes from shownFieldValue. IntegrationFields' form defaults now use the same helper, so the two can't drift.
  • Unchanged:
    • an unopened secret placeholder is still left out, so the stored secret is kept
    • creating an account and the OAuth popup still omit empty fields (withoutClearedFields)

Acceptance criteria

  • Editing an account, emptying an optional non-secret field that showed a value, and saving sends the field as null, so the API removes it. Reopening edit shows it empty, or showing the connector default as on a fresh link.
  • A field the form showed empty is never sent as null.
  • Secrets are never cleared: neither an unopened placeholder nor one emptied by the Edit button.
  • After switching to another profile while editing, nothing is cleared, because nothing stored was shown.
  • Creating an account and the OAuth redirect still omit empty fields.
  • Either deploy order is safe: the old API drops the nulls, which is today's behaviour.

Sibling surfaces

Surface Decision
connectAccount (create) left: null-free filledCredentials, as before
OAuth relink popup URL deferred: a query string can't carry null; needs the cleared keys on the connection attempt (follow-up ticket)
IntegrationFields default values changed: shares shownFieldValue with the clear rule

Test plan

  • credentialsPayload.test.ts, 10 specs:
    • an emptied shown field is sent as null
    • a connector default (including a numeric 0) emptied is sent as null
    • a field shown empty is left out
    • an unopened placeholder is left out
    • an opened-and-emptied secret is left out
    • a secret showing a real value is never cleared
    • an opened placeholder is sent as is
    • an entered value is sent
    • new account or another profile: empty fields are left out
    • withoutClearedFields
  • Mutation-checked: dropping the secret guard, the shown-value guard or the editingSecrets guard fails a spec.
  • npm run lint, tsc --noEmit, npm test (58), npm run build + verify:build.

Review notes

A bounded review loop ran 3 rounds. These rows were dismissed or deferred:

  • Optional secrets can't be cleared: deliberate. The Edit button empties a secret before any typing, so "opened and left empty" can't be told apart from "cleared". The ticket's fields are all secret: false.
  • A field showing only a connector default, with nothing stored, is sent as null: intended, since the user emptied a value they saw. The API removes an absent key, which is a no-op, and never stores null.
  • No hook-level test for the prefillAccount choice: the Hub has no hook test harness. The editing: false case is specced in the util.
  • Specs have no arrange/act/assert markers: they match the neighbouring Hub specs (requestErrors.test.ts, zodSchema.test.ts).
  • Use RHF dirtyFields instead of re-deriving the shown value: a wider refactor. The shared shownFieldValue removes the duplication instead.
  • Deferred: the OAuth relink path, as above.

🤖 Generated with Claude Code

jpmiguel and others added 4 commits November 19, 2026 13:42
The relink PATCH dropped every empty field, so the API's merge kept the
stored value and the field came back on the next edit. An edit now sends a
cleared field as null, which the API removes. Creating an account and the
OAuth popup still omit empty fields, and an unopened secret placeholder is
still left out so the stored secret is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sending null for every empty field deleted values the user never saw: a
secret emptied by its Edit button, a stored value the prefill hides, and
everything after switching to another profile. Now only a non-secret field
whose stored value was shown and then emptied is sent as null.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-deriving "was it shown" from setupInformation copied the prefill rule and
missed a stored 0 shown as the connector default. The prefilled field
already carries what the form showed, so the payload reads that. Create goes
back to the null-free payload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The form's default values and the payload's "was it shown" check now come
from one shownFieldValue, so a numeric 0 the form displays as "0" can be
cleared too. Secret placeholders are stripped whenever present, since they
only ever arrive through the prefill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jpmiguel

Copy link
Copy Markdown
Contributor Author

@s2d2-agent can you review the code?

@s2d2-agent

s2d2-agent Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Done in 2m 6s · 14 tool calls · answer

@s2d2-agent

s2d2-agent Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Approve-worthy as written: I found no blockers. I checked how the hook, the form and the API half (StackOneHQ/unified-cloud-api#9777) fit together, and the PR's claims hold. CI is green on d6ec9ca.

Checked and correct

  • Hub and API agree on what counts as a secret. isSecretField (secret !== false || type === 'password') matches the API's new isSetupField (secret === false). Fields with no secret flag are treated as secrets on both sides.
  • The edit flag and the field values always match. editing: !!prefillAccount is computed in the same render as the fields memo, which also builds from prefillAccount. After a switch to another profile, the flag is false and the fields have no stored values, so nothing can be sent as null.
  • Either deploy order is safe. On current unified-cloud-api main, PatchAccountDto.credentials doesn't validate values. HubService.placeholderToSecrets keeps only notMissing(v), so the old API drops a null before the merge, which is today's behaviour.
  • The nulls only go where they should. They reach updateAccount, which takes Record<string, unknown>. connectAccount and the OAuth URL get filledCredentials.
  • Moving the secret guard is fine. Untouched placeholders used to be stripped only when accountData was set; now they are always stripped. Placeholders only show up when prefillAccount is set, so behaviour doesn't change.
  • The shared helper stops the two copies drifting. IntegrationFields defaults and the clear rule now both use shownFieldValue.

Nits (optional)

  • emptiedKeys doesn't check required. Zod validation stops a required field being submitted empty, and the API also limits clears to required !== true, so this is safe today. Adding field.required !== true (and possibly !field.readOnly) would make the Hub rule match the API's exactly, instead of relying on the form. It only matters if a rule ever makes required conditional.
                    (field) =>
                        !isSecretField(field) &&
                        field.required !== true &&
                        shownFieldValue(field) !== '' &&
                        formData[field.key] === '',
    
  • Link the OAuth relink follow-up ticket. The "Sibling surfaces" table says the OAuth relink needs a follow-up ticket but doesn't link one. Until it ships, a cleared field on an oauth2 relink is still silently kept.
  • Unrelated bug, separate ticket (not a blocker). In useIntegrationPicker.ts, if (!prefilledValue) return { ...field } means a stored value of 0 or '' shows the connector default instead. This PR then sends null if the user empties that field. The API removes the key, so the end state is correct, but the form shows something other than what is stored.

Merge order

#9777 is still open. Merging this first is safe, but the fix only works for users once the API PR is deployed.

The API only clears optional fields; the hub now applies the same rule
instead of relying on form validation to keep a required field from going
out empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jpmiguel

Copy link
Copy Markdown
Contributor Author

Thanks. Taking each nit in turn:

  • required in emptiedKeys: applied in 9745845, with a spec ("never clears a required field, leaving the stored value to the API"). The Hub rule now matches the API's isSetupField && required !== true. I left readOnly out: a readOnly input can't be emptied by the user, so it never reaches the rule.
  • OAuth relink follow-up: agreed it needs a linked ticket. It isn't filed yet, and I'll link it here once it is.
  • Prefill hiding a stored 0/'': agreed it's a separate, pre-existing display bug, and out of scope here. As you note, the end state is correct, since emptying the field the user sees removes the key.

let windowUrl = `${baseUrl}/connect/oauth2/${selectedIntegration.integration_id}?redirect_uri=${callbackEmbeddedAccountsUrl}&token=${token}&connection_attempt_id=${attemptId}`;
Object.keys(cleanedFormData).forEach((key) => {
windowUrl += `&${key}=${encodeURIComponent(cleanedFormData[key])}`;
Object.keys(filledCredentials).forEach((key) => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On OAuth2 accounts a cleared optional field is still kept. filledCredentials has the null keys stripped, so the popup URL carries nothing for a field the user emptied, and the account keeps the old value. For example, with region=eu stored, emptying region and reconnecting leaves region=eu.

Could we pass cleared keys through the URL too (an empty value the API reads as removal), or call out in the PR that OAuth edits are out of scope for ENG-902?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope for ENG-902, as the PR body's "Sibling surfaces" table says (deferred). An empty query param alone wouldn't fix it: the OAuth callback merges the URL params onto the stored credentials, the same way the PATCH did. So removal needs an API change on the /connect/oauth2 callback too, ideally carrying the cleared keys on the connection attempt rather than in the URL. That's a separate change across hub and uc-api, so it'll get its own follow-up ticket, linked here once it's filed.

export const shownFieldValue = (field: ConnectorConfigField): string =>
field.value === undefined ? '' : field.value.toString();

const isSecretField = (field: ConnectorConfigField) =>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional secrets still can't be cleared. emptiedKeys skips every field where isSecretField holds, so if the user clicks Edit on an optional secret (key joins editingSecrets) and saves it empty, the key is left out and the API keeps the stored secret. Same for a type: 'password' field with secret: false.

Should an optional secret in editingSecrets that is left empty go out as null? If not clearing secrets is intended, worth noting on ENG-902.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not clearing secrets is intentional. The Edit button on a secret runs setValue(key, '') and adds the key to editingSecrets before the user types anything, and there's no cancel. So "opened and left empty" looks exactly like "deliberately cleared". Sending null there would turn one stray click into a deleted credential, which is worse than not supporting the clear. The API side refuses it as well (uc-api#9777 only clears optional secret: false fields), and logs the refused key names.

A type: 'password' field with secret: false is treated as a secret here because the hub prefills it as one (field.secret !== false || field.type === 'password' in the prefill), so it gets the same Edit-button behaviour. Agreed this belongs on ENG-902; I'll get it noted there.

field.value === undefined ? '' : field.value.toString();

const isSecretField = (field: ConnectorConfigField) =>
field.secret !== false || field.type === 'password';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does /connector config always send secret as a boolean? With secret !== false, a field that omits it counts as a secret, so it can never be cleared. Every fixture sets secret: false, so the tests don't cover that case. If it can be missing, secret === true || type === 'password' plus a fixture without secret would close it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A field with no secret flag is a secret everywhere else, so it shouldn't be clearable as a setup field:

  • The API's partitionSecrets stores it in the secret bundle (!isSetupField, where isSetupField is secret === false).
  • The hub prefill reads it from prefillAccount.secrets (field.secret !== false || field.type === 'password').

Switching to secret === true would send null for a field the API keeps as a secret, and the API refuses that clear anyway. I kept the rule and added the fixture you suggested in 9407133: "treats a field without a secret flag as a secret, as the API stores it".

it('sends a field showing a connector default the user emptied as null', () => {
const actual = buildCredentialsPayload({
formData: { apiUrl: 'https://api.example.com', timeout: '' },
fields: [...shownFields, field({ key: 'timeout', type: 'number', value: 0 })],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hook never produces this state. The prefill in useIntegrationPicker does if (!prefilledValue) return { ...field } (around line 274, and again in the legacy branch near 341), so a 0 default reaches the form with no value, shows empty, and emptying it sends nothing. The test passes while the real flow doesn't clear the field.

Either use a truthy default like '30' here, or change the prefill check to prefilledValue === undefined || prefilledValue === '' and keep the test. That check predates this PR, but it also hides stored falsy values on edit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That state is reachable for a connector default, as opposed to a stored value. When nothing is stored, prefilledValue is the config's field.value. With 0, !prefilledValue takes the return { ...field, key: field.key } branch, and the spread keeps the config's own value: 0. IntegrationFields' defaults render that as "0" through shownFieldValue, so the user sees "0", empties it, and null goes out. That's what the test covers.

What you're describing is the stored-value case: a stored 0 or '' with a truthy default shows the default, because !prefilledValue falls back to { ...field }. That's real but predates this PR. s2d2 flagged it too, and it's being kept as a separate ticket. The end state is still correct, since emptying what the user sees removes the key.

The API stores a field with no `secret` flag in the secret bundle, and the
hub prefills it from secrets, so the clear rule treats it as a secret too.
A fixture without the flag now pins that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants