Repository navigation
Upload: wait up to 6 h for the Cloud SQL CSV import (was 20 min) - #259
Merged
Merged
Conversation
On minnie (2026-10-25) a 337M-row upload's import ran 28.5 min and finished fine, but upload_to_database gave up waiting after 1200 s and failed the job, so the indexes, spatial lookup and transfer never ran. CSV processing used to take hours, so the import's wait was never the limit before. The default is now 21600 s (SQL_IMPORT_TIMEOUT_SECONDS still overrides it); upload_to_database is acked on receipt, so a long wait cannot be redelivered. The import logs its operation name when it starts, and a timeout says the operation may still be running and how to check it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On minnie on 2026-10-25, a 337M-row upload's Cloud SQL CSV import ran for 28.5 min and finished successfully. But
upload_to_databasestopped waiting after 1200 s (SQL_IMPORT_TIMEOUT_SECONDSdefault) and marked the job failed, so the indexes, spatial lookup and transfer never ran. Before #257, CSV processing took hours, so this wait was never the limiting step.SQL_IMPORT_TIMEOUT_SECONDSstill overrides it.upload_to_databaseis acked on receipt, so a long wait can't cause a redelivery.🤖 Generated with Claude Code