Skip to content

perf(cleanup): Optimize cleanup job - #9166

Merged
mejo- merged 1 commit into
mainfrom
feature/optimize-cleanup-job
Oct 13, 2026
Merged

mejo- merged 1 commit into
mainfrom
feature/optimize-cleanup-job

Conversation

@Koc

@Koc Koc commented Sep 3, 2026 •

Copy link
Copy Markdown

馃摑 Summary

This is continuation of the #8421.

Problem:

The text app cleanup background job took roughly 800 seconds per run. The loop over documents with no active session unconditionally called cleanupAttachments, even for documents that had unsaved changes and could not be reset. The same ~8,000 documents kept re-entering the cleanup loop every cron run, and cleanupAttachments is expensive because it resolves the file, reads Markdown content, scans the database, and iterates the attachment directory.

Solution:

... 馃毀

Impact:
... 馃毀

馃弫 Checklist

  • Code is properly formatted (npm run lint / npm run stylelint / composer run cs:check)
  • Sign-off message is added to all commits
  • Tests (unit, integration and/or end-to-end) passing and the changes are covered with tests
  • Documentation (README or documentation) has been updated or is not required

馃 AI (if applicable)

  • The content of this PR was partly or fully generated using AI tools
  • The AI-generated content was reviewed, comprehended and tested by a human

@Koc
Koc force-pushed the feature/optimize-cleanup-job branch 2 times, most recently from 41cd3d3 to 723e88c Compare September 13, 2026 14:38
@Koc
Koc marked this pull request as ready for review September 13, 2026 15:52
@Koc
Koc force-pushed the feature/optimize-cleanup-job branch from 723e88c to 0a05ff3 Compare September 14, 2026 21:47
@Koc
Koc force-pushed the feature/optimize-cleanup-job branch from 0a05ff3 to ddad96c Compare September 26, 2026 22:25
@Koc

Koc commented Sep 28, 2026

Copy link
Copy Markdown
Author

@max-nextcloud @mejo- mates, please take a look on this

Comment thread lib/Cron/Cleanup.php Outdated
@mejo-

mejo- commented Sep 29, 2026

Copy link
Copy Markdown
Member

I thought about this a bit more. The expensive task in the cleanup job is $this->attachmentService->cleanupAttachments($documentId), which runs for each document from the text_documents table without an active session each time. Since we introduced y-indexeddb, we no longer clean up text_documents on purpose.

One solution would be to only do the attachments cleanup if the last saved timestamp is newer than the last attachment cleanup run for that document. To do this, we would have to add a last_attachment_cleanup field to text_documents and then turn DocumentMapper::findAllWithNoActiveSessions() into findAllPendingAttachmentCleanup() which returns only documents with not active session and where last_saved_version_time > last_attachment_cleanup.

What do you think about this approach @Koc @max-nextcloud @blizzz?

@blizzz

blizzz commented Oct 9, 2026

Copy link
Copy Markdown
Member

What do you think about this approach @Koc @max-nextcloud @blizzz?

Sound reasonable to me 馃憤

@Koc
Koc force-pushed the feature/optimize-cleanup-job branch 4 times, most recently from c8c1c87 to 4ddaeb5 Compare October 11, 2026 23:59
@mejo-
mejo- force-pushed the feature/optimize-cleanup-job branch from 4ddaeb5 to c1759c6 Compare October 13, 2026 07:35

@mejo- mejo- left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great now. I pushed a small typo fix.

@mejo- mejo- self-assigned this Oct 13, 2026
@github-project-automation github-project-automation Bot moved this to 馃Л Planning evaluation (don't pick) in 馃摑 Productivity team Oct 13, 2026
@mejo- mejo- moved this from 馃Л Planning evaluation (don't pick) to 馃憖 In review in 馃摑 Productivity team Oct 13, 2026
@mejo-
mejo- force-pushed the feature/optimize-cleanup-job branch from c1759c6 to cf26231 Compare October 13, 2026 09:19
Comment thread lib/Cron/Cleanup.php
@mejo-
mejo- force-pushed the feature/optimize-cleanup-job branch from cf26231 to 8541262 Compare October 13, 2026 11:31
Comment thread lib/Cron/Cleanup.php Outdated
Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>
@mejo-
mejo- force-pushed the feature/optimize-cleanup-job branch from 8541262 to cb51bd1 Compare October 13, 2026 12:29
@mejo-
mejo- merged commit a8041db into main Oct 13, 2026
67 of 69 checks passed
@mejo-
mejo- deleted the feature/optimize-cleanup-job branch October 13, 2026 13:36
@github-project-automation github-project-automation Bot moved this from 馃憖 In review to 鈽戯笍 Done in 馃摑 Productivity team Oct 13, 2026
@Koc

Koc commented Oct 13, 2026

Copy link
Copy Markdown
Author

@mejo- thank you, Jonas! I鈥檓 very appreciative of everything what you鈥檙e doing! 馃憦

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants