fix(github): recheck in-progress workflow runs using low-water-mark lookback (#9125)

[Thundercloud12]

Summary

Problem

In cicd_run_collector.go, the collector explicitly skipped workflow runs that were not completed:

if run.Status != "completed" {
    continue
}

When runs were skipped while in_progress, CollectorStateManager.Close() still advanced LatestSuccessStart to until.

Because GitHub's /actions/runs API only filters by created date (created=<from>..<to>) and does not support filtering by updated, subsequent incremental syncs queried:

created=LatestSuccessStart+1s..until

Any long-running workflow that was created before LatestSuccessStart but completed later was therefore permanently missed.

Solution: Low-Water-Mark Lookback

1. Persist unfinished runs

   Removed the run.Status != "completed" skip in ResponseParser so non-terminal runs (in_progress, queued, waiting, etc.) are written to _raw_github_api_runs and extracted into _tool_github_runs.

2. Dynamic query start adjustment

   In CollectRuns, before bisecting leaf windows during incremental syncs, query _tool_github_runs for the oldest unfinished run:

   SELECT MIN(github_created_at)
   FROM _tool_github_runs
   WHERE connection_id = ?
     AND repo_id = ?
     AND status != 'completed'
     AND github_created_at >= ?

   If an unfinished run is found and its github_created_at predates windowStart, windowStart is moved back to re-include it in the GitHub API query range.

3. Preserve high-water mark

   CollectorStateManager.Close() continues to advance LatestSuccessStart = until.

   The shared database watermark therefore moves forward normally, and as soon as all unfinished runs complete, windowStart automatically snaps back to LatestSuccessStart + 1s.

4. Zombie run guardrail

   Added:

   maxUnfinishedRunLookback = 14 * 24 * time.Hour

   This ensures abandoned or killed runs cannot hold the collection window back indefinitely.

5. Downstream compatibility

   Downstream tasks (cicd_run_extractor, cicd_run_convertor, and cicd_job_collector) already support primary-key upserts and github_updated_at >= since.

   Therefore, completed runs and their jobs are converted and extracted automatically once the workflow status becomes completed.

Does this close any open issues?

Closes #9125

Screenshots

N/A (Backend collector logic fix)

Other Information

• Unit tests added in cicd_run_collector_test.go:

  • TestCicdRunLoadOldestUnfinishedRunCreatedAt_ReturnsOldestTimestamp
  • TestCicdRunLoadOldestUnfinishedRunCreatedAt_NotFoundReturnsNil
  • TestCicdRunLoadOldestUnfinishedRunCreatedAt_PropagatesError

• All unit tests in plugins/github/tasks pass.

• Note for existing deployments: Workflows that were skipped in previous DevLake versions were dropped before reaching _tool_github_runs. A one-time backfill with TimeAfter covering the affected window will be needed to capture those older runs. After that, the low-water-mark mechanism will track everything automatically.

[Thundercloud12]

@klesh hope this helps!