Skip to content

Feature/wavelog sync pr - #1199

Open
twicklund76 wants to merge 4 commits into
accius:Stagingfrom
twicklund76:feature/wavelog-sync-pr
Open

twicklund76 wants to merge 4 commits into
accius:Stagingfrom
twicklund76:feature/wavelog-sync-pr

Conversation

@twicklund76

Copy link
Copy Markdown

What does this PR do?

Adds Wavelog pull synchronization to OpenHamClock's native logbook.

This complements the existing Wavelog/Cloudlog push integration by allowing OpenHamClock to import QSOs from Wavelog for native logbook features such as worked-before tracking and awards.

The integration:

  • Pulls QSOs incrementally from Wavelog using its ADIF export API
  • Supports one or more Wavelog station profiles
  • Stores imported QSOs in the native OpenHamClock logbook
  • Tracks the last fetched Wavelog QSO ID so subsequent pulls only request new contacts
  • Adds newly fetched grid-bearing QSOs to the existing Logged QSOs (API) map layer
  • Optionally uses MQTT events to trigger an immediate incremental pull when a QSO is logged in Wavelog
  • Uses SSE between the OpenHamClock server and browser; MQTT credentials remain server-side
  • Treats MQTT only as an event trigger; Wavelog remains the source of truth
  • Falls back cleanly to manual pull synchronization when MQTT is not configured
  • Performs an incremental catch-up when an existing synchronized client reconnects

The initial historical import remains an explicit user action via "Sync now". Automatic event synchronization only begins after a Wavelog cursor exists.

Native logbook persistence completes before the Wavelog cursor advances, so a failed local write cannot cause QSOs to be skipped on the next pull. Map-layer updates are best-effort and do not affect the synchronization cursor.

The optional MQTT configuration is documented in the Docker documentation and environment example.

Tested with Wavelog 3.0.1, including an initial 930-QSO import, subsequent incremental pulls, live event-triggered synchronization, map-layer updates, and reconnect catch-up.

Automated validation:

  • npm run test:run — 110 test files passed; 1631 tests passed, 3 skipped, 12 todo
  • npm run format:check — passed
  • npm run lang:check — passed
  • git diff --check — passed

Type of change

  • Bug fix
  • New feature
  • Performance improvement
  • Refactor / code cleanup
  • Documentation
  • Translation
  • Map layer plugin

How to test

  1. Configure a Wavelog URL and API key in Log Sync settings, enable "Pull QSOs from Wavelog into this logbook", click Test, and select one or more station profiles.
  2. Click Sync now and verify the QSOs are imported into the native logbook. Add another QSO in Wavelog and verify a subsequent sync only imports the new QSO.
  3. With the optional MQTT server configuration enabled, log a new QSO in Wavelog and verify it is imported automatically. For a QSO with a grid, enable Logged QSOs (API) and verify the contact appears on the map. Reconnecting an already-synchronized client should perform an incremental catch-up.

Checklist

  • App loads without console errors
  • Tested in Dark, Light, and Retro themes
  • Responsive at different screen sizes (desktop + mobile)
  • If touching server.js: caches have TTLs and size caps (we do not modify server.js)
  • If adding an API route: includes caching and error handling (the new route is an SSE event stream and intentionally sends Cache-Control: no-cache; optional MQTT configuration and disconnect/reconnect behavior are handled)
  • If adding a panel: wired into Modern, Classic, and Dockable layouts (no panel added)
  • No hardcoded colors — uses CSS variables (var(--accent-cyan), etc.)
  • No .bak, .old, console.log debug lines, or test scripts included

Screenshots (if visual change)

Wavelog pull configuration in Logbook Sync:
Wavelog pull configuration in Logbook Sync

@accius accius left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — this is a well-built addition. The separation of concerns is the right one: the browser owns the credentials and the cursor, the server only proxies and rings the MQTT doorbell, and the cursor advances only after the native logbook has persisted, with the strict rollback in addMany making a failed import safe to retry. The SSRF guard already covers the new pull endpoint, MQTT is already a dependency, tests are included and CI is green.

Two things I'd like changed before merging, and a few notes.

1. The first pull will time out on a big log. syncWavelogQsos asks for one page of 5000 QSOs through fetchWithTimeout, which uses the 20 s PUSH_TIMEOUT_MS. Wavelog generating 5000 ADIF records can take longer than that on a modest host, and a log larger than 5000 needs repeated "Sync now" clicks to walk the cursor forward (the hook loop only re-runs on events, not on a full page). Suggest: loop in syncWavelogQsos while exportedQsos === limit (cursor advances each page, so it is safe), and give the pull endpoint its own longer timeout (60 s?). A smaller page (1000–2000) would help too.

2. Update the header comment in server/routes/logsync.js (line 14) — it still documents POST …/api/station_info {key} while the code now uses GET …/api/station_info/<key>. The change itself is right: both Wavelog and Cloudlog document the GET form, so this quietly fixes the Test button. Worth a line in the PR description since it changes existing behaviour.

Notes, no change needed

  • connectWavelogMqtt is started lazily by the first SSE client and never torn down; fine for a long-running server, and the reconnect period handles broker restarts.
  • The SSE endpoint ends immediately when MQTT isn't configured, so on the hosted site this is one short request per page load for users with pull enabled. Good.
  • The API key goes into the URL for station_info; that's inherent to the Wavelog/Cloudlog API, but it will appear in their access logs. Nothing to do here.
  • Map-layer updates are best-effort and can't block the cursor — good call.

Once (1) and (2) are in I'll merge this to Staging for the next release. 73 de K0CJH

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants