diff --git a/docs/ui-ux-improvements.md b/docs/ui-ux-improvements.md
new file mode 100644
index 0000000..5c8bcea
--- /dev/null
+++ b/docs/ui-ux-improvements.md
@@ -0,0 +1,79 @@
+# UI/UX improvement backlog
+
+Suggestions collected from users. Items marked **done** ship in `feat/map-ux-improvements`; the rest are open.
+
+Effort scale: **S** = small, **M** = medium, **L** = large.
+
+## Done
+
+| Source | Item |
+|---|---|
+| user1 #1 | Easier line selection on map (canvas renderer, 8 px click tolerance) |
+| user1 #6 | Map scale bar |
+| user1 #3 (map part), user2 | Length, frequency and polarization in the map popup |
+| user2 | Default Grafana refresh set to 1 min |
+| user1 #2 (partly) | Choosing a CML in the Grafana dropdown opens its map popup (user and admin map) and highlights the line; the map pans only if the line is out of view |
+| user1 #3, user2 | Metadata in the admin map popup (replaces the hover tooltip) |
+
+## Open
+
+### 1. Always zoom to the CML chosen in the dropdown (user1 #2)
+
+- **Remaining:** the map only pans when the selected line is out of view; optionally zoom with `map.fitBounds()` instead.
+- **Effort:** S.
+
+### 2. Show length and frequency band above the graph (user1 #3)
+
+- **To do:**
+ - Option A: add a Grafana text/stat panel or change the panel title using template variables backed by a `cml_metadata` query (`length`, `frequency`, `polarization` for `$cml_id`).
+ - Option B: render a small header bar in `realtime.html` above the iframe and fill it from `cmlMeta` when a CML is selected (no Grafana change; pairs with item 1).
+ - The iframe uses `viewPanel=2&kiosk`, so a Grafana-side change only shows up if it is part of panel 2's title. Option B is simpler.
+- **Effort:** S (option B) / M (option A).
+
+### 3. Compare two CMLs (user1 #4)
+
+- **To do:**
+ - Make the Grafana `cml_id` variable multi-value and let the panel query return one series per CML (colour-coded), or add a second iframe stacked below for a split view.
+ - Add map UI to pick a second CML (e.g. shift+click) and a toggle for merged vs. split.
+ - Make sure the time sync and the grey coloring-window annotation (`/api/coloring-annotation`) apply to both panels.
+ - Update the dashboard provisioning JSON and the SQL used by the panel (`cml_data_secure`).
+- **Effort:** L. Decide first on merged vs. split layout.
+
+### 4. Valid-record counts for 1 day and 3 days in the popup (user1 #5)
+
+- **To do:**
+ - Add `total_records_1d`, `valid_records_1d`, `total_records_3d`, `valid_records_3d` to `cml_stats` and `cml_stats_history` (new migration in `database/migrations/`, plus `init.sql`).
+ - Extend `update_cml_stats()` and `get_cml_stats_at()`. Keep RLS and `user_id` handling consistent.
+ - Add the fields to `/api/cml-stats` (live and historical column order) and to `buildPopupText()`.
+ - Update `parser/backfill_stats_history.py` and the tests for the stats API.
+ - Check the cost of the 3 d window on `cml_data` (compressed chunks); consider computing from `cml_data_1h` instead.
+- **Effort:** M. Database and API changes with a migration.
+
+### 5. Search and display by link number (user2)
+
+- **Request:** search by link number, i.e. IDs with `55` at the 3rd and 4th digit (e.g. `30655 1234`); IDs with `25` or `27` at that position are unwanted.
+- **Needs clarification:** is the `55` ID a separate field from the current `cml_id`, or a transformation of it? Which IDs should be hidden or mapped?
+- **To do:**
+ - Add a search box to the map (filter or jump to a CML, zoom via item 1).
+ - If the 55-number is a different identifier, add a column to `cml_metadata` (migration, parser mapping, `db_writer.py`) and show it in the popup and the search.
+ - If only a filter is needed, filter or flag the list server-side in `get_available_cmls()` and `/api/cml-map`.
+- **Effort:** S (search by existing ID) / M (new identifier column through the pipeline).
+
+### 6. Sublink numbers (network element numbers with 55) (user2)
+
+- **Needs clarification:** the source of these numbers. `cml_metadata.sublink_id` currently holds names such as `sublink_1`.
+- **To do:** add a network-element ID per sublink in the metadata (migration, parser mappings, `db_writer.py`), then show it in the popup next to frequency and polarization and make it searchable (item 5).
+- **Effort:** M. Depends on what the provider data contains.
+
+### 7. Grey area for periods with possible rain-related outage (user2)
+
+- **Needs clarification:** what defines such a period (a wet/dry flag from the processing, rain gauge or radar data, or a RSL-drop threshold)? Is it per CML or for the whole area?
+- **To do:**
+ - Define and compute the rain-event periods (processor service or SQL), and store them (table or Grafana annotations).
+ - Draw them as shaded regions in the Grafana panel, e.g. via an annotation query on the dashboard, similar to the existing coloring-window annotation.
+ - Optionally mark affected CMLs on the map.
+- **Effort:** L. Mostly a data and algorithm question; the display part is S-M.
+
+## Follow-ups on done items
+
+- Confirm units in `cml_metadata`: the popup assumes `length` in metres and `frequency` in MHz (as in the archive CSV). The `other_mno_csv` parser maps `length_km` straight into `length`, so that source may be stored in km. Normalize in the parser if so. Effort: S.
diff --git a/grafana/admin_dashboards/cml-admin-realtime.json b/grafana/admin_dashboards/cml-admin-realtime.json
index 4e597f3..d4e63cc 100644
--- a/grafana/admin_dashboards/cml-admin-realtime.json
+++ b/grafana/admin_dashboards/cml-admin-realtime.json
@@ -4,7 +4,7 @@
"graphTooltip": 0,
"links": [],
"liveNow": false,
- "refresh": "10s",
+ "refresh": "1m",
"schemaVersion": 38,
"style": "dark",
"tags": [
diff --git a/grafana/provisioning/dashboards/definitions/cml-realtime.json b/grafana/provisioning/dashboards/definitions/cml-realtime.json
index bea7728..71450de 100644
--- a/grafana/provisioning/dashboards/definitions/cml-realtime.json
+++ b/grafana/provisioning/dashboards/definitions/cml-realtime.json
@@ -874,7 +874,7 @@
"type": "geomap"
}
],
- "refresh": "10s",
+ "refresh": "1m",
"schemaVersion": 38,
"style": "dark",
"tags": [
diff --git a/webserver/main.py b/webserver/main.py
index f03addb..63d30d7 100644
--- a/webserver/main.py
+++ b/webserver/main.py
@@ -537,7 +537,17 @@ def api_cml_map():
with user_db_scope(current_user.id) as conn:
cur = conn.cursor()
cur.execute(
- "SELECT DISTINCT ON (cml_id) cml_id::text, site_0_lon, site_0_lat, site_1_lon, site_1_lat FROM cml_metadata ORDER BY cml_id"
+ """
+ SELECT cml_id::text,
+ MIN(site_0_lon), MIN(site_0_lat), MIN(site_1_lon), MIN(site_1_lat),
+ MIN(length),
+ json_agg(json_build_object('frequency', frequency,
+ 'polarization', polarization)
+ ORDER BY sublink_id)
+ FROM cml_metadata
+ GROUP BY cml_id
+ ORDER BY cml_id
+ """
)
data = cur.fetchall()
cur.close()
@@ -547,6 +557,9 @@ def api_cml_map():
"cml_id": str(row[0]),
"site_0": {"lon": float(row[1]), "lat": float(row[2])},
"site_1": {"lon": float(row[3]), "lat": float(row[4])},
+ # length in metres, sublink frequency in MHz
+ "length": safe_float(row[5]),
+ "sublinks": row[6] or [],
}
for row in data
]
@@ -911,11 +924,16 @@ def api_admin_cml_map():
with admin_db_scope() as conn:
cur = conn.cursor()
cur.execute(
- """SELECT DISTINCT ON (user_id, cml_id)
- user_id, cml_id::text, site_0_lon, site_0_lat, site_1_lon, site_1_lat
+ """SELECT user_id, cml_id::text,
+ MIN(site_0_lon), MIN(site_0_lat), MIN(site_1_lon), MIN(site_1_lat),
+ MIN(length),
+ json_agg(json_build_object('frequency', frequency,
+ 'polarization', polarization)
+ ORDER BY sublink_id)
FROM cml_metadata
WHERE site_0_lon IS NOT NULL AND site_0_lat IS NOT NULL
AND site_1_lon IS NOT NULL AND site_1_lat IS NOT NULL
+ GROUP BY user_id, cml_id
ORDER BY user_id, cml_id"""
)
rows = cur.fetchall()
@@ -931,6 +949,9 @@ def api_admin_cml_map():
"cml_id": str(row[1]),
"site_0": {"lon": float(row[2]), "lat": float(row[3])},
"site_1": {"lon": float(row[4]), "lat": float(row[5])},
+ # length in metres, sublink frequency in MHz
+ "length": safe_float(row[6]),
+ "sublinks": row[7] or [],
}
for row in rows
]
diff --git a/webserver/templates/admin_realtime.html b/webserver/templates/admin_realtime.html
index 8b71dc8..21f4932 100644
--- a/webserver/templates/admin_realtime.html
+++ b/webserver/templates/admin_realtime.html
@@ -51,6 +51,12 @@
height: 100%;
}
+ /* Place the scale bar beside the zoom buttons instead of below them */
+ .leaflet-top.leaflet-left .leaflet-control-scale {
+ clear: none;
+ margin-left: 8px;
+ }
+
.legend-panel {
position: absolute;
top: 10px;
@@ -156,7 +162,7 @@
{% endif %}
@@ -176,6 +182,7 @@
});
const map = L.map('admin-map', { preferCanvas: true }).setView([50, 10], 5);
+ L.control.scale({ position: 'topleft', metric: true, imperial: false }).addTo(map);
// Esri World Light Gray Canvas - keyless light basemap (tile order is z/y/x)
L.tileLayer('https://server.arcgisonline.com/ArcGIS/rest/services/Canvas/World_Light_Gray_Base/MapServer/tile/{z}/{y}/{x}', {
attribution: 'Tiles © Esri', maxZoom: 16
@@ -223,9 +230,16 @@
function tooltipText(c) {
const s = stats[c.user_id + '|' + c.cml_id];
- let t = esc(providerNames[c.user_id] || c.user_id) + ' · CML ' + esc(c.cml_id);
- if (!s) return t + '
no data';
- t += '
Completeness (6h): ' + s.completeness_percent + '%';
+ let t = '' + esc(providerNames[c.user_id] || c.user_id) + ' · CML ' + esc(c.cml_id) + '
';
+ if (c.length !== null) t += 'Length: ' + (c.length / 1000).toFixed(2) + ' km
';
+ c.sublinks.forEach(sl => {
+ const parts = [];
+ if (sl.frequency !== null) parts.push((sl.frequency / 1000).toFixed(1) + ' GHz');
+ if (sl.polarization) parts.push(esc(sl.polarization.toUpperCase()));
+ if (parts.length) t += 'Sublink: ' + parts.join(' ') + '
';
+ });
+ if (!s) return t + 'no data';
+ t += 'Completeness (6h): ' + s.completeness_percent + '%';
if (s.last_rsl !== null) t += '
Last RSL: ' + s.last_rsl + ' dBm';
if (s.stddev_last_60min !== null) t += '
Std Dev (1h): ' + s.stddev_last_60min;
return t;
@@ -235,7 +249,7 @@
Object.keys(lines).forEach(key => {
const line = lines[key];
line.setStyle({ color: lineColor(line._provider, key) });
- line.setTooltipContent(tooltipText(line._cml));
+ line.setPopupContent(tooltipText(line._cml));
});
}
@@ -278,12 +292,17 @@
});
}
- // Select a CML and point the Grafana iframe at it
- function selectCml(providerId, cmlId) {
- const key = providerId + '|' + cmlId;
+ // Highlight a CML on the map
+ function highlightCml(key) {
if (selectedKey && lines[selectedKey]) lines[selectedKey].setStyle({ weight: 2.5 });
selectedKey = key;
lines[key].setStyle({ weight: 6 }).bringToFront();
+ }
+
+ // Select a CML and point the Grafana iframe at it
+ function selectCml(providerId, cmlId) {
+ const key = providerId + '|' + cmlId;
+ highlightCml(key);
const panel = document.getElementById('grafana-panel');
if (!panel) return;
@@ -295,6 +314,49 @@
win.dispatchEvent(new PopStateEvent('popstate', { state: null }));
}
+ // Open the popup of a CML, first bringing it into the part of the map not covered by the chart
+ function showCmlPopup(key) {
+ const line = lines[key];
+ if (!line || !line._map) return;
+
+ const chart = document.querySelector('.chart-container');
+ const chartHeight = chart ? chart.offsetHeight : 0;
+ const size = map.getSize();
+ const p = map.latLngToContainerPoint(line.getCenter());
+ const visible = p.x > 0 && p.x < size.x && p.y > 120 && p.y < size.y - chartHeight - 40;
+ if (!visible) {
+ map.fitBounds(line.getBounds(), {
+ paddingTopLeft: [60, 120],
+ paddingBottomRight: [60, Math.max(60, chartHeight + 40)],
+ maxZoom: map.getZoom(),
+ animate: false
+ });
+ }
+ line.openPopup();
+ }
+
+ // The provider/CML dropdowns live inside the Grafana iframe; poll its URL to react to changes
+ function watchGrafanaSelection() {
+ const panel = document.getElementById('grafana-panel');
+ if (!panel) return;
+ let primed = false;
+ setInterval(() => {
+ let params;
+ try {
+ params = new URL(panel.contentWindow.location.href).searchParams;
+ } catch (e) {
+ return;
+ }
+ const key = params.get('var-provider') + '|' + params.get('var-cml_id');
+ // Skip until both variables match a CML on the map (cml_id lags behind a provider change)
+ if (key === selectedKey || !lines[key]) return;
+ highlightCml(key);
+ // The first match is the dashboard's initial selection: highlight only, no popup
+ if (primed) showCmlPopup(key);
+ primed = true;
+ }, 500);
+ }
+
fetch('/api/admin/cml-map')
.then(r => { if (!r.ok) throw new Error('HTTP ' + r.status); return r.json(); })
.then(cmls => {
@@ -305,7 +367,7 @@
if (!layers[c.user_id]) layers[c.user_id] = L.featureGroup().addTo(map);
counts[c.user_id] = (counts[c.user_id] || 0) + 1;
const line = L.polyline([a, b], { color: NO_DATA, weight: 2.5, opacity: 0.85 })
- .bindTooltip(tooltipText(c), { sticky: true })
+ .bindPopup(tooltipText(c), { autoPan: false })
.on('click', () => selectCml(c.user_id, c.cml_id))
.addTo(layers[c.user_id]);
line._provider = c.user_id;
@@ -319,6 +381,7 @@
renderLegend();
loadStats();
setInterval(loadStats, STATS_REFRESH_MS);
+ watchGrafanaSelection();
})
.catch(err => {
document.getElementById('map-status').textContent = 'Failed to load CMLs (' + err.message + ')';
diff --git a/webserver/templates/realtime.html b/webserver/templates/realtime.html
index 929ab6c..796a6ce 100644
--- a/webserver/templates/realtime.html
+++ b/webserver/templates/realtime.html
@@ -51,6 +51,12 @@
height: 100%;
}
+ /* Place the scale bar beside the zoom buttons instead of below them */
+ .leaflet-top.leaflet-left .leaflet-control-scale {
+ clear: none;
+ margin-left: 8px;
+ }
+
.chart-container {
position: absolute;
bottom: 0;
@@ -104,7 +110,7 @@
@@ -116,7 +122,11 @@
let map;
let cmlLayers = {};
let cmlStats = {};
+ let cmlMeta = {};
let selectedCmlId = '10001';
+ let highlightedCmlId = null;
+ const DEFAULT_STYLE = { weight: 2, opacity: 0.7 };
+ const SELECTED_STYLE = { weight: 5, opacity: 1 };
let currentColoringOption = 'std_dev';
// Time slider state
@@ -181,8 +191,26 @@
};
}
+ function escapeHtml(s) {
+ var d = document.createElement('div');
+ d.textContent = String(s);
+ return d.innerHTML;
+ }
+
function buildPopupText(cmlId, stats) {
- var popupText = 'CML ' + cmlId + '
';
+ var popupText = 'CML ' + escapeHtml(cmlId) + '
';
+ var meta = cmlMeta[cmlId];
+ if (meta) {
+ if (meta.length !== null) {
+ popupText += 'Length: ' + (meta.length / 1000).toFixed(2) + ' km
';
+ }
+ meta.sublinks.forEach(function (s) {
+ var parts = [];
+ if (s.frequency !== null) parts.push((s.frequency / 1000).toFixed(1) + ' GHz');
+ if (s.polarization) parts.push(escapeHtml(s.polarization.toUpperCase()));
+ if (parts.length) popupText += 'Sublink: ' + parts.join(' ') + '
';
+ });
+ }
if (stats) {
popupText += 'Completeness (6h): ' + stats.completeness_percent + '%
';
popupText += 'Valid records (6h): ' + stats.valid_records + '/' + stats.total_records + '
';
@@ -215,7 +243,10 @@
function initializeMap() {
console.log('Initializing map...');
try {
- map = L.map('cml-map').setView([57.705, 11.97], 11);
+ // Canvas renderer with tolerance makes thin CML lines easier to hit
+ map = L.map('cml-map', { renderer: L.canvas({ tolerance: 8 }) }).setView([57.705, 11.97], 11);
+ // Top-left: the default bottom-left spot is hidden behind the chart panel
+ L.control.scale({ position: 'topleft', metric: true, imperial: false }).addTo(map);
// Use Carto Positron - clean, minimal light basemap
L.tileLayer('https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png', {
attribution: '© OpenStreetMap contributors © CARTO',
@@ -492,6 +523,7 @@
var cmlId = cml.cml_id;
var stats = cmlStats[cmlId];
var color = getColor(stats, currentColoringOption);
+ cmlMeta[cmlId] = { length: cml.length, sublinks: cml.sublinks || [] };
var latlngs = [
[cml.site_0.lat, cml.site_0.lon],
@@ -517,15 +549,31 @@
selectCml(cmlId);
});
- // Add hover effects
+ // Add hover effects (the selected line keeps its highlight)
polyline.on('mouseover', function () {
- this.setStyle({ weight: 4, opacity: 1 });
+ this.setStyle({ weight: this._selected ? 6 : 4, opacity: 1 });
});
polyline.on('mouseout', function () {
- this.setStyle({ weight: 2, opacity: 0.7 });
+ this.setStyle(this._selected ? SELECTED_STYLE : DEFAULT_STYLE);
});
cmlLayers[cmlId] = polyline;
+ if (cmlId === selectedCmlId) highlightCml(cmlId);
+ }
+
+ // Thicker, opaque line for the selected CML
+ function highlightCml(cmlId) {
+ var prev = cmlLayers[highlightedCmlId];
+ if (prev) {
+ prev._selected = false;
+ prev.setStyle(DEFAULT_STYLE);
+ }
+ var line = cmlLayers[cmlId];
+ if (!line) return;
+ line._selected = true;
+ line.setStyle(SELECTED_STYLE);
+ line.bringToFront();
+ highlightedCmlId = cmlId;
}
// Update all map line colors based on current coloring option
@@ -729,6 +777,7 @@
// Select CML and update Grafana dashboard
function selectCml(cmlId) {
selectedCmlId = cmlId;
+ highlightCml(cmlId);
var grafanaPanel = document.getElementById('grafana-panel');
var iframeWindow = grafanaPanel.contentWindow;
@@ -739,11 +788,50 @@
iframeWindow.dispatchEvent(new PopStateEvent('popstate', { state: null }));
}
+ // Open the popup of a CML, first bringing it into the part of the map not covered by the chart
+ function showCmlPopup(cmlId) {
+ var line = cmlLayers[cmlId];
+ if (!line) return;
+
+ highlightCml(cmlId);
+ var pad = getMapViewportPadding();
+ var size = map.getSize();
+ var p = map.latLngToContainerPoint(line.getCenter());
+ var visible = p.x > 0 && p.x < size.x && p.y > 120 && p.y < size.y - pad.bottomRight[1];
+ if (!visible) {
+ map.fitBounds(line.getBounds(), {
+ paddingTopLeft: [60, 120],
+ paddingBottomRight: pad.bottomRight,
+ maxZoom: map.getZoom(),
+ animate: false
+ });
+ }
+ line.openPopup();
+ }
+
+ // The CML dropdown lives inside the Grafana iframe; poll its URL to react to changes
+ function watchGrafanaSelection() {
+ setInterval(function () {
+ var id;
+ try {
+ var panel = document.getElementById('grafana-panel');
+ id = new URL(panel.contentWindow.location.href).searchParams.get('var-cml_id');
+ } catch (e) {
+ return;
+ }
+ if (id && id !== selectedCmlId) {
+ selectedCmlId = id;
+ showCmlPopup(id);
+ }
+ }, 500);
+ }
+
// Initialize on page load
document.addEventListener('DOMContentLoaded', function () {
initializeMap();
initializeResizer();
cleanupColoringAnnotations();
+ watchGrafanaSelection();
});
// Initialize resizer for adjustable dashboard height
diff --git a/webserver/tests/test_admin.py b/webserver/tests/test_admin.py
index cc00172..89d55f6 100644
--- a/webserver/tests/test_admin.py
+++ b/webserver/tests/test_admin.py
@@ -120,15 +120,20 @@ def test_admin_realtime_without_grafana_org_has_map_only(users):
def test_admin_cml_map_returns_all_providers(users, cursor):
+ sublinks = [{"frequency": 28000.0, "polarization": "v"}]
cursor.fetchall.return_value = [
- ("tenant_a", "C1", 10.0, 50.0, 10.1, 50.1),
- ("tenant_b", "C1", 11.0, 51.0, 11.1, 51.1),
+ ("tenant_a", "C1", 10.0, 50.0, 10.1, 50.1, 691.4, sublinks),
+ ("tenant_b", "C1", 11.0, 51.0, 11.1, 51.1, None, None),
]
resp = _login("boss").get("/api/admin/cml-map")
assert resp.status_code == 200
data = resp.get_json()
assert [d["user_id"] for d in data] == ["tenant_a", "tenant_b"]
assert data[0]["site_0"] == {"lon": 10.0, "lat": 50.0}
+ assert data[0]["length"] == 691.4
+ assert data[0]["sublinks"] == sublinks
+ assert data[1]["length"] is None
+ assert data[1]["sublinks"] == []
def test_admin_cml_stats_returns_all_providers(users, cursor):
diff --git a/webserver/tests/test_api_routes.py b/webserver/tests/test_api_routes.py
index 40485b0..c826399 100644
--- a/webserver/tests/test_api_routes.py
+++ b/webserver/tests/test_api_routes.py
@@ -56,13 +56,16 @@ def test_api_cml_metadata_empty(auth_client):
def test_api_cml_map_returns_list(auth_client):
client, cursor = auth_client
- cursor.fetchall.return_value = [("CML01", 10.0, 5.0, 10.5, 5.5)]
+ sublinks = [{"frequency": 28000.0, "polarization": "v"}]
+ cursor.fetchall.return_value = [("CML01", 10.0, 5.0, 10.5, 5.5, 691.4, sublinks)]
resp = client.get("/api/cml-map")
assert resp.status_code == 200
data = resp.get_json()
assert data[0]["cml_id"] == "CML01"
assert data[0]["site_0"] == {"lon": 10.0, "lat": 5.0}
assert data[0]["site_1"] == {"lon": 10.5, "lat": 5.5}
+ assert data[0]["length"] == 691.4
+ assert data[0]["sublinks"] == sublinks
def test_api_data_time_range_with_data(auth_client):