Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions docs/ui-ux-improvements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# UI/UX improvement backlog

Suggestions collected from users. Items marked **done** ship in `feat/map-ux-improvements`; the rest are open.

Effort scale: **S** = small, **M** = medium, **L** = large.

## Done

| Source | Item |
|---|---|
| user1 #1 | Easier line selection on map (canvas renderer, 8 px click tolerance) |
| user1 #6 | Map scale bar |
| user1 #3 (map part), user2 | Length, frequency and polarization in the map popup |
| user2 | Default Grafana refresh set to 1 min |
| user1 #2 (partly) | Choosing a CML in the Grafana dropdown opens its map popup (user and admin map) and highlights the line; the map pans only if the line is out of view |
| user1 #3, user2 | Metadata in the admin map popup (replaces the hover tooltip) |

## Open

### 1. Always zoom to the CML chosen in the dropdown (user1 #2)

- **Remaining:** the map only pans when the selected line is out of view; optionally zoom with `map.fitBounds()` instead.
- **Effort:** S.

### 2. Show length and frequency band above the graph (user1 #3)

- **To do:**
- Option A: add a Grafana text/stat panel or change the panel title using template variables backed by a `cml_metadata` query (`length`, `frequency`, `polarization` for `$cml_id`).
- Option B: render a small header bar in `realtime.html` above the iframe and fill it from `cmlMeta` when a CML is selected (no Grafana change; pairs with item 1).
- The iframe uses `viewPanel=2&kiosk`, so a Grafana-side change only shows up if it is part of panel 2's title. Option B is simpler.
- **Effort:** S (option B) / M (option A).

### 3. Compare two CMLs (user1 #4)

- **To do:**
- Make the Grafana `cml_id` variable multi-value and let the panel query return one series per CML (colour-coded), or add a second iframe stacked below for a split view.
- Add map UI to pick a second CML (e.g. shift+click) and a toggle for merged vs. split.
- Make sure the time sync and the grey coloring-window annotation (`/api/coloring-annotation`) apply to both panels.
- Update the dashboard provisioning JSON and the SQL used by the panel (`cml_data_secure`).
- **Effort:** L. Decide first on merged vs. split layout.

### 4. Valid-record counts for 1 day and 3 days in the popup (user1 #5)

- **To do:**
- Add `total_records_1d`, `valid_records_1d`, `total_records_3d`, `valid_records_3d` to `cml_stats` and `cml_stats_history` (new migration in `database/migrations/`, plus `init.sql`).
- Extend `update_cml_stats()` and `get_cml_stats_at()`. Keep RLS and `user_id` handling consistent.
- Add the fields to `/api/cml-stats` (live and historical column order) and to `buildPopupText()`.
- Update `parser/backfill_stats_history.py` and the tests for the stats API.
- Check the cost of the 3 d window on `cml_data` (compressed chunks); consider computing from `cml_data_1h` instead.
- **Effort:** M. Database and API changes with a migration.

### 5. Search and display by link number (user2)

- **Request:** search by link number, i.e. IDs with `55` at the 3rd and 4th digit (e.g. `30655 1234`); IDs with `25` or `27` at that position are unwanted.
- **Needs clarification:** is the `55` ID a separate field from the current `cml_id`, or a transformation of it? Which IDs should be hidden or mapped?
- **To do:**
- Add a search box to the map (filter or jump to a CML, zoom via item 1).
- If the 55-number is a different identifier, add a column to `cml_metadata` (migration, parser mapping, `db_writer.py`) and show it in the popup and the search.
- If only a filter is needed, filter or flag the list server-side in `get_available_cmls()` and `/api/cml-map`.
- **Effort:** S (search by existing ID) / M (new identifier column through the pipeline).

### 6. Sublink numbers (network element numbers with 55) (user2)

- **Needs clarification:** the source of these numbers. `cml_metadata.sublink_id` currently holds names such as `sublink_1`.
- **To do:** add a network-element ID per sublink in the metadata (migration, parser mappings, `db_writer.py`), then show it in the popup next to frequency and polarization and make it searchable (item 5).
- **Effort:** M. Depends on what the provider data contains.

### 7. Grey area for periods with possible rain-related outage (user2)

- **Needs clarification:** what defines such a period (a wet/dry flag from the processing, rain gauge or radar data, or a RSL-drop threshold)? Is it per CML or for the whole area?
- **To do:**
- Define and compute the rain-event periods (processor service or SQL), and store them (table or Grafana annotations).
- Draw them as shaded regions in the Grafana panel, e.g. via an annotation query on the dashboard, similar to the existing coloring-window annotation.
- Optionally mark affected CMLs on the map.
- **Effort:** L. Mostly a data and algorithm question; the display part is S-M.

## Follow-ups on done items

- Confirm units in `cml_metadata`: the popup assumes `length` in metres and `frequency` in MHz (as in the archive CSV). The `other_mno_csv` parser maps `length_km` straight into `length`, so that source may be stored in km. Normalize in the parser if so. Effort: S.
2 changes: 1 addition & 1 deletion grafana/admin_dashboards/cml-admin-realtime.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"graphTooltip": 0,
"links": [],
"liveNow": false,
"refresh": "10s",
"refresh": "1m",
"schemaVersion": 38,
"style": "dark",
"tags": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -874,7 +874,7 @@
"type": "geomap"
}
],
"refresh": "10s",
"refresh": "1m",
"schemaVersion": 38,
"style": "dark",
"tags": [
Expand Down
27 changes: 24 additions & 3 deletions webserver/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -537,7 +537,17 @@ def api_cml_map():
with user_db_scope(current_user.id) as conn:
cur = conn.cursor()
cur.execute(
"SELECT DISTINCT ON (cml_id) cml_id::text, site_0_lon, site_0_lat, site_1_lon, site_1_lat FROM cml_metadata ORDER BY cml_id"
"""
SELECT cml_id::text,
MIN(site_0_lon), MIN(site_0_lat), MIN(site_1_lon), MIN(site_1_lat),
MIN(length),
json_agg(json_build_object('frequency', frequency,
'polarization', polarization)
ORDER BY sublink_id)
FROM cml_metadata
GROUP BY cml_id
ORDER BY cml_id
"""
)
data = cur.fetchall()
cur.close()
Expand All @@ -547,6 +557,9 @@ def api_cml_map():
"cml_id": str(row[0]),
"site_0": {"lon": float(row[1]), "lat": float(row[2])},
"site_1": {"lon": float(row[3]), "lat": float(row[4])},
# length in metres, sublink frequency in MHz
"length": safe_float(row[5]),
"sublinks": row[6] or [],
}
for row in data
]
Expand Down Expand Up @@ -911,11 +924,16 @@ def api_admin_cml_map():
with admin_db_scope() as conn:
cur = conn.cursor()
cur.execute(
"""SELECT DISTINCT ON (user_id, cml_id)
user_id, cml_id::text, site_0_lon, site_0_lat, site_1_lon, site_1_lat
"""SELECT user_id, cml_id::text,
MIN(site_0_lon), MIN(site_0_lat), MIN(site_1_lon), MIN(site_1_lat),
MIN(length),
json_agg(json_build_object('frequency', frequency,
'polarization', polarization)
ORDER BY sublink_id)
FROM cml_metadata
WHERE site_0_lon IS NOT NULL AND site_0_lat IS NOT NULL
AND site_1_lon IS NOT NULL AND site_1_lat IS NOT NULL
GROUP BY user_id, cml_id
ORDER BY user_id, cml_id"""
)
rows = cur.fetchall()
Expand All @@ -931,6 +949,9 @@ def api_admin_cml_map():
"cml_id": str(row[1]),
"site_0": {"lon": float(row[2]), "lat": float(row[3])},
"site_1": {"lon": float(row[4]), "lat": float(row[5])},
# length in metres, sublink frequency in MHz
"length": safe_float(row[6]),
"sublinks": row[7] or [],
}
for row in rows
]
Expand Down
81 changes: 72 additions & 9 deletions webserver/templates/admin_realtime.html
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@
height: 100%;
}

/* Place the scale bar beside the zoom buttons instead of below them */
.leaflet-top.leaflet-left .leaflet-control-scale {
clear: none;
margin-left: 8px;
}

.legend-panel {
position: absolute;
top: 10px;
Expand Down Expand Up @@ -156,7 +162,7 @@
<div class="resizer" id="resizer"></div>
<div class="chart-container">
<iframe class="grafana-container" id="grafana-panel"
src="/grafana/d/cml-admin-realtime/cml-real-time-data-all-providers?orgId={{ grafana_org_id }}&refresh=10s&theme=light&viewPanel=2&kiosk"
src="/grafana/d/cml-admin-realtime/cml-real-time-data-all-providers?orgId={{ grafana_org_id }}&refresh=1m&theme=light&viewPanel=2&kiosk"
allowfullscreen></iframe>
</div>
{% endif %}
Expand All @@ -176,6 +182,7 @@
});

const map = L.map('admin-map', { preferCanvas: true }).setView([50, 10], 5);
L.control.scale({ position: 'topleft', metric: true, imperial: false }).addTo(map);
// Esri World Light Gray Canvas - keyless light basemap (tile order is z/y/x)
L.tileLayer('https://server.arcgisonline.com/ArcGIS/rest/services/Canvas/World_Light_Gray_Base/MapServer/tile/{z}/{y}/{x}', {
attribution: 'Tiles &copy; Esri', maxZoom: 16
Expand Down Expand Up @@ -223,9 +230,16 @@

function tooltipText(c) {
const s = stats[c.user_id + '|' + c.cml_id];
let t = esc(providerNames[c.user_id] || c.user_id) + ' &middot; CML ' + esc(c.cml_id);
if (!s) return t + '<br>no data';
t += '<br>Completeness (6h): ' + s.completeness_percent + '%';
let t = '<strong>' + esc(providerNames[c.user_id] || c.user_id) + ' &middot; CML ' + esc(c.cml_id) + '</strong><br>';
if (c.length !== null) t += 'Length: ' + (c.length / 1000).toFixed(2) + ' km<br>';
c.sublinks.forEach(sl => {
const parts = [];
if (sl.frequency !== null) parts.push((sl.frequency / 1000).toFixed(1) + ' GHz');
if (sl.polarization) parts.push(esc(sl.polarization.toUpperCase()));
if (parts.length) t += 'Sublink: ' + parts.join(' ') + '<br>';
});
if (!s) return t + 'no data';
t += 'Completeness (6h): ' + s.completeness_percent + '%';
if (s.last_rsl !== null) t += '<br>Last RSL: ' + s.last_rsl + ' dBm';
if (s.stddev_last_60min !== null) t += '<br>Std Dev (1h): ' + s.stddev_last_60min;
return t;
Expand All @@ -235,7 +249,7 @@
Object.keys(lines).forEach(key => {
const line = lines[key];
line.setStyle({ color: lineColor(line._provider, key) });
line.setTooltipContent(tooltipText(line._cml));
line.setPopupContent(tooltipText(line._cml));
});
}

Expand Down Expand Up @@ -278,12 +292,17 @@
});
}

// Select a CML and point the Grafana iframe at it
function selectCml(providerId, cmlId) {
const key = providerId + '|' + cmlId;
// Highlight a CML on the map
function highlightCml(key) {
if (selectedKey && lines[selectedKey]) lines[selectedKey].setStyle({ weight: 2.5 });
selectedKey = key;
lines[key].setStyle({ weight: 6 }).bringToFront();
}

// Select a CML and point the Grafana iframe at it
function selectCml(providerId, cmlId) {
const key = providerId + '|' + cmlId;
highlightCml(key);

const panel = document.getElementById('grafana-panel');
if (!panel) return;
Expand All @@ -295,6 +314,49 @@
win.dispatchEvent(new PopStateEvent('popstate', { state: null }));
}

// Open the popup of a CML, first bringing it into the part of the map not covered by the chart
function showCmlPopup(key) {
const line = lines[key];
if (!line || !line._map) return;

const chart = document.querySelector('.chart-container');
const chartHeight = chart ? chart.offsetHeight : 0;
const size = map.getSize();
const p = map.latLngToContainerPoint(line.getCenter());
const visible = p.x > 0 && p.x < size.x && p.y > 120 && p.y < size.y - chartHeight - 40;
if (!visible) {
map.fitBounds(line.getBounds(), {
paddingTopLeft: [60, 120],
paddingBottomRight: [60, Math.max(60, chartHeight + 40)],
maxZoom: map.getZoom(),
animate: false
});
}
line.openPopup();
}

// The provider/CML dropdowns live inside the Grafana iframe; poll its URL to react to changes
function watchGrafanaSelection() {
const panel = document.getElementById('grafana-panel');
if (!panel) return;
let primed = false;
setInterval(() => {
let params;
try {
params = new URL(panel.contentWindow.location.href).searchParams;
} catch (e) {
return;
}
const key = params.get('var-provider') + '|' + params.get('var-cml_id');
// Skip until both variables match a CML on the map (cml_id lags behind a provider change)
if (key === selectedKey || !lines[key]) return;
highlightCml(key);
// The first match is the dashboard's initial selection: highlight only, no popup
if (primed) showCmlPopup(key);
primed = true;
}, 500);
}

fetch('/api/admin/cml-map')
.then(r => { if (!r.ok) throw new Error('HTTP ' + r.status); return r.json(); })
.then(cmls => {
Expand All @@ -305,7 +367,7 @@
if (!layers[c.user_id]) layers[c.user_id] = L.featureGroup().addTo(map);
counts[c.user_id] = (counts[c.user_id] || 0) + 1;
const line = L.polyline([a, b], { color: NO_DATA, weight: 2.5, opacity: 0.85 })
.bindTooltip(tooltipText(c), { sticky: true })
.bindPopup(tooltipText(c), { autoPan: false })
.on('click', () => selectCml(c.user_id, c.cml_id))
.addTo(layers[c.user_id]);
line._provider = c.user_id;
Expand All @@ -319,6 +381,7 @@
renderLegend();
loadStats();
setInterval(loadStats, STATS_REFRESH_MS);
watchGrafanaSelection();
})
.catch(err => {
document.getElementById('map-status').textContent = 'Failed to load CMLs (' + err.message + ')';
Expand Down
Loading
Loading