Skip to content

fix: preserve original telemetry timestamp on periodic resend#2120

Open
tijn-hedgehog wants to merge 1 commit into
thingsboard:masterfrom
tijn-hedgehog:fix/preserve-telemetry-timestamp-on-periodic-resend
Open

fix: preserve original telemetry timestamp on periodic resend#2120
tijn-hedgehog wants to merge 1 commit into
thingsboard:masterfrom
tijn-hedgehog:fix/preserve-telemetry-timestamp-on-periodic-resend

Conversation

@tijn-hedgehog

@tijn-hedgehog tijn-hedgehog commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fixes periodic reporting overwriting the original telemetry timestamp with current wall-clock time in __periodical_reporting()
  • Periodic resends now preserve the original source timestamp (report_strategy_data_record.get_ts()) so downstream consumers can detect stale data
  • receivedTs in metadata still reflects when the gateway forwarded the data

Problem

When using ON_REPORT_PERIOD or ON_CHANGE_OR_REPORT_PERIOD strategies, the periodic reporting thread replaced the original telemetry timestamp with int(time() * 1000) before re-sending. This made stale data appear fresh in ThingsBoard. If a device stopped updating, the ts kept advancing every report period, breaking any staleness detection in dashboards, rule chains, or frontends.

Change

In report_strategy_service.py, __periodical_reporting():

# Before (overwrites timestamp):
current_ts = int(time() * 1000)
data_entry.add_to_telemetry(TelemetryEntry({key: value}, current_ts))
report_strategy_data_record.update_ts(current_ts)

# After (preserves original timestamp):
data_entry.add_to_telemetry(TelemetryEntry({key: value}, report_strategy_data_record.get_ts()))

The commented-out line that already existed for this purpose has been activated and the timestamp-overwriting code removed.

Test plan

  • Configure a device with ON_CHANGE_OR_REPORT_PERIOD strategy (e.g. 60s report period)
  • Send one telemetry value, then stop the device
  • Verify ThingsBoard latest telemetry ts stays at the original value (does not advance every 60s)
  • Verify receivedTs in metadata still updates on each resend
  • Verify that when a new value arrives, ts updates correctly via filter_datapoint_and_cache
  • Test with ON_REPORT_PERIOD strategy as well

Fixes #2119

The periodic reporting thread in __periodical_reporting() was overwriting
the original telemetry timestamp with the current wall-clock time before
re-sending cached values. This made it impossible for downstream consumers
to detect stale data — the timestamp always looked fresh even if the source
device had stopped updating.

Now periodic resends use the original timestamp from when the value was
last received from the device (via report_strategy_data_record.get_ts()).
The receivedTs in metadata still reflects when the gateway forwarded the
data, covering the "last forwarded" use case.

Fixes thingsboard#2119
@iamkaran

iamkaran commented Apr 11, 2026

Copy link
Copy Markdown
Contributor

Correct me if i am wrong but doesn't this defeat the purpose of ON_REPORT_PERIOD as it serves the purpose of maintaining connection activity by sending data periodically even if it is stale.
Its also mentioned in the docs:
image

The current_ts behavior is intentional: the periodic updates are heartbeats disguised as telemetry

@tijn-hedgehog

Copy link
Copy Markdown
Contributor Author

If that's the case, then I propose a new reporting strategy that either sends data on change, or every reporting period (but preserves the timestamp so the user knows if it's stale).

In my customer dashboard, I want to show data updates periodically (eg, every 30s), but also instant updates when values have changed. When a device has gone stale, the current reporting strategy would keep sending the stale data as if it were fresh, misleading the customer. Currently, there is no (to my understanding, correct me if I'm wrong) report strategy that has this behavior.

@Stefan-1313

Stefan-1313 commented Apr 28, 2026

Copy link
Copy Markdown

I hope I'm not too late. This is exaclty the feature I'm also looking for.
I'd like to add a concrete use case in support of this PR's direction.

I use ON_CHANGE_OR_REPORT_PERIOD to avoid storing excessive telemetry while still keeping time-series queries simple — having at least one real data point per reporting interval means I can query a bounded time range and always get a meaningful result, without expensive "look back until last known value" queries.

The current behavior breaks this: when a device goes silent, ON_CHANGE_OR_REPORT_PERIOD keeps forwarding the last known value with a fresh timestamp, so the absence of new data is invisible. No-data is also data — it tells me a device may be offline, a sensor may have failed, or a process has stopped. Hiding that with synthetic heartbeat timestamps is genuinely misleading in dashboards and rule chains.

My preferred behavior:

  • If at least one real measurement was received during the reporting interval → send it, preserving the original source timestamp.
  • If no measurement was received → send nothing. Let the gap speak for itself.

If I understand correctly, this is exactly what this PR moves toward. The concern that ON_REPORT_PERIOD serves as a "heartbeat" is valid for some users, but it shouldn't be the only mode. I'd suggest making it configurable with a flag, e.g. suppress_on_no_data: true/false (defaulting to false for backward compatibility), so both behaviors are supported:

  • suppress_on_no_data: false (current/default): keeps the heartbeat behavior. Ideally preserving the original last datapoint timestamp so staleness detection is still possible but this part is optional but this is what this pull request does if I understand correctly.
  • suppress_on_no_data: true: no real data in the interval = no telemetry written. Clean gaps in the time series.

This makes the feature backward compatible while giving users like me — and presumably the PR author's customers — a way to build honest dashboards.

Happy to help draft or test this if the maintainers are open to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report strategy overwrites original telemetry timestamp with current time on periodic resend

3 participants