Support for shallow copy-modify messages

[gtoombs-avidrone]

I have a use case where I want to modify the time of a message and then re-serialize it. I did get it working, via the clunky

def swap_time(
    message: pynmeagps.NMEAMessage,
    new_time: datetime.time,
) -> pynmeagps.NMEAMessage:
    # NMEAMessage is immutable, with seemingly no good way to make shallow
    # modifications or partial copies :(

    fields = message._get_dict().keys()

    return pynmeagps.NMEAMessage(
        talker=message.talker,
        msgID=message.msgID,
        msgmode=message.msgmode,
        hpnmeamode=message._hpnmeamode,
        validate=pynmeagps.VALCKSUM | pynmeagps.VALMSGID,
        # checksum will be auto-populated
        **{
            field:
            # This is sent to the constructor as a time object; NMEAMessage auto-formats it
            # back to a string
            new_time if field == 'time'
            else getattr(message, field)
            for field in fields
        },
    )

Is there a better way, either with the current codebase or an added feature?

[semuadmin]

Hi @gtoombs-avidrone

So NMEAMessage objects are intentionally immutable after creation as they are intended to represent point-in-time GNSS state.

There is no easy way to "copy and amend" an existing NMEAMessage object other than by creating a new NMEAMessage and setting (some of) its attributes to those of the original message, much as you have done in your code snippet. I would, however, be very wary of using a message.__get_dict__() construct without further qualification for this purpose, as this would encompass both public and private attributes. It is, as you say, somewhat clunky and it may not serialize correctly.

I would simply set the individual public attributes for the particular message type you're interested in - most NMEA sentences are not that large.

I'd be interested to understand what use case would warrant this?

[gtoombs-avidrone]

Is 'setting public attributes' even possible? When I tried this, the object rejected the set due to immutability.

The use case is to adjust times for a canned data replay scenario to avoid non-monotonic time series points.

[semuadmin]

Is 'setting public attributes' even possible? When I tried this, the object rejected the set due to immutability.

I mean something like this:

from pynmeagps import NMEAMessage, GET

oldmessage = YOUR_ORIGINAL_NMEAMESSAGE_OBJECT
newtime = YOUR_NEW_TIME_VALUE

newmsg = NMEAMessage(
    "GN",
    "GLL",
    GET,
    lat=oldmessage.lat,
    lon=oldmessage.lon,
    time=newtime,
    status=oldmessage.status,
    posMode=oldmessage.posMode,
)

Obviously you'd have to do this for each different message type (GSA, RMC, etc), which might get a bit onerous. Can you not just manipulate the time value within the code which handles these NMEAMessage objects, or capture an entire time series of NMEA data rather than just a single message?

[gtoombs-avidrone]

The message type proliferation is indeed onerous.

Can you not just manipulate the time value within the code which handles these NMEAMessage objects

I'm not entirely sure what you mean. I'm not particularly handling any of the messages; I'm consuming them from one (static file) stream, modifying them, and then using the stock pynmeagps serializer.

capture an entire time series of NMEA data rather than just a single message?

I do that already. Then your library decodes the series to single messages, I mutate and reserialize them.

[semuadmin]

I'm not entirely sure what you mean. I'm not particularly handling any of the messages; I'm consuming them from one (static file) stream, modifying them, and then using the stock pynmeagps serializer.

Sorry - I misread your code snippet and thought you were using __dict__ rather than _get_dict(). Obviously the latter is not intended to be a public method - if you're only intending to update standard NMEA output (GET) message types, the following would be a simpler solution without having to invoke private methods:

from pynmeagps import NMEA_PAYLOADS_GET
...
    fields = NMEA_PAYLOADS_GET[message.msgID]
...

[semuadmin]

@gtoombs-avidrone

FYI it may not be exactly what you're looking for, but the ubxsimulator utility within the pyubxutils library is capable of generating synthetic time series data for selected NMEA, UBX or RTCM message types, based on a json configuration file. It was developed precisely to facilitate time series testing of my own GNSS applications. I'd be happy to consider enhancements to this utility if it would benefit wider testing use cases.

[gtoombs-avidrone]

That's interesting enough, but I'm unclear on whether that simulator would be able to accept and replay a pre-existing raw stream.

[semuadmin]

So the ubxsimulator is intended to create synthetic time series data in situations where 'real world' time series data is not available, or where you want to set a specific attribute to a specific value for testing purposes.

Obviously, if you already have a raw data stream then there may be no value in using a simulator.