Add shared state network nodes

[spiccinini]

This PR adds an rpcd/ubus service that provides the list of the nodes of the network and the ability to remove one node from the list. This uses the shared-state multiwriter as main storage so it should be easiear to first review and merge the shared-state changes of PR #872 .

Most of the code was done by @germanferrero , great work!

Fixes #867

Implementation details (see rationale in the related issue #867):

• Use the new shared-state-multiwriter:
  • The keys are the hostnames of the nodes
  • Each node inserts its own key in the database
  • Any node can modify the membership of other node (because the community should be able to mark a node as not participating anymore, for example people moved, or the name of the node changed). This is provided by ubus in this PR and its frontend in the Lime-App PR Add Reachable/Unreachable Nodes and Delete Nodes Screens lime-app#301
  • This database is persistent
  • attributes stored in the multiwriter database: hostname, ipv4, ipv6, board, fw_version, member
  • attributes computed from the nodes_and_links database plus the membership attribute: status. It can be: recently_connected, disconnected, gone. A node is considered disconnected if its member attribute is True but its key is not in the nodes_and_links database. As its key can be present but the node may be off or disconnected for a couple of minutes (until the TTL drops to 0 and it is removed from the nodes_and_links db) the status is recently_connected and not just connected so that it does not give "false information".
  • A cli to show the network nodes list

This is the CLI provided, it only shows and can't be used to remove nodes (use the ubus interface for that):

[nicopace]

haven't reviewed yet, but looks very nice! thanks for the work put in this!

[germanferrero]

The motivation for recently_connected also applies for recently_disconnected as the "disconnected" node could recently come back to the network but synchronization of it's nodes_and_links key between that node and the current node didn't happened yet.
Should we named it recently_disconnected too?

[germanferrero]

The motivation for recently_connected also applies for recently_disconnected as the "disconnected" node could recently come back to the network but synchronization of it's nodes_and_links key between that node and the current node didn't happened yet.
Should we named it recently_disconnected too?

After chatting with @spiccinini, recently_disconnected wouldn't be accurate neither, since the node not in sync may be the current node. disconnected_or_not_sync or very_likely_disconnected would, as the time needed to spread nodes_and_links key after boot should be very short.

[germanferrero]

Looks great @spiccinini good job :)

[codecov-commenter]

Codecov Report

Merging #873 (d6cd542) into master (55ec0b8) will increase coverage by 0.58%.
The diff coverage is 98.57%.

❗ Current head d6cd542 differs from pull request most recent head 1cc901c. Consider uploading reports for the commit 1cc901c to get more accurate results

@@            Coverage Diff             @@
##           master     #873      +/-   ##
==========================================
+ Coverage   73.92%   74.51%   +0.58%     
==========================================
  Files          41       42       +1     
  Lines        3559     3629      +70     
==========================================
+ Hits         2631     2704      +73     
+ Misses        928      925       -3     

Impacted Files	Coverage Δ
...kages/lime-system/files/usr/lib/lua/lime/utils.lua	81.06% <75.00%> (+0.82%)	⬆️
...-network_nodes/files/usr/lib/lua/network-nodes.lua	100.00% <100.00%> (ø)
...es/shared-state/files/usr/lib/lua/shared-state.lua	77.23% <0.00%> (+0.44%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55ec0b8...1cc901c. Read the comment docs.

[germanferrero]

I left minor changes request in the comments :) Looks great!

[nicopace]

this is not the first time I see this function.
should it become part of a library?

[spiccinini]

Yes, when we create the lime-utils package :)

[nicopace]

this function feels like a private function within the module... could it be the case?

[spiccinini]

Thanks for pointing it. I've pushed a refactor for this and all the non public functions to be accesed with a leading underscore (as they need to be reachable for the tests).