Distributed Data Logger (DDL) was created with the purpose of extending and boosting the well-known Masked Authenticated Messaging (MAM) second IOTA layer, allowing a MAM channel reader to access additional features.
More specifically a MAM channel has these properties:
These properties are not reducing MAM’s usefulness, but in some cases they are inconvenient.
For example both MAM writer and receiver, communicating on a channel for a long time, need to store the root id of the last message exchanged safely, in order to be able to start again from that point if the system needs to be restarted losing the state.
In doing this, they lose the opportunity to retrieve all the messages older than the last one they have managed.
To solve this issue, the first root id of the channel can be stored too, allowing the parties to start from the end or from the beginning to retrieve some data from the channel.
But what happens if the channel contains thousands of messages?
The transmitter is able to skip all the channel’s content and restart from the last root id, but the receiver, who needs to retrieve a specific message in a time frame, needs to process all the channel from the beginning with a lot of effort.
We found it really unpractical, and that’s why we started searching for a way to not only speed up the search, but also filter it: this is basically how our Distributed Data Logger came to life.
More specifically a MAM channel has these properties:
- it can be read only starting from a known root id (the unique message identification number) up to the last message in the channel;
- it can be read only sequentially, because the fetch function, given a root id, returns one message, a requested number of messages, or all messages in the channel starting from that root;
- there is no way to know in advance how many messages are stored in a MAM channel from a given root id, without reading all the channel up to the last messages.
These properties are not reducing MAM’s usefulness, but in some cases they are inconvenient.
For example both MAM writer and receiver, communicating on a channel for a long time, need to store the root id of the last message exchanged safely, in order to be able to start again from that point if the system needs to be restarted losing the state.
In doing this, they lose the opportunity to retrieve all the messages older than the last one they have managed.
To solve this issue, the first root id of the channel can be stored too, allowing the parties to start from the end or from the beginning to retrieve some data from the channel.
But what happens if the channel contains thousands of messages?
The transmitter is able to skip all the channel’s content and restart from the last root id, but the receiver, who needs to retrieve a specific message in a time frame, needs to process all the channel from the beginning with a lot of effort.
We found it really unpractical, and that’s why we started searching for a way to not only speed up the search, but also filter it: this is basically how our Distributed Data Logger came to life.