[Ndn-lib] PSync PartialProducer on multiple nodes
shijunxiao at email.arizona.edu
Tue Jan 29 15:22:32 PST 2019
> As I said, this approach is not scalable: it requires one Consumer for
> each participant, which is expensive in terms of traffic overhead and
> memory usage.
> Why not merge FullSync and PartialSync so that multiple producers can
> publish their user nodes on the same sync prefix?
> That’s something we can think about, but they do work differently (the
> PartialSync interest carries the producer’s previous IBF when the
> subscriber last sync’ed with the producer (not the subscriber’s own IBF),
> while the Fullsync interest carries the node’s own IBF. I don’t have a
> clear idea of how to combine them or if it’s possible right now.
I see, the reason is down in the protocol. It is indeed impossible to
combine them because they need different IBFs.
I think the solution is to provide a higher level facility in the library
that supports PartialSync from multiple producers. Inside this facility:
1. It runs one instance of FullSync to maintain the roster. Every
producer and consumer participate in this FullSync group.
2. Each producer has one UserNode in the FullSync group. If two
producers are serving the same dataset, they should participate FullSync
under the same UserNode. Consumers do not have UserNodes in the FullSync
3. Each producer runs one instance of PartialSync. Whenever the list of
UserNodes in PartialSync changes, the producer increments the sequence
number of its UserNode in FullSync.
4. Consumer sends hello to a producer whenever the producer's FullSync
UserNode has a new sequence number.
This would accommodate a PartialSync UserNode to be served from any
producer. PartialSync UserNodes can even be moved between producers.
One potential problem is what to do when multiple distinct producers have
the same PartialSync UserNode.
> If you are concerned about the number of consumers. We can change the
> code to allow one consumer to subscribe to multiple different sync
> prefixes. That should not be too difficult an implementation change.
If the library has the higher level facility as described above, number of
consumers isn't really an issue because it's hidden from the application.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Ndn-lib