[Ndn-lib] PSync PartialProducer on multiple nodes

Junxiao Shi shijunxiao at email.arizona.edu
Tue Jan 29 15:22:32 PST 2019

Hi Lan

> As I said, this approach is not scalable: it requires one Consumer for
> each participant, which is expensive in terms of traffic overhead and
> memory usage.
> Why not merge FullSync and PartialSync so that multiple producers can
> publish their user nodes on the same sync prefix?
> That’s something we can think about, but they do work differently (the
> PartialSync interest carries the producer’s previous IBF when the
> subscriber last sync’ed with the producer (not the subscriber’s own IBF),
> while the Fullsync interest carries the node’s own IBF.  I don’t have a
> clear idea of how to combine them or if it’s possible right now.

I see, the reason is down in the protocol. It is indeed impossible to
combine them because they need different IBFs.

I think the solution is to provide a higher level facility in the library
that supports PartialSync from multiple producers. Inside this facility:

   1. It runs one instance of FullSync to maintain the roster. Every
   producer and consumer participate in this FullSync group.
   2. Each producer has one UserNode in the FullSync group. If two
   producers are serving the same dataset, they should participate FullSync
   under the same UserNode. Consumers do not have UserNodes in the FullSync
   3. Each producer runs one instance of PartialSync. Whenever the list of
   UserNodes in PartialSync changes, the producer increments the sequence
   number of its UserNode in FullSync.
   4. Consumer sends hello to a producer whenever the producer's FullSync
   UserNode has a new sequence number.

This would accommodate a PartialSync UserNode to be served from any
producer. PartialSync UserNodes can even be moved between producers.
One potential problem is what to do when multiple distinct producers have
the same PartialSync UserNode.

> If you are concerned about the number of consumers.  We can change the
> code to allow one consumer to subscribe to multiple different sync
> prefixes.  That should not be too difficult an implementation change.

If the library has the higher level facility as described above, number of
consumers isn't really an issue because it's hidden from the application.

Yours, Junxiao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-lib/attachments/20190129/43df4eca/attachment.html>

More information about the Ndn-lib mailing list