[Nfd-dev] Handling new content in app for pending interest in NFD

Wed Apr 1 10:50:34 PDT 2015

On Wed, Apr 1, 2015 at 10:21 AM Dave Oran (oran) <oran at cisco.com> wrote:

>
> > On Apr 1, 2015, at 12:59 PM, Wentao Shang <wentaoshang at gmail.com> wrote:
> >
> > I think this problem shows one limitation of in-memory cache (and repo
> in general). The cache/repo is a "passive" storage: once the data gets in,
> it never gets out unless someone explicitly request it.
> And why is that not exactly what you want? A place that can respond to
> interests without interrupting or otherwise causing thread scheduling
> problems for an application generating data open-loop.
>

I think in general this is what most application wants. The problem that
Jeff mentioned is that by the time the data is inserted into repo, the
interest has already resided inside NFD. The repo may have seen that
interest before but did not response because the data wasn't ready yet.

But I totally agree that repo should not do anything more than just storing
the data and responding to incoming interest. It's the application who can
be more intelligent in deciding when to push out data.

>
> The repo is protected from infinite growth by the expiration time of the
> data.
>
> > The applications, on the other hand, can be more "active" in deciding
> when to send out data in response to previous interests.
> >
> You;ll have to explain why this is a good thing? Can the application
> smooth jitter this way? Maybe. What other benefits might accrue?
>

I believe the application should have the freedom to insert an arbitrary
delay before responding to the interest. That delay may range from zero to
infinite. There are various reasons to cause the delay: maybe the
application needs to fetch some data from some other remote host; maybe the
application wants to adjust jitter; or maybe the machine that runs the
application is just super slow...

>
> > I remember Illya and Lijing's ndnvideo implementation also needs to
> handle similar problem. Their current solution (as far as I understand) is
> to push out the data twice: once to the NFD for satisfying request, once to
> the repo for archiving. The second push uses repo insertion command so it's
> not just a plain data packet (like in the first push).
> >
> Why not put everything into the repo and let the Repo respond to all
> interests? Is it the cost of context switches?
>

Still, the problem is that if the repo already missed the interest, it has
to wait for the retransmission of that interest, unless someone (most
likely the application) pulls the trigger and sends out the data before the
previous interest expires...

>
> Is it because the architectural model of the repo is that it had to have
> non-volatile backing store to survive host crashes which might be
> irrelevant for pure realtime applications (i.e. you don’t care about
> recordings).
>
>
> > Wentao
> >
> > On Wed, Apr 1, 2015 at 8:38 AM Burke, Jeff <jburke at remap.ucla.edu>
> wrote:
> >
> > Thanks for all of the discussion. I am not sure if I have been able to
> extract an answer to the original question I was asking.  Let me try again
> with more information:
> >
> > - The original question was intended to be only about a local NFD (for
> now).
> > - The use case is a (soft) real-time application, like videoconferencing
> or streaming sensor data for interactive applications. Specifically, we
> have encountered the issue in 1) ndnrtc/ndncon for videoconferencing and 2)
> ndn-opt, which streams data from OpenPTrack, a person tracking system built
> on the robot operating system (ROS).
> >
> > - In these use cases, consumers want to receive data with low latency,
> so they send interests intended to arrive before the time the data is
> produced, but not so long before that the Interests expire before
> corresponding data is produced.
> >
> > - As they are born, samples are signed and put into a repository by the
> publisher.  Currently these apps use an in-memory repo specific to the
> application, via the MemoryContentCache class provided by the NDN-CCL
> library. But this could just as well be a repo outside the application.
> >
> > - So, when Interests arrive, they go in the local  NFD's PIT, and are
> forwarded to the application.   We are interested in the case where the
> data is not born yet, so the publisher cannot respond to the Interest.
> >
> > - Then, before the Interest expires from the local NFD's PIT, the
> corresponding data is born in the application and should be used to answer
> the Interest.  But how does that data get back to the local forwarder?
> >
> > - What should the application do?  1) Maintain its own PIT?   2) Or,
> push the data to the local NFD, letting it drop it if it doesn't satisfy
> anything.  3) Or, as we have previously discussed about the
> MemoryContentCache, do we need to consider a more sophisticated interaction
> model between the application and the daemon's content store and/or PIT
> that reduces duplication of these capabilities in applications?
> >
> > - For now, we have implemented an in-application PIT in ndnrtc.  (I am
> not sure what method we ended up using in ndn-opt, perhaps the unsolicited
> data method.)
> >
> > - Discussion on this topic will help inform what library support will be
> provided in the short term.  (I don't think we want every application
> author to have to write a PIT, so am thinking in the short term we should
> provide one to complement the MemoryContentCache class.  But this is
> duplicating data structures and functionality in NFD – is that natural, or
> should we look to handle it through a different interaction with the local
> forwarder?)
> >
> > Thanks,
> > Jeff
> >
> >
> > From: <Ignacio.Solis at parc.com>
> > Date: Tue, 31 Mar 2015 00:06:33 +0000
> > To: <wentaoshang at gmail.com>, <Marc.Mosko at parc.com>
> >
> > Cc: <nfd-dev at lists.cs.ucla.edu>
> > Subject: Re: [Nfd-dev] Handling new content in app for pending interest
> in NFD
> >
> > I'm going to guess that caching data rules will be the same as
> forwarding rules. If the interface is not allowed to produce that content
> (due to routing / fib entries, local naming policy) data should not be
> forwarded OR cached.
> >
> > NDN has a slightly more complicated problem with caching than CCN given
> then LPM matching rules.
> >
> > If you have a repo that has permission to serve this content then you
> changed the problem. Now you need the repo protocol to give you trust.
> >
> > For regular forwarding you are relying on trusting routing. For local
> apps you will rely (I assume)  on local permissions. For repo you will need
> both of those plus now a repo permission  protocol.
> >
> > Dropping unsolicited data may end up being your only recourse.
> >
> > There are obviously other approaches for pre-populating caches that
> involve higher level protocols, but I assume that's not what you're talking
> about.
> >
> > Nacho (Ignacio) Solis
> > Principal Scientist
> > Palo Alto Research Center
> >
> > From: Wentao Shang <wentaoshang at gmail.com>
> > Sent: Mar 30, 2015 4:50 PM
> > To: "Solis, Ignacio <Ignacio.Solis at parc.com>" <Ignacio.Solis at parc.com>;Mosko,
> Marc <Marc.Mosko at parc.com>
> > Cc: nfd-dev at lists.cs.ucla.edu
> > Subject: Re: [Nfd-dev] Handling new content in app for pending interest
> in NFD
> >
> > Hi Nacho,
> >
> > I don't have any argument against this rule right now. But it seems more
> relevant to "forwarding" data packets, while the problem we were discussing
> is about "caching" unsolicited data packets, which cannot be forwarded
> anyway because there is no PIT. The simplest solution I would suggest to
> the problem is to drop any unsolicited data in the forwarder.
> >
> > Wentao
> >
> > On Mon, Mar 30, 2015 at 3:42 PM <Ignacio.Solis at parc.com> wrote:
> > This rule is important, router or not. You can't have "non-priviledged"
> applications generating replies at servers. Specially important at systems
> with multiple users,  tenants, virtual systems.
> >
> > At the local router there are even more restrictions.  We haven't even
> gotten to talking about privileged names and restrictions and end-h out
> behavior.   We have a design  for this for CCN that we'll be talking about
> at CCNxCon.
> >
> > Nacho (Ignacio) Solis
> > Principal Scientist
> > Palo Alto Research Center
> >
> > From: Wentao Shang <wentaoshang at gmail.com>
> > Sent: Mar 30, 2015 1:07 PM
> > To: Mosko, Marc <Marc.Mosko at parc.com>
> >
> > Cc: nfd-dev at lists.cs.ucla.edu
> > Subject: Re: [Nfd-dev] Handling new content in app for pending interest
> in NFD
> >
> > On Mon, Mar 30, 2015 at 12:52 PM <Marc.Mosko at parc.com> wrote:
> > I’ll add that in addition to the content having a PIT entry, Nacho Solis
> has suggested that a router should verify that a Content Object came in a
> face from which it was requested.  Otherwise, there’s a potential for
> off-path attacks sending content objects that match popular well-known
> names.  A variation on that, rather than tracking forward paths, would be
> to at minimum verify that a content object comes from a face for which
> there’s a corresponding FIB entry for the name.
> >
> > This requirement makes sense for transit routers that receive packets
> from other routers. But I'm not sure whether this is necessary for
> forwarding daemons handling local applications... The pushed data should
> never get out of the local forwarder unless there is a PIT entry specifying
> a path pointing to some other router.
> >
> > Wentao
> >
> >
> > Marc
> >
> > On Mar 30, 2015, at 12:08 PM, Wentao Shang <wentaoshang at gmail.com>
> wrote:
> >
> >> I agree with Dave.
> >>
> >> In principle, router should not cache unsolicited data. In the
> situation we are discussing, the application should either just push the
> data out, which may be dropped by NFD if the interest has expired, or store
> the data in some application-level cache (or repo) for future fetching.
> >>
> >> Wentao
> >>
> >> On Mon, Mar 30, 2015 at 11:52 AM Dave Oran (oran) <oran at cisco.com>
> wrote:
> >> Isn’t this what the repo was invented for?
> >>
> >> Holding packets in a router that has forgotten that they were asked for
> is a giant invitation to cache pollution/poisoning attacks.
> >>
> >> > On Mar 30, 2015, at 2:24 PM, Haowei Yuan <hyuan at wustl.edu> wrote:
> >> >
> >> > I think as long as the data has actually been requested by an Interest
> >> > packet, it is safe to send the Data packet to NFD. The NFD will either
> >> > forward or drop the Data packet by checking if the Interest has
> >> > expired.
> >> >
> >> > If the Interest has expired, Data packet is dropped, and the consumer
> >> > is still interested in the data, the consumer could resend the
> >> > Interest. Hopefully this time, the Data packet can be generated and
> >> > sent faster by the application so that NFD will forward it.
> >> >
> >> > Haowei
> >> >
> >> >
> >> > On Mon, Mar 30, 2015 at 1:12 PM, Anil Jangam <anilj.mailing at gmail.com>
> wrote:
> >> >>
> >> >> On Mar 30, 2015 11:08 AM, "Dehart, John" <jdd at wustl.edu> wrote:
> >> >>>
> >> >>>
> >> >>> Is there any harm in it pushing the data out without knowing for
> sure if
> >> >>> the
> >> >>> Interest is still active?
> >> >>>
> >> >> If data is so critical, can the Interest be refreshed proactively
> before it
> >> >> expires?
> >> >>
> >> >> /anil
> >> >>
> >> >>> John
> >> >>>
> >> >>>> On Mar 30, 2015, at 1:05 PM, Burke, Jeff <jburke at remap.ucla.edu>
> wrote:
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> On Mon, Mar 30, 2015 at 10:28 AM Burke, Jeff <
> jburke at remap.ucla.edu>
> >> >>>>> wrote:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> Hi folks,
> >> >>>>>>
> >> >>>>>> We are facing this scenario in a few applications:
> >> >>>>>>
> >> >>>>>> 1) Interest received by NFD, passed to an application
> >> >>>>>> 2) Application not able to respond to interest, so interest
> stays in
> >> >>>>>> NFD PIT
> >> >>>>>> 3) Some time passes, but not enough for the Interest to expire
> >> >>>>>> 4) Application generates data (e.g., from a sensor reading) that
> would
> >> >>>>>> answer the Interest in the NFD PIT
> >> >>>>>>
> >> >>>>>> Question: How does app know to inform NFD it has the data after
> step 4,
> >> >>>>>> and how should it do that?
> >> >>>>>>
> >> >>>>>> - In this type of app, should it push the data unsolicited to
> the NFD
> >> >>>>>> and let it decide if there is something to do?
> >> >>>>>
> >> >>>>>
> >> >>>>> In my opinion, as long as the application is certain that the
> Interest
> >> >>>>> has arrived and is stored in NFD's PIT, it can just push the data
> out to
> >> >>>>> NFD.
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> How certain does it have to be?  There is a chance it could have
> >> >>>> expired...
> >> >>>> jeff
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>>
> >> >>>>> Wentao
> >> >>>>>
> >> >>>>>>
> >> >>>>>> - Is it recommended to implement an application-level PIT so the
> app is
> >> >>>>>> sure this data is solicited?  (Why add another PIT?)
> >> >>>>>>
> >> >>>>>> Thank you,
> >> >>>>>> Jeff
> >> >>>>>>
> >> >>>>>> _______________________________________________
> >> >>>>>> Nfd-dev mailing list
> >> >>>>>> Nfd-dev at lists.cs.ucla.edu
> >> >>>>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> >> >>>>
> >> >>>> _______________________________________________
> >> >>>> Nfd-dev mailing list
> >> >>>> Nfd-dev at lists.cs.ucla.edu
> >> >>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Nfd-dev mailing list
> >> >>> Nfd-dev at lists.cs.ucla.edu
> >> >>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> >> >>>
> >> > _______________________________________________
> >> > Nfd-dev mailing list
> >> > Nfd-dev at lists.cs.ucla.edu
> >> > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> >>
> >>
> >> _______________________________________________
> >> Nfd-dev mailing list
> >> Nfd-dev at lists.cs.ucla.edu
> >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> >> _______________________________________________
> >> Nfd-dev mailing list
> >> Nfd-dev at lists.cs.ucla.edu
> >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> >
> > _______________________________________________ Nfd-dev mailing list
> Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/
> mailman/listinfo/nfd-dev
> > _______________________________________________
> > Nfd-dev mailing list
> > Nfd-dev at lists.cs.ucla.edu
> > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
> > _______________________________________________
> > Nfd-dev mailing list
> > Nfd-dev at lists.cs.ucla.edu
> > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20150401/b1842f26/attachment.html>