[Nfd-dev] Handling new content in app for pending interest in NFD

Burke, Jeff jburke at remap.UCLA.EDU
Wed Apr 1 11:04:01 PDT 2015 

I can't speak for the other ndnvideo implementation (next-ndvideo) but
generally think that putting this type of content in a repo is a good
idea.  (The same issue I raised would exist, though.) We'll eventually use
a repo in ndnrtc, either in addition to or instead of the app cache, for
call recording - I think we didn't at first because we weren't sure about
the current implementation's impact on latency if it was the sole
mechanism used to publish the data.

Jeff


On 4/1/15, 10:21 AM, "Dave Oran (oran)" <oran at cisco.com> wrote:

>
>> On Apr 1, 2015, at 12:59 PM, Wentao Shang <wentaoshang at gmail.com> wrote:
>> 
>> I think this problem shows one limitation of in-memory cache (and repo
>>in general). The cache/repo is a "passive" storage: once the data gets
>>in, it never gets out unless someone explicitly request it.
>And why is that not exactly what you want? A place that can respond to
>interests without interrupting or otherwise causing thread scheduling
>problems for an application generating data open-loop.
>
>The repo is protected from infinite growth by the expiration time of the
>data. 
>
>> The applications, on the other hand, can be more "active" in deciding
>>when to send out data in response to previous interests.
>> 
>You;ll have to explain why this is a good thing? Can the application
>smooth jitter this way? Maybe. What other benefits might accrue?
>
>> I remember Illya and Lijing's ndnvideo implementation also needs to
>>handle similar problem. Their current solution (as far as I understand)
>>is to push out the data twice: once to the NFD for satisfying request,
>>once to the repo for archiving. The second push uses repo insertion
>>command so it's not just a plain data packet (like in the first push).
>> 
>Why not put everything into the repo and let the Repo respond to all
>interests? Is it the cost of context switches?
>
>Is it because the architectural model of the repo is that it had to have
>non-volatile backing store to survive host crashes which might be
>irrelevant for pure realtime applications (i.e. you don¹t care about
>recordings).
>
>
>> Wentao
>> 
>> On Wed, Apr 1, 2015 at 8:38 AM Burke, Jeff <jburke at remap.ucla.edu>
>>wrote:
>> 
>> Thanks for all of the discussion. I am not sure if I have been able to
>>extract an answer to the original question I was asking.  Let me try
>>again with more information:
>> 
>> - The original question was intended to be only about a local NFD (for
>>now).
>> - The use case is a (soft) real-time application, like
>>videoconferencing or streaming sensor data for interactive applications.
>>Specifically, we have encountered the issue in 1) ndnrtc/ndncon for
>>videoconferencing and 2) ndn-opt, which streams data from OpenPTrack, a
>>person tracking system built on the robot operating system (ROS).
>> 
>> - In these use cases, consumers want to receive data with low latency,
>>so they send interests intended to arrive before the time the data is
>>produced, but not so long before that the Interests expire before
>>corresponding data is produced.
>> 
>> - As they are born, samples are signed and put into a repository by the
>>publisher.  Currently these apps use an in-memory repo specific to the
>>application, via the MemoryContentCache class provided by the NDN-CCL
>>library. But this could just as well be a repo outside the application.
>> 
>> - So, when Interests arrive, they go in the local  NFD's PIT, and are
>>forwarded to the application.   We are interested in the case where the
>>data is not born yet, so the publisher cannot respond to the Interest.
>> 
>> - Then, before the Interest expires from the local NFD's PIT, the
>>corresponding data is born in the application and should be used to
>>answer the Interest.  But how does that data get back to the local
>>forwarder? 
>> 
>> - What should the application do?  1) Maintain its own PIT?   2) Or,
>>push the data to the local NFD, letting it drop it if it doesn't satisfy
>>anything.  3) Or, as we have previously discussed about the
>>MemoryContentCache, do we need to consider a more sophisticated
>>interaction model between the application and the daemon's content store
>>and/or PIT that reduces duplication of these capabilities in
>>applications?   
>> 
>> - For now, we have implemented an in-application PIT in ndnrtc.  (I am
>>not sure what method we ended up using in ndn-opt, perhaps the
>>unsolicited data method.)
>> 
>> - Discussion on this topic will help inform what library support will
>>be provided in the short term.  (I don't think we want every application
>>author to have to write a PIT, so am thinking in the short term we
>>should provide one to complement the MemoryContentCache class.  But this
>>is duplicating data structures and functionality in NFD ­ is that
>>natural, or should we look to handle it through a different interaction
>>with the local forwarder?)
>> 
>> Thanks,
>> Jeff
>> 
>> 
>> From: <Ignacio.Solis at parc.com>
>> Date: Tue, 31 Mar 2015 00:06:33 +0000
>> To: <wentaoshang at gmail.com>, <Marc.Mosko at parc.com>
>> 
>> Cc: <nfd-dev at lists.cs.ucla.edu>
>> Subject: Re: [Nfd-dev] Handling new content in app for pending interest
>>in NFD
>> 
>> I'm going to guess that caching data rules will be the same as
>>forwarding rules. If the interface is not allowed to produce that
>>content (due to routing / fib entries, local naming policy) data should
>>not be forwarded OR cached.
>> 
>> NDN has a slightly more complicated problem with caching than CCN given
>>then LPM matching rules.
>> 
>> If you have a repo that has permission to serve this content then you
>>changed the problem. Now you need the repo protocol to give you trust.
>> 
>> For regular forwarding you are relying on trusting routing. For local
>>apps you will rely (I assume)  on local permissions. For repo you will
>>need both of those plus now a repo permission  protocol.
>> 
>> Dropping unsolicited data may end up being your only recourse.
>> 
>> There are obviously other approaches for pre-populating caches that
>>involve higher level protocols, but I assume that's not what you're
>>talking about. 
>> 
>> Nacho (Ignacio) Solis
>> Principal Scientist
>> Palo Alto Research Center
>> 
>> From: Wentao Shang <wentaoshang at gmail.com>
>> Sent: Mar 30, 2015 4:50 PM
>> To: "Solis, Ignacio <Ignacio.Solis at parc.com>"
>><Ignacio.Solis at parc.com>;Mosko, Marc <Marc.Mosko at parc.com>
>> Cc: nfd-dev at lists.cs.ucla.edu
>> Subject: Re: [Nfd-dev] Handling new content in app for pending interest
>>in NFD
>> 
>> Hi Nacho,
>> 
>> I don't have any argument against this rule right now. But it seems
>>more relevant to "forwarding" data packets, while the problem we were
>>discussing is about "caching" unsolicited data packets, which cannot be
>>forwarded anyway because there is no PIT. The simplest solution I would
>>suggest to the problem is to drop any unsolicited data in the forwarder.
>> 
>> Wentao
>> 
>> On Mon, Mar 30, 2015 at 3:42 PM <Ignacio.Solis at parc.com> wrote:
>> This rule is important, router or not. You can't have "non-priviledged"
>> applications generating replies at servers. Specially important at
>>systems with multiple users,  tenants, virtual systems.
>> 
>> At the local router there are even more restrictions.  We haven't even
>>gotten to talking about privileged names and restrictions and end-h out
>>behavior.   We have a design  for this for CCN that we'll be talking
>>about at CCNxCon.
>> 
>> Nacho (Ignacio) Solis
>> Principal Scientist
>> Palo Alto Research Center
>> 
>> From: Wentao Shang <wentaoshang at gmail.com>
>> Sent: Mar 30, 2015 1:07 PM
>> To: Mosko, Marc <Marc.Mosko at parc.com>
>> 
>> Cc: nfd-dev at lists.cs.ucla.edu
>> Subject: Re: [Nfd-dev] Handling new content in app for pending interest
>>in NFD
>> 
>> On Mon, Mar 30, 2015 at 12:52 PM <Marc.Mosko at parc.com> wrote:
>> I¹ll add that in addition to the content having a PIT entry, Nacho
>>Solis has suggested that a router should verify that a Content Object
>>came in a face from which it was requested.  Otherwise, there¹s a
>>potential for off-path attacks sending content objects that match
>>popular well-known names.  A variation on that, rather than tracking
>>forward paths, would be to at minimum verify that a content object comes
>>from a face for which there¹s a corresponding FIB entry for the name.
>> 
>> This requirement makes sense for transit routers that receive packets
>>from other routers. But I'm not sure whether this is necessary for
>>forwarding daemons handling local applications... The pushed data should
>>never get out of the local forwarder unless there is a PIT entry
>>specifying a path pointing to some other router.
>> 
>> Wentao
>>  
>> 
>> Marc
>> 
>> On Mar 30, 2015, at 12:08 PM, Wentao Shang <wentaoshang at gmail.com>
>>wrote:
>> 
>>> I agree with Dave.
>>> 
>>> In principle, router should not cache unsolicited data. In the
>>>situation we are discussing, the application should either just push
>>>the data out, which may be dropped by NFD if the interest has expired,
>>>or store the data in some application-level cache (or repo) for future
>>>fetching.
>>> 
>>> Wentao
>>> 
>>> On Mon, Mar 30, 2015 at 11:52 AM Dave Oran (oran) <oran at cisco.com>
>>>wrote:
>>> Isn¹t this what the repo was invented for?
>>> 
>>> Holding packets in a router that has forgotten that they were asked
>>>for is a giant invitation to cache pollution/poisoning attacks.
>>> 
>>> > On Mar 30, 2015, at 2:24 PM, Haowei Yuan <hyuan at wustl.edu> wrote:
>>> >
>>> > I think as long as the data has actually been requested by an
>>>Interest
>>> > packet, it is safe to send the Data packet to NFD. The NFD will
>>>either
>>> > forward or drop the Data packet by checking if the Interest has
>>> > expired.
>>> >
>>> > If the Interest has expired, Data packet is dropped, and the consumer
>>> > is still interested in the data, the consumer could resend the
>>> > Interest. Hopefully this time, the Data packet can be generated and
>>> > sent faster by the application so that NFD will forward it.
>>> >
>>> > Haowei
>>> >
>>> >
>>> > On Mon, Mar 30, 2015 at 1:12 PM, Anil Jangam
>>><anilj.mailing at gmail.com> wrote:
>>> >>
>>> >> On Mar 30, 2015 11:08 AM, "Dehart, John" <jdd at wustl.edu> wrote:
>>> >>>
>>> >>>
>>> >>> Is there any harm in it pushing the data out without knowing for
>>>sure if
>>> >>> the
>>> >>> Interest is still active?
>>> >>>
>>> >> If data is so critical, can the Interest be refreshed proactively
>>>before it
>>> >> expires?
>>> >>
>>> >> /anil
>>> >>
>>> >>> John
>>> >>>
>>> >>>> On Mar 30, 2015, at 1:05 PM, Burke, Jeff <jburke at remap.ucla.edu>
>>>wrote:
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> On Mon, Mar 30, 2015 at 10:28 AM Burke, Jeff
>>><jburke at remap.ucla.edu>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> Hi folks,
>>> >>>>>>
>>> >>>>>> We are facing this scenario in a few applications:
>>> >>>>>>
>>> >>>>>> 1) Interest received by NFD, passed to an application
>>> >>>>>> 2) Application not able to respond to interest, so interest
>>>stays in
>>> >>>>>> NFD PIT
>>> >>>>>> 3) Some time passes, but not enough for the Interest to expire
>>> >>>>>> 4) Application generates data (e.g., from a sensor reading)
>>>that would
>>> >>>>>> answer the Interest in the NFD PIT
>>> >>>>>>
>>> >>>>>> Question: How does app know to inform NFD it has the data after
>>>step 4,
>>> >>>>>> and how should it do that?
>>> >>>>>>
>>> >>>>>> - In this type of app, should it push the data unsolicited to
>>>the NFD
>>> >>>>>> and let it decide if there is something to do?
>>> >>>>>
>>> >>>>>
>>> >>>>> In my opinion, as long as the application is certain that the
>>>Interest
>>> >>>>> has arrived and is stored in NFD's PIT, it can just push the
>>>data out to
>>> >>>>> NFD.
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> How certain does it have to be?  There is a chance it could have
>>> >>>> expired...
>>> >>>> jeff
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>>
>>> >>>>> Wentao
>>> >>>>>
>>> >>>>>>
>>> >>>>>> - Is it recommended to implement an application-level PIT so
>>>the app is
>>> >>>>>> sure this data is solicited?  (Why add another PIT?)
>>> >>>>>>
>>> >>>>>> Thank you,
>>> >>>>>> Jeff
>>> >>>>>>
>>> >>>>>> _______________________________________________
>>> >>>>>> Nfd-dev mailing list
>>> >>>>>> Nfd-dev at lists.cs.ucla.edu
>>> >>>>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Nfd-dev mailing list
>>> >>>> Nfd-dev at lists.cs.ucla.edu
>>> >>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>> >>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Nfd-dev mailing list
>>> >>> Nfd-dev at lists.cs.ucla.edu
>>> >>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>> >>>
>>> > _______________________________________________
>>> > Nfd-dev mailing list
>>> > Nfd-dev at lists.cs.ucla.edu
>>> > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>> 
>>> 
>>> _______________________________________________
>>> Nfd-dev mailing list
>>> Nfd-dev at lists.cs.ucla.edu
>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>> _______________________________________________
>>> Nfd-dev mailing list
>>> Nfd-dev at lists.cs.ucla.edu
>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>> 
>> _______________________________________________ Nfd-dev mailing list
>>Nfd-dev at lists.cs.ucla.edu
>>http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>> _______________________________________________
>> Nfd-dev mailing list
>> Nfd-dev at lists.cs.ucla.edu
>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>> _______________________________________________
>> Nfd-dev mailing list
>> Nfd-dev at lists.cs.ucla.edu
>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>

More information about the Nfd-dev mailing list