[Nfd-dev] Handling new content in app for pending interest in NFD

Wed Apr 1 10:25:06 PDT 2015

I think this might depend the application's behavior and requirements.

If the app, by design, knows that new data generation is always
triggered by an Interest request, then when data becomes available, it
would be ok to just assume there is a pending Interest in NFD, and NFD
make the decision.

If the app generates lots of unsolicited data (for example, a user
starts to publish video on ndncon, but there is no consumer..). I'm
not sure if it is a good idea to consult the NFD's PIT for every data
in this case.

In addition, there might be cases where apps want to keep local states
for generating replies. For instance, some app may like to send a
message if no data is found within some time. Current web applications
probably also need to take care of this.

Haowei

On Wed, Apr 1, 2015 at 11:59 AM, Wentao Shang <wentaoshang at gmail.com> wrote:
> I think this problem shows one limitation of in-memory cache (and repo in
> general). The cache/repo is a "passive" storage: once the data gets in, it
> never gets out unless someone explicitly request it. The applications, on
> the other hand, can be more "active" in deciding when to send out data in
> response to previous interests.
>
> I remember Illya and Lijing's ndnvideo implementation also needs to handle
> similar problem. Their current solution (as far as I understand) is to push
> out the data twice: once to the NFD for satisfying request, once to the repo
> for archiving. The second push uses repo insertion command so it's not just
> a plain data packet (like in the first push).
>
> Wentao
>
>
> On Wed, Apr 1, 2015 at 8:38 AM Burke, Jeff <jburke at remap.ucla.edu> wrote:
>>
>>
>> Thanks for all of the discussion. I am not sure if I have been able to
>> extract an answer to the original question I was asking.  Let me try again
>> with more information:
>>
>> - The original question was intended to be only about a local NFD (for
>> now).
>> - The use case is a (soft) real-time application, like videoconferencing
>> or streaming sensor data for interactive applications. Specifically, we have
>> encountered the issue in 1) ndnrtc/ndncon for videoconferencing and 2)
>> ndn-opt, which streams data from OpenPTrack, a person tracking system built
>> on the robot operating system (ROS).
>>
>> - In these use cases, consumers want to receive data with low latency, so
>> they send interests intended to arrive before the time the data is produced,
>> but not so long before that the Interests expire before corresponding data
>> is produced.
>>
>> - As they are born, samples are signed and put into a repository by the
>> publisher.  Currently these apps use an in-memory repo specific to the
>> application, via the MemoryContentCache class provided by the NDN-CCL
>> library. But this could just as well be a repo outside the application.
>>
>> - So, when Interests arrive, they go in the local  NFD's PIT, and are
>> forwarded to the application.   We are interested in the case where the data
>> is not born yet, so the publisher cannot respond to the Interest.
>>
>> - Then, before the Interest expires from the local NFD's PIT, the
>> corresponding data is born in the application and should be used to answer
>> the Interest.  But how does that data get back to the local forwarder?
>>
>> - What should the application do?  1) Maintain its own PIT?   2) Or, push
>> the data to the local NFD, letting it drop it if it doesn't satisfy
>> anything.  3) Or, as we have previously discussed about the
>> MemoryContentCache, do we need to consider a more sophisticated interaction
>> model between the application and the daemon's content store and/or PIT that
>> reduces duplication of these capabilities in applications?
>>
>> - For now, we have implemented an in-application PIT in ndnrtc.  (I am not
>> sure what method we ended up using in ndn-opt, perhaps the unsolicited data
>> method.)
>>
>> - Discussion on this topic will help inform what library support will be
>> provided in the short term.  (I don't think we want every application author
>> to have to write a PIT, so am thinking in the short term we should provide
>> one to complement the MemoryContentCache class.  But this is duplicating
>> data structures and functionality in NFD – is that natural, or should we
>> look to handle it through a different interaction with the local forwarder?)
>>
>> Thanks,
>> Jeff
>>
>>
>> From: <Ignacio.Solis at parc.com>
>> Date: Tue, 31 Mar 2015 00:06:33 +0000
>> To: <wentaoshang at gmail.com>, <Marc.Mosko at parc.com>
>>
>> Cc: <nfd-dev at lists.cs.ucla.edu>
>> Subject: Re: [Nfd-dev] Handling new content in app for pending interest in
>> NFD
>>
>> I'm going to guess that caching data rules will be the same as forwarding
>> rules. If the interface is not allowed to produce that content (due to
>> routing / fib entries, local naming policy) data should not be forwarded OR
>> cached.
>>
>> NDN has a slightly more complicated problem with caching than CCN given
>> then LPM matching rules.
>>
>> If you have a repo that has permission to serve this content then you
>> changed the problem. Now you need the repo protocol to give you trust.
>>
>> For regular forwarding you are relying on trusting routing. For local apps
>> you will rely (I assume)  on local permissions. For repo you will need both
>> of those plus now a repo permission  protocol.
>>
>> Dropping unsolicited data may end up being your only recourse.
>>
>> There are obviously other approaches for pre-populating caches that
>> involve higher level protocols, but I assume that's not what you're talking
>> about.
>>
>> Nacho (Ignacio) Solis
>> Principal Scientist
>> Palo Alto Research Center
>>
>> From: Wentao Shang <wentaoshang at gmail.com>
>> Sent: Mar 30, 2015 4:50 PM
>> To: "Solis, Ignacio <Ignacio.Solis at parc.com>"
>> <Ignacio.Solis at parc.com>;Mosko, Marc <Marc.Mosko at parc.com>
>> Cc: nfd-dev at lists.cs.ucla.edu
>> Subject: Re: [Nfd-dev] Handling new content in app for pending interest in
>> NFD
>>
>> Hi Nacho,
>>
>> I don't have any argument against this rule right now. But it seems more
>> relevant to "forwarding" data packets, while the problem we were discussing
>> is about "caching" unsolicited data packets, which cannot be forwarded
>> anyway because there is no PIT. The simplest solution I would suggest to the
>> problem is to drop any unsolicited data in the forwarder.
>>
>> Wentao
>>
>> On Mon, Mar 30, 2015 at 3:42 PM <Ignacio.Solis at parc.com> wrote:
>>>
>>> This rule is important, router or not. You can't have "non-priviledged"
>>> applications generating replies at servers. Specially important at systems
>>> with multiple users,  tenants, virtual systems.
>>>
>>> At the local router there are even more restrictions.  We haven't even
>>> gotten to talking about privileged names and restrictions and end-h out
>>> behavior.   We have a design  for this for CCN that we'll be talking about
>>> at CCNxCon.
>>>
>>> Nacho (Ignacio) Solis
>>> Principal Scientist
>>> Palo Alto Research Center
>>>
>>> From: Wentao Shang <wentaoshang at gmail.com>
>>> Sent: Mar 30, 2015 1:07 PM
>>> To: Mosko, Marc <Marc.Mosko at parc.com>
>>>
>>> Cc: nfd-dev at lists.cs.ucla.edu
>>> Subject: Re: [Nfd-dev] Handling new content in app for pending interest
>>> in NFD
>>>
>>> On Mon, Mar 30, 2015 at 12:52 PM <Marc.Mosko at parc.com> wrote:
>>>>
>>>> I’ll add that in addition to the content having a PIT entry, Nacho Solis
>>>> has suggested that a router should verify that a Content Object came in a
>>>> face from which it was requested.  Otherwise, there’s a potential for
>>>> off-path attacks sending content objects that match popular well-known
>>>> names.  A variation on that, rather than tracking forward paths, would be to
>>>> at minimum verify that a content object comes from a face for which there’s
>>>> a corresponding FIB entry for the name.
>>>
>>>
>>> This requirement makes sense for transit routers that receive packets
>>> from other routers. But I'm not sure whether this is necessary for
>>> forwarding daemons handling local applications... The pushed data should
>>> never get out of the local forwarder unless there is a PIT entry specifying
>>> a path pointing to some other router.
>>>
>>> Wentao
>>>
>>>>
>>>>
>>>> Marc
>>>>
>>>> On Mar 30, 2015, at 12:08 PM, Wentao Shang <wentaoshang at gmail.com>
>>>> wrote:
>>>>
>>>> I agree with Dave.
>>>>
>>>> In principle, router should not cache unsolicited data. In the situation
>>>> we are discussing, the application should either just push the data out,
>>>> which may be dropped by NFD if the interest has expired, or store the data
>>>> in some application-level cache (or repo) for future fetching.
>>>>
>>>> Wentao
>>>>
>>>> On Mon, Mar 30, 2015 at 11:52 AM Dave Oran (oran) <oran at cisco.com>
>>>> wrote:
>>>>>
>>>>> Isn’t this what the repo was invented for?
>>>>>
>>>>> Holding packets in a router that has forgotten that they were asked for
>>>>> is a giant invitation to cache pollution/poisoning attacks.
>>>>>
>>>>> > On Mar 30, 2015, at 2:24 PM, Haowei Yuan <hyuan at wustl.edu> wrote:
>>>>> >
>>>>> > I think as long as the data has actually been requested by an
>>>>> > Interest
>>>>> > packet, it is safe to send the Data packet to NFD. The NFD will
>>>>> > either
>>>>> > forward or drop the Data packet by checking if the Interest has
>>>>> > expired.
>>>>> >
>>>>> > If the Interest has expired, Data packet is dropped, and the consumer
>>>>> > is still interested in the data, the consumer could resend the
>>>>> > Interest. Hopefully this time, the Data packet can be generated and
>>>>> > sent faster by the application so that NFD will forward it.
>>>>> >
>>>>> > Haowei
>>>>> >
>>>>> >
>>>>> > On Mon, Mar 30, 2015 at 1:12 PM, Anil Jangam
>>>>> > <anilj.mailing at gmail.com> wrote:
>>>>> >>
>>>>> >> On Mar 30, 2015 11:08 AM, "Dehart, John" <jdd at wustl.edu> wrote:
>>>>> >>>
>>>>> >>>
>>>>> >>> Is there any harm in it pushing the data out without knowing for
>>>>> >>> sure if
>>>>> >>> the
>>>>> >>> Interest is still active?
>>>>> >>>
>>>>> >> If data is so critical, can the Interest be refreshed proactively
>>>>> >> before it
>>>>> >> expires?
>>>>> >>
>>>>> >> /anil
>>>>> >>
>>>>> >>> John
>>>>> >>>
>>>>> >>>> On Mar 30, 2015, at 1:05 PM, Burke, Jeff <jburke at remap.ucla.edu>
>>>>> >>>> wrote:
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Mon, Mar 30, 2015 at 10:28 AM Burke, Jeff
>>>>> >>>>> <jburke at remap.ucla.edu>
>>>>> >>>>> wrote:
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> Hi folks,
>>>>> >>>>>>
>>>>> >>>>>> We are facing this scenario in a few applications:
>>>>> >>>>>>
>>>>> >>>>>> 1) Interest received by NFD, passed to an application
>>>>> >>>>>> 2) Application not able to respond to interest, so interest
>>>>> >>>>>> stays in
>>>>> >>>>>> NFD PIT
>>>>> >>>>>> 3) Some time passes, but not enough for the Interest to expire
>>>>> >>>>>> 4) Application generates data (e.g., from a sensor reading) that
>>>>> >>>>>> would
>>>>> >>>>>> answer the Interest in the NFD PIT
>>>>> >>>>>>
>>>>> >>>>>> Question: How does app know to inform NFD it has the data after
>>>>> >>>>>> step 4,
>>>>> >>>>>> and how should it do that?
>>>>> >>>>>>
>>>>> >>>>>> - In this type of app, should it push the data unsolicited to
>>>>> >>>>>> the NFD
>>>>> >>>>>> and let it decide if there is something to do?
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> In my opinion, as long as the application is certain that the
>>>>> >>>>> Interest
>>>>> >>>>> has arrived and is stored in NFD's PIT, it can just push the data
>>>>> >>>>> out to
>>>>> >>>>> NFD.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> How certain does it have to be?  There is a chance it could have
>>>>> >>>> expired...
>>>>> >>>> jeff
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>>
>>>>> >>>>> Wentao
>>>>> >>>>>
>>>>> >>>>>>
>>>>> >>>>>> - Is it recommended to implement an application-level PIT so the
>>>>> >>>>>> app is
>>>>> >>>>>> sure this data is solicited?  (Why add another PIT?)
>>>>> >>>>>>
>>>>> >>>>>> Thank you,
>>>>> >>>>>> Jeff
>>>>> >>>>>>
>>>>> >>>>>> _______________________________________________
>>>>> >>>>>> Nfd-dev mailing list
>>>>> >>>>>> Nfd-dev at lists.cs.ucla.edu
>>>>> >>>>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>>>> >>>>
>>>>> >>>> _______________________________________________
>>>>> >>>> Nfd-dev mailing list
>>>>> >>>> Nfd-dev at lists.cs.ucla.edu
>>>>> >>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> _______________________________________________
>>>>> >>> Nfd-dev mailing list
>>>>> >>> Nfd-dev at lists.cs.ucla.edu
>>>>> >>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>>>> >>>
>>>>> > _______________________________________________
>>>>> > Nfd-dev mailing list
>>>>> > Nfd-dev at lists.cs.ucla.edu
>>>>> > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Nfd-dev mailing list
>>>>> Nfd-dev at lists.cs.ucla.edu
>>>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>>>
>>>> _______________________________________________
>>>> Nfd-dev mailing list
>>>> Nfd-dev at lists.cs.ucla.edu
>>>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>>>
>>>>
>> _______________________________________________ Nfd-dev mailing list
>> Nfd-dev at lists.cs.ucla.edu
>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev
>>
>> _______________________________________________
>> Nfd-dev mailing list
>> Nfd-dev at lists.cs.ucla.edu
>> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev