[Nfd-dev] Handling new content in app for pending interest in NFD

[Burke, Jeff]

Hi Junxiao,

Thanks!  It would be great to summarize  such things in a design document somewhere rather than relying on the mailing list traffic / issues... Has the team considered this?

One question:  The approach suggested by these recommendations seems to be motivated by having the application's in-memory cache play the role of the content store...  I'm not sure that necessarily they have the same goals.  (I have to think about this more.) For example, the app may just want to support a modest history (last few seconds of video, for example), while the forwarder may have its own goals that result in caching more or less than this.   So is what you're describing on the level of an architectural recommendation or simply the best approach we have for now... ?

Jeff


From: Junxiao Shi <shijunxiao at email.arizona.edu<mailto:shijunxiao at email.arizona.edu>>
Date: Wed, 1 Apr 2015 10:21:02 -0700
To: Jeff Burke <jburke at remap.ucla.edu<mailto:jburke at remap.ucla.edu>>
Cc: "nfd-dev at lists.cs.ucla.edu<mailto:nfd-dev at lists.cs.ucla.edu>" <nfd-dev at lists.cs.ucla.edu<mailto:nfd-dev at lists.cs.ucla.edu>>
Subject: Re: [Nfd-dev] Handling new content in app for pending interest in NFD

Hi Jeff

The recommendation for "soft realtime app" is given below.
Please note that some features are not yet implemented, but the specifications are already approved.
Watch each listed feature for implementation progress.


The forwarder MUST NOT cache unsolicited Data, regardless of its origin.
Feature: http://redmine.named-data.net/issues/2181
Announcement: http://www.lists.cs.ucla.edu/pipermail/nfd-dev/2014-November/000605.html

A soft realtime app MUST maintain an in-memory storage.
It MUST NOT rely on the forwarder to cache its Data, even if they are sent to the forwarder in response to an unexpired Interest.
The data structure is referred to as "in-memory storage" rather than "cache", because it SHOULD be fully managed by the app to keep all Data that the app wants to serve. It's not a cache where useful Data can be evicted.
Reason: the forwarder is free to evict any Data.

A soft realtime app SHOULD set CachingPolicy=NoCache in LocalControlHeader (or NDNLPv2 header) on a Data served from an in-memory storage.
The local forwarder ContentStore SHOULD NOT cache a Data with CachingPolicy=NoCache.
CachingPolicy field is not carried to remote forwarders, so that remote forwarders can still cache the Data. It avoids duplicate Data copies on localhost.
Protocol: http://redmine.named-data.net/projects/nfd/wiki/LocalControlHeader#CachingPolicy-feature
Feature: http://redmine.named-data.net/issues/2183 http://redmine.named-data.net/issues/2185 http://redmine.named-data.net/issues/2184

A soft realtime app MAY maintain its PIT to avoid sending unsolicited Data.
When an app chooses not to maintain a PIT, sending unsolicited Data has no harmful effect.



Yours, Junxiao

On Wed, Apr 1, 2015 at 8:38 AM, Burke, Jeff <jburke at remap.ucla.edu<mailto:jburke at remap.ucla.edu>> wrote:

Thanks for all of the discussion. I am not sure if I have been able to extract an answer to the original question I was asking.  Let me try again with more information:

- The original question was intended to be only about a local NFD (for now).
- The use case is a (soft) real-time application, like videoconferencing or streaming sensor data for interactive applications. Specifically, we have encountered the issue in 1) ndnrtc/ndncon<https://github.com/remap/ndncon/> for videoconferencing and 2) ndn-opt, which streams data from OpenPTrack<http://openptrack.org/>, a person tracking system built on the robot operating system (ROS).

- In these use cases, consumers want to receive data with low latency, so they send interests intended to arrive before the time the data is produced, but not so long before that the Interests expire before corresponding data is produced.

- As they are born, samples are signed and put into a repository by the publisher.  Currently these apps use an in-memory repo specific to the application, via the MemoryContentCache class provided by the NDN-CCL library. But this could just as well be a repo outside the application.

- So, when Interests arrive, they go in the local  NFD's PIT, and are forwarded to the application.   We are interested in the case where the data is not born yet, so the publisher cannot respond to the Interest.

- Then, before the Interest expires from the local NFD's PIT, the corresponding data is born in the application and should be used to answer the Interest.  But how does that data get back to the local forwarder?

- What should the application do?  1) Maintain its own PIT?   2) Or, push the data to the local NFD, letting it drop it if it doesn't satisfy anything.  3) Or, as we have previously discussed about the MemoryContentCache, do we need to consider a more sophisticated interaction model between the application and the daemon's content store and/or PIT that reduces duplication of these capabilities in applications?

- For now, we have implemented an in-application PIT in ndnrtc.  (I am not sure what method we ended up using in ndn-opt, perhaps the unsolicited data method.)

- Discussion on this topic will help inform what library support will be provided in the short term.  (I don't think we want every application author to have to write a PIT, so am thinking in the short term we should provide one to complement the MemoryContentCache class.  But this is duplicating data structures and functionality in NFD – is that natural, or should we look to handle it through a different interaction with the local forwarder?)

Thanks,
Jeff

On Mon, Mar 30, 2015 at 10:28 AM, Burke, Jeff <jburke at remap.ucla.edu<mailto:jburke at remap.ucla.edu>> wrote:

Hi folks,

We are facing this scenario in a few applications:

1) Interest received by NFD, passed to an application
2) Application not able to respond to interest, so interest stays in NFD PIT
3) Some time passes, but not enough for the Interest to expire
4) Application generates data (e.g., from a sensor reading) that would answer the Interest in the NFD PIT

Question: How does app know to inform NFD it has the data after step 4, and how should it do that?

- In this type of app, should it push the data unsolicited to the NFD and let it decide if there is something to do?
- Is it recommended to implement an application-level PIT so the app is sure this data is solicited?  (Why add another PIT?)

Thank you,
Jeff

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20150401/a8c42935/attachment.html>