[Nfd-dev] [Ndn-app] Close NFD backdoor

Junxiao Shi shijunxiao at email.arizona.edu
Tue Sep 9 21:50:18 PDT 2014 

Dear folks

Below is a summary of the problem, and a comparison of three solutions.
"In-Forwarder Storage" is my proposed solution.

Yours, Junxiao

BackgroundContentStore concept

ContentStore is an element of NFD which serves as an *opportunistic cache*.
Incoming Data packets have the opportunity to be admitted into the
ContentStore. Data packets are not persistent in ContentStore: they can be
evicted when ContentStore is full.

Admission to the ContentStore is a two step process:

   1. Forwarding pipelines must decide whether a Data packet *SHOULD* be
   admitted. A Data packet should be admitted if it's believed to be
   legitimate (ie. not part of an attack).
   2. ContentStore must decide whether there is space for this Data packet.

Data packets in the ContentStore can be used to satisfy incoming Interests.
When an incoming Interest contains *MustBeFresh=yes* selector, only
*non-stale* Data packets can be returned. A Data packet is *stale* if it
has been in the ContentStore for longer than*FreshnessPeriod*; note that if
a Data packet is admitted while the same Data packet already exists in the
ContentStore, the admission timestamp is reset, so this is equivalent to
deleting the old Data packet and admitting the new one.

Retention in the ContentStore is not guaranteed. The ContentStore is
permitted to *evict* any Data packet at any time, regardless of its origin,
staleness, admission time, etc.
ContentStore in NFD 0.2.0

*Admission*: *Incoming Data pipeline* determines whether a Data packet
*SHOULD* be admitted. If a Data packet satisfies one or more PIT entries,
it should be admitted; this is regardless of whether an Interest has been
forwarded to the origin of this Data packet. Otherwise, the Data packet is
said to be *unsolicited*, and decision is given to *Data unsolicited
pipeline*.

Generally, unsolicited Data needs to be dropped as it poses security risks
to the forwarder. However, there are cases when unsolicited Data packets
needs to be accepted to the ContentStore. In particular, the current
implementation allows any unsolicited Data packet to be cached if this Data
packet arrives from a local Face.

Once forwarding pipelines decide a Data should be admitted, ContentStore
always admits it. If the ContentStore is full and a new Data packet needs
to be admitted, one old Data packet is evicted.

*Eviction*: The ContentStore logically maintains three FIFO queues:

   1. unsolicited Data
   2. stale Data
   3. fresh Data

The front Data packet in the first non-empty queue is evicted.
Data Generation in ApplicationData Generation Misaligned with Interest
Arrival

NDN content distribution is receiver-driven. The producer cannot push Data
packets into the network; it must wait for consumer to express Interests,
before responding with Data packets.

However, *Data generation can happen before Interest arrival*.

   - a video camera captures a consequtive video stream, and generates Data
   packets as a frame is captured
   - a NFD StatusDataset
   <http://redmine.named-data.net/projects/nfd/wiki/StatusDataset> publisher
   generates all segments of a StatusDataset in response to a request, not one
   segment at a time
   - a ChronoChat <https://github.com/bruinfish/ChronoChat> client
   generates a Data packet when the user enters a message, before other users
   discover the existence of this Data packet (through synchronization) and
   retrieve it
   - a NLSR <https://github.com/named-data/NLSR> daemon generates a Data
   packet for a Link State Announcement, before other routers discover the
   existence of this Data packet (through synchronization) and retrieve it

Producer is not allowed to send those Data packets into the network before
Interest arrival. Therefore, those Data packets must be stored *somewhere*.
Latest Version and Short Freshness Period

The most useful Data packets are those representing the latest version of
some information. The producer has the final authority: when an incoming
Interest asks for the latest version, the producer would respond with the
latest version.

*FreshnessPeriod* field is an indication about how long the Data packet may
remain the latest version. Typically, the producer sets this field to a
duration in which it believes a newer version won't be generated. However,
two situations cause the producer to prefer a shorter *FreshnessPeriod*:

   - *FreshnessPeriod* is per network node: if a Data packet is near the
   end of its freshness period at one node, and it's delivered to another
   node, the admitted Data packet on the second node would remain
   *non-stale* for the entire *FreshnessPeriod*. This means, if there are N
   hops between consumer and producer, in the worst case, the Data may appear
   fresh to consumer for up to N times*FreshnessPeriod*.
   - The producer is sometimes unable to predict how long a version can
   remain to be the latest version.

Problem

The "NFD backdoor" is a *feature*: NFD allows unsolicited Data from a local
face to be admitted to the ContentStore.

This feature allows NDN applications to "pre-publish" Data packets when
future Interests are anticipated. At the other end, the NDN applications
themselves do not retain the Data packet.

The drawback of this approach is: *retention is not guaranteed*.
Although pre-published Data packets are admitted to the ContentStore, they
are tagged as unsolicited. Unsolicited Data packets are the first to be
evicted from ContentStore when ContentStore is full. Therefore, when there
are enough solicited Data packets in the ContentStore, unsolicited Data
packets can hardly be retained in the ContentStore.
If a Data packet is evicted, the Interest would be forwarded to the
application. The application either has to generate the Data again, or is
unable to respond at all.

Another problem of application not storing generated Data packets is: *a
Data packet may remain to be the latest version beyond FreshnessPeriod*.
The ContentStore can only use *non-stale* Data to satisfy an Interest with
*MustBeFresh=yes*. If the producer sets a short *FreshnessPeriod*, the
Interest would be forwarded to the application, even if the Data packet is
still the latest version.
The application would have to generate (and sign) the Data again.
Necessity of Application-Controllable Data Storage

Both problems have a common solution: a Data storage controlled by
application.

This Data storage should have the following characteristics:

   - Data packets *useful* to a producer application are stored. A Data
   packet is *useful*, if the producer application would generate the same
   again when an Interest asks for it.
   - Producer can explicitly delete Data packets that are no longer *useful*.
   The most common time to delete Data packets is when a new version is
   generated.
   - Producer should delete Data packets when the Data storage is close to
   its capacity. Such deletions can either be explicit, or be delegated to an
   eviction policy.
   - The Data storage is not a cache.

Solutions

The application-controllable Data storage can either be placed in the
application, or be placed in the forwarder.
In-App Storage

In-App Storage
<http://redmine.named-data.net/projects/ndn-cxx/wiki/InMemoryStorage> provides
an application-controllable Data storage within the application. The module
is typically provided by the library.

Benefits:

   - keep the forwarder simple
   - application has a chance to inspect the Data again before using it to
   satisfy incoming Interest

Drawbacks:

   - duplication: when a Data packet is used to satisfy an Interest, it
   would have copies in both In-App Storage and ContentStore
   - risk of Data loss: when a Data packet is used to satisfy an Interest,
   if application chooses to delete the copy in In-App Storage, the Data would
   be lost when the copy in ContentStore is evicted

Managed ContentStore

Managed ContentStore
<http://redmine.named-data.net/projects/nfd/wiki/ManagedContentStore> provides
an application-controllable Data storage as part of the ContentStore. A
Data packet is explicitly inserted to the ContentStore, and the
ContentStore guarantees to retain the Data packet until it becomes *stale* or
the application requests to delete it.

Benefits:

   - no duplication

Drawbacks:

   - ContentStore is no longer a cache; eviction policy becomes more complex
   - application must refresh the stored Data packet, if it remains the
   latest version beyond *FreshnessPeriod*; setting a large
   *FreshnessPeriod* is still infeasible because in-network caches may
   retain the Data packet for too long, this means refreshing would be very
   frequent
   - higher deletion cost: deletion needs a command
   - inability to enumerate: unlike In-App Storage, the application is
   unable to enumerate stored Data packets

In-Forwarder Storage

An application-controllable Data storage can be placed in the forwarder,
but out of the ContentStore.

   - The application sends a command to create an application-controllable
   Data storage in the forwarder.
      - The Data storage is logically attached to the face of this
      application. Its capacity and replacement policy are determined when it's
      created. Its destroyed when face is closing.
   - To insert a Data packet to the Data storage, the application must
   explicitly set a field in the LocalControlHeader.
      - Insertion to the Data storage is independant from forwarding
      pipelines, but the application can elect to deliver the same Data packet
      into forwarding pipelines.
      - *Data unsolicited pipeline* would not admit unsolicited Data
      packets to the ContentStore, except in special environments such
as vehicle
      networks.
   - The application can scheduled the deletion of a Data packet during
   insertion.
      - The deletion timer can be set differently from *FreshnessPeriod*.
   - The application can delete Data packets by sending a command.
      - Command parameter is a Name prefix. All stored Data packets under
      this prefix are deleted.
   - Stored Data packets are considered only if an Interest is to be
   forwarded to the application.
      - When an Interest is to be forwarded to the application, the Data
      storage attached to the face is queried. If a matching Data packet is
      found, the Interest is not sent, and the Data packet is treated
as if it's
      received from the application.

Benefits:

   - no duplication: if a Data packet appears in both
   application-controllable Data storage and the ContentStore, it has only one
   in-memory copy, with references from two places
   - less inter-process communication and encoding/decoding, when most
   stored Data packets are requested more than once
   - no risk of cache poisoning: the application-controllable Data storage
   is not considered until an Interest is to be forwarded to the application
   - no refreshing needed

Drawbacks:

   - higher deletion cost: deletion needs a command
   - inability to enumerate: unlike In-App Storage, the application is
   unable to enumerate stored Data packets
   - address space consumption: stored Data packets are mapped in
   forwarder's address space, and address space is limited on 32-bit platforms
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20140909/9dce95ae/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: contentstore-appstorage_20140909.md
Type: application/octet-stream
Size: 10882 bytes
Desc: not available
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20140909/9dce95ae/attachment.obj>

More information about the Nfd-dev mailing list