From shijunxiao at email.arizona.edu Tue Sep 2 10:26:25 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 2 Sep 2014 10:26:25 -0700 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Hi Lan Thanks for the report. This shall be a bug of NCC strategy. In ccnd, 1. When first Data comes back that satisfies the PIT entry, the incoming face of that Data is remembered. 2. The PIT entry is deleted as soon as it's satisfied. 3. When second Data comes back, it's discarded as unsolicited. In NFD, 1. When first Data comes back that satisfies the PIT entry, the incoming face of that Data is remembered. 2. The PIT entry is kept, and will be deleted after straggler timer. 3. When second Data comes back, the strategy is notified, and remembers the incoming face. NFD's NCC strategy incorrectly assumes the incoming Data is from the fastest upstream, causing the bug. I'll report the bug soon. Yours, Junxiao On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) wrote: > Junxiao, > > We observed something strange in one of our experiments when we used the > ncc strategy (and link state routing). In some of the pings from Arizona > to PKU, the ndnping delays were really large (>350ms) even though the > shortest path delay is only around 200ms. We looked at the ndnping data, > and found where the delay jumped from ~200ms to ~350ms. It turns out that > instead of going through the shortest path (Arizona->UCLA->PKU), it is > going through a much longer path (Arizona->CSU->?->NEU->PKU). > > To explain what happened, I'll call the ping before the change P1 and the > ping after the change P2. Further looking into the nfd logs, it seems for > P1 Arizona first sent the Interest to UCLA (which we expected), but while > waiting for the data to come back, it also tried CSU and some other > neighbors. The ping Data did come back from the shortest path > (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes > back from the other much longer paths. The problem is that Arizona seems > to remember the most recent successful paths instead of the previously > successful shorter paths. Next time when P2 came, it directly sent the > Interest to CSU instead of UCLA. > > Is this expected behavior of ncc? I want to emphasize that the shorter > path did return the same Ping data (and much earlier than the longer path). > > Lan -------------- next part -------------- An HTML attachment was scrubbed... URL: From obaidasyed at gmail.com Tue Sep 2 11:05:11 2014 From: obaidasyed at gmail.com (Syed Obaid Amin) Date: Tue, 2 Sep 2014 13:05:11 -0500 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Thanks Junxiao for the insight. Any idea when you'll be able to fix this bug? Obaid On Tue, Sep 2, 2014 at 12:26 PM, Junxiao Shi wrote: > Hi Lan > > Thanks for the report. This shall be a bug of NCC strategy. > > In ccnd, > > 1. When first Data comes back that satisfies the PIT entry, the > incoming face of that Data is remembered. > 2. The PIT entry is deleted as soon as it's satisfied. > 3. When second Data comes back, it's discarded as unsolicited. > > In NFD, > > 1. When first Data comes back that satisfies the PIT entry, the > incoming face of that Data is remembered. > 2. The PIT entry is kept, and will be deleted after straggler timer. > 3. When second Data comes back, the strategy is notified, and > remembers the incoming face. > > NFD's NCC strategy incorrectly assumes the incoming Data is from the > fastest upstream, causing the bug. > > I'll report the bug soon. > > Yours, Junxiao > > > On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) > wrote: > >> Junxiao, >> >> We observed something strange in one of our experiments when we used the >> ncc strategy (and link state routing). In some of the pings from Arizona >> to PKU, the ndnping delays were really large (>350ms) even though the >> shortest path delay is only around 200ms. We looked at the ndnping data, >> and found where the delay jumped from ~200ms to ~350ms. It turns out that >> instead of going through the shortest path (Arizona->UCLA->PKU), it is >> going through a much longer path (Arizona->CSU->?->NEU->PKU). >> >> To explain what happened, I'll call the ping before the change P1 and the >> ping after the change P2. Further looking into the nfd logs, it seems for >> P1 Arizona first sent the Interest to UCLA (which we expected), but while >> waiting for the data to come back, it also tried CSU and some other >> neighbors. The ping Data did come back from the shortest path >> (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes >> back from the other much longer paths. The problem is that Arizona seems >> to remember the most recent successful paths instead of the previously >> successful shorter paths. Next time when P2 came, it directly sent the >> Interest to CSU instead of UCLA. >> >> Is this expected behavior of ncc? I want to emphasize that the shorter >> path did return the same Ping data (and much earlier than the longer path). >> >> Lan > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Tue Sep 2 17:03:01 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 2 Sep 2014 17:03:01 -0700 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Hi Obaid Please watch Bug 1961 for progress. This Bug is assigned priority=High so you can expect a fix before Issues with lower priorities. I cannot promise any specific date at this moment. Yours, Junxiao On Tue, Sep 2, 2014 at 11:05 AM, Syed Obaid Amin wrote: > Thanks Junxiao for the insight. Any idea when you'll be able to fix this > bug? > > Obaid > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From agawande at memphis.edu Tue Sep 2 17:08:58 2014 From: agawande at memphis.edu (Ashlesh Gawande (agawande)) Date: Wed, 3 Sep 2014 00:08:58 +0000 Subject: [Nfd-dev] Fwd: Re: Help needed with PyNDN2 Message-ID: Hi I was working with PyNDN2 library which uses NFD. In PyNDN2 Is there a way to check if NFD has gone down (or should there be a way to that in the library) ? If so how can it be done? (I have added the conversation I had with Jeff T) Thanks Ashlesh ---------- Forwarded message ---------- From: "Thompson, Jeff" Date: Sep 2, 2014 6:38 PM Subject: Re: Help needed with PyNDN2 To: "Ashlesh Gawande (agawande)" Cc: > Hi Ashlesh. This is a good question which applies to all the libraries. ?Are you on the nfd-dev list? ?Can you send a question there asking whether the library should monitor whether the connected NFD is up, and how to do that? > > Thanks, > - Jeff T > > From: "Ashlesh Gawande (agawande)" > Date: Tuesday, September 2, 2014 1:21 PM > To: Jeff Thompson > Subject: Help needed with PyNDN2 > > Hi Jeff > > > In PyNDN2?Is there a way to check if NFD has gone down? Since we need to re-register our prefixes when NFD comes back up. > > > Regards > > Ashlesh From shijunxiao at email.arizona.edu Tue Sep 2 17:25:25 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 2 Sep 2014 17:25:25 -0700 Subject: [Nfd-dev] Fwd: Re: Help needed with PyNDN2 In-Reply-To: References: Message-ID: Hi Ashlesh I don't known PyNDN so I cannot answer the specific question. The answer below is applicable to library design in general. A client library can choose what to do when the forwarder has crashed. In most cases, the library would crash or raise an error if the connection to the server is lost. This is common among popular client libraries such as libmysql. ndn-cxx chooses to have this behavior, because ndn-cxx developers believe that an application is useless when the forwarder is down. This means, when NFD is closing, all ndn-cxx applications connected to NFD will crash as well. An external monitoring script can detect this, and restart NFD, followed by applications. This behavior is acceptable on servers for those applications that don't have much internal states. However, it's undesirable to let a user-facing application crash when NFD is closing. The application can catch the error raised by the library, and periodically attempt to connect the Face until a new connection is established, and then register the prefixes. Alternatively, the library can choose to detect the problem, and attempt to recover. If a library chooses to do this, it should still inform the application through the event or callback system when forwarder becomes unavailable and available again. In both cases, the application may need to recover network-visible states after forwarder recovery (eg. ChronoSync entity may need to rejoin the sync-group). Yours, Junxiao On Tue, Sep 2, 2014 at 5:08 PM, Ashlesh Gawande (agawande) < agawande at memphis.edu> wrote: > Hi > > I was working with PyNDN2 library which uses NFD. > > In PyNDN2 Is there a way to check if NFD has gone down (or should there be > a way to that in the library) ? > If so how can it be done? > > (I have added the conversation I had with Jeff T) > > Thanks > Ashlesh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Wed Sep 3 05:16:49 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Wed, 3 Sep 2014 05:16:49 -0700 Subject: [Nfd-dev] ndn-cxx Application Cache In-Reply-To: References: Message-ID: Hi Jiewen I cannot approve the application cache commit http://gerrit.named-data.net/1213 , because I cannot understand it. As I pointed out, the code is mixing data structure operations with logic. One goal of NFD and ndn-cxx is simplicity, and this code violates this goal. Please implement a simpler version. One design is to use boost::multi_index. Specifically, take the multi_index defined in ContentStore, and add a new index that orders by data->getFullName(). All lookups can then take place on this index. Although multi_index may be slower than skip list, this design is sufficient for usage in client library, because it is expected to contain less packets than a ContentStore. The benefit besides code simplicity is that it's expected to consume less memory. Yours, Junxiao -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Sun Sep 7 21:41:57 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Sun, 7 Sep 2014 21:41:57 -0700 Subject: [Nfd-dev] Close NFD backdoor Message-ID: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Hi all, Beichuan and I had a discussion about a backdoor of nfd which allows apps to dump unsolicited data packets into content store of local nfd and how to close the backdoor. Here is a summary of the discussion, we would like to hear your comments. If people agree with it, we will start the implementation soon. ================ 1. NFD backdoor Current NFD implementation allows local applications to dump unsolicited data packets into content store. This backdoor enables malware to cache poison local content store. To close the backdoor, nfd should 1) reject unsolicited data packets and 2) authenticate prefix registration on local nfd. 2. In-app storage Some existing applications (such as ChronoChat) are using the backdoor to abuse local content store as an in-memory storage of data packets, because current library does not provide in-app storage. The differences between content store and in-app storage include: a) content store is a cache, it is not guaranteed that a data packet will always be there. In contrast, the existence of a data packet in an in-app storage is completely under the app's control. b) content store is aware of FreshnessPeriod of data packets, but an in-app storage should not. The working flow of in-app storage is: an app creates its own in-app storage. If the app generates some unsolicited data packets and insert them in the in-app storage. When receiving an interest, the app will first try to see if there is a matched data packet in the in-app storage and then process the interest if there is no matched data. With the in-app storage, apps do not need to abuse content store, and we can eliminate the source of unsolicited data in the localhost environment. For now, we need to provide two types of in-app storage: a) explicit-controlled storage: data packets need to be explicitly deleted; when storage is full, all data insertions must fail until app explicitly increases the capacity of the storage. b) FIFO: capacity is fixed, the first inserted data will be automatically deleted when the storage is full. The implementation of the in-app storage could be more flexible to support other strategies, but the two above seems to be sufficient for now. 3. Authentication of prefix registration on local nfd Currently all apps are using the system-default key to sign prefix registration command. There are several drawbacks of this mechanism: a) every app get the access to the system-default private key. b) the system-default key is trusted to register any prefix. A better model should be: users should have per-prefix registration keys (the privilege of the key is expressed in the key name), and authorize applications to get the access of key for particular prefixes. This resembles the network control model of current os, e.g., when a user starts a new app which will listen to some ports, the os will ask user to give the permission to the app. More specifically, when an app wants to register a prefix "/a/b", it will first look for a key "/localhost/prefix-register-key/a/b/[keyId]" in the user's keystore. If there is no such key, the app will create one and ask for user's permission. User authorizes that by signing the prefix-register-key using user's own key (whose privilege is pre-configured at nfd side). After that the app will use key "/localhost/prefix-register-key/a/b/[keyId]" to sign request for prefix "/a/b". At the nfd side, nfd will check: a) whether the requested prefix matches the signing key name. b) whether the key "/localhost/prefix-register-key/a/b/[keyId]" is signed by a trusted user. c) whether the user is allowed to register the prefix. The request will be accepted only if all the three conditions are satisfied. If an app finds that the prefix-register-key it needs has already existed in the user's keystore (e.g., created by some other apps), it will ask the user for the access to the existing key. This access control feature is already supported by the osx-keychain based keystore. ================ Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jefft0 at remap.ucla.edu Mon Sep 8 02:53:03 2014 From: jefft0 at remap.ucla.edu (Thompson, Jeff) Date: Mon, 8 Sep 2014 09:53:03 +0000 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: The NDN-RTC video conferencing app uses the MemoryContentCache in-app storage. It keeps data in the cache based on the data packet's FreshnessPeriod. http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html You say "content store is aware of FreshnessPeriod of data packets, but an in-app storage should not." Why should the in-app storage not be aware of the FreshnessPeriod? - Jeff T From: Yingdi Yu > Date: Sunday, September 7, 2014 9:41 PM To: nfd-dev >, NDN App > Subject: [Nfd-dev] Close NFD backdoor Hi all, Beichuan and I had a discussion about a backdoor of nfd which allows apps to dump unsolicited data packets into content store of local nfd and how to close the backdoor. Here is a summary of the discussion, we would like to hear your comments. If people agree with it, we will start the implementation soon. ================ 1. NFD backdoor Current NFD implementation allows local applications to dump unsolicited data packets into content store. This backdoor enables malware to cache poison local content store. To close the backdoor, nfd should 1) reject unsolicited data packets and 2) authenticate prefix registration on local nfd. 2. In-app storage Some existing applications (such as ChronoChat) are using the backdoor to abuse local content store as an in-memory storage of data packets, because current library does not provide in-app storage. The differences between content store and in-app storage include: a) content store is a cache, it is not guaranteed that a data packet will always be there. In contrast, the existence of a data packet in an in-app storage is completely under the app's control. b) content store is aware of FreshnessPeriod of data packets, but an in-app storage should not. The working flow of in-app storage is: an app creates its own in-app storage. If the app generates some unsolicited data packets and insert them in the in-app storage. When receiving an interest, the app will first try to see if there is a matched data packet in the in-app storage and then process the interest if there is no matched data. With the in-app storage, apps do not need to abuse content store, and we can eliminate the source of unsolicited data in the localhost environment. For now, we need to provide two types of in-app storage: a) explicit-controlled storage: data packets need to be explicitly deleted; when storage is full, all data insertions must fail until app explicitly increases the capacity of the storage. b) FIFO: capacity is fixed, the first inserted data will be automatically deleted when the storage is full. The implementation of the in-app storage could be more flexible to support other strategies, but the two above seems to be sufficient for now. 3. Authentication of prefix registration on local nfd Currently all apps are using the system-default key to sign prefix registration command. There are several drawbacks of this mechanism: a) every app get the access to the system-default private key. b) the system-default key is trusted to register any prefix. A better model should be: users should have per-prefix registration keys (the privilege of the key is expressed in the key name), and authorize applications to get the access of key for particular prefixes. This resembles the network control model of current os, e.g., when a user starts a new app which will listen to some ports, the os will ask user to give the permission to the app. More specifically, when an app wants to register a prefix "/a/b", it will first look for a key "/localhost/prefix-register-key/a/b/[keyId]" in the user's keystore. If there is no such key, the app will create one and ask for user's permission. User authorizes that by signing the prefix-register-key using user's own key (whose privilege is pre-configured at nfd side). After that the app will use key "/localhost/prefix-register-key/a/b/[keyId]" to sign request for prefix "/a/b". At the nfd side, nfd will check: a) whether the requested prefix matches the signing key name. b) whether the key "/localhost/prefix-register-key/a/b/[keyId]" is signed by a trusted user. c) whether the user is allowed to register the prefix. The request will be accepted only if all the three conditions are satisfied. If an app finds that the prefix-register-key it needs has already existed in the user's keystore (e.g., created by some other apps), it will ask the user for the access to the existing key. This access control feature is already supported by the osx-keychain based keystore. ================ Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Mon Sep 8 09:04:25 2014 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Mon, 8 Sep 2014 16:04:25 +0000 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: Hi Yingdi, Some comments below. Thanks, Jeff From: Yingdi Yu > Date: Sun, 7 Sep 2014 21:41:57 -0700 To: ">" >, "ndn-app at lists.cs.ucla.edu" > Subject: [Nfd-dev] Close NFD backdoor Hi all, Beichuan and I had a discussion about a backdoor of nfd which allows apps to dump unsolicited data packets into content store of local nfd and how to close the backdoor. Here is a summary of the discussion, we would like to hear your comments. If people agree with it, we will start the implementation soon. While it makes sense to address this in the implementation soon, I would really encourage that this be considered as part of an evolving but comprehensive design and description about "how NDN applications interact with the (local) forwarder" and "how end-users interact with NDN security." A little more on this below. ================ 1. NFD backdoor Current NFD implementation allows local applications to dump unsolicited data packets into content store. This backdoor enables malware to cache poison local content store. To close the backdoor, nfd should 1) reject unsolicited data packets and 2) authenticate prefix registration on local nfd. Solicitation = having an Interest in the PIT, right? 2. In-app storage Some existing applications (such as ChronoChat) are using the backdoor to abuse local content store as an in-memory storage of data packets, because current library does not provide in-app storage. This is only the case in ndn-cxx, currently. NDN-CCL provides the MemoryContentCache, per JeffT's e-mail, as a result of this pattern showing up repeatedly in our applications. The differences between content store and in-app storage include: a) content store is a cache, it is not guaranteed that a data packet will always be there. In contrast, the existence of a data packet in an in-app storage is completely under the app's control. Ilya has previously proposed the notion of an API that would enable applications to request local storage from the content store. This still seems like an interesting notion to consider long-term. As AlexA has pointed out, in-application stores are likely redundant uses of memory, to some extent. Is there a reason that we are not considering a content store API and providing it in the library instead? b) content store is aware of FreshnessPeriod of data packets, but an in-app storage should not. Not sure what this means. FreshnessPeriod can have application-defined semantics, depending on how the app uses MustBeFresh... ? The working flow of in-app storage is: an app creates its own in-app storage. If the app generates some unsolicited data packets and insert them in the in-app storage. When receiving an interest, the app will first try to see if there is a matched data packet in the in-app storage and then process the interest if there is no matched data. With the in-app storage, apps do not need to abuse content store, and we can eliminate the source of unsolicited data in the localhost environment. For now, we need to provide two types of in-app storage: a) explicit-controlled storage: data packets need to be explicitly deleted; when storage is full, all data insertions must fail until app explicitly increases the capacity of the storage. b) FIFO: capacity is fixed, the first inserted data will be automatically deleted when the storage is full. The implementation of the in-app storage could be more flexible to support other strategies, but the two above seems to be sufficient for now. I generally agree but what use cases were considered to come up with these two approaches? Another possibility could be object-size based, # of request-based, etc. I also assume these would be configure per-namespace rather than per-application? (e.g., an application might handle /foo/archive differently than /foo/latest) 3. Authentication of prefix registration on local nfd Currently all apps are using the system-default key to sign prefix registration command. There are several drawbacks of this mechanism: a) every app get the access to the system-default private key. b) the system-default key is trusted to register any prefix. A better model should be: users should have per-prefix registration keys (the privilege of the key is expressed in the key name), and authorize applications to get the access of key for particular prefixes. This resembles the network control model of current os, e.g., when a user starts a new app which will listen to some ports, the os will ask user to give the permission to the app. This is where I think we should consider this within a more comprehensive model about how end-users interact with NDN security. (e.g., what are all the things that an application has to configure/ask/do when first installed? how much real transparency can be given to its use? and so on) The current way that most applications and operating systems interact with users about security and privacy should not necessarily be our model. :) Perhaps this is a topic for some background research - I haven't looked at the literature for a bit, but consider papers like: Dourish, Paul, et al. "Security in the wild: user strategies for managing security as an everyday, practical problem." Personal and Ubiquitous Computing 8.6 (2004): 391-401. [Dourish is at UCI] Smetters, Diana K., and Rebecca E. Grinter. "Moving from the design of usable security technologies to the design of useful secure applications." Proceedings of the 2002 workshop on New security paradigms. ACM, 2002. [Diana worked with Van on CCNx] I'm on a slow connection so can't do a search, but would guess there are also some papers on interaction design / user experience for OS security settings. (Once we get a student aboard on Katie's next ViD grant, perhaps they could contribute to this design effort.) More specifically, when an app wants to register a prefix "/a/b", it will first look for a key "/localhost/prefix-register-key/a/b/[keyId]" in the user's keystore. If there is no such key, the app will create one and ask for user's permission. User authorizes that by signing the prefix-register-key using user's own key (whose privilege is pre-configured at nfd side). After that the app will use key "/localhost/prefix-register-key/a/b/[keyId]" to sign request for prefix "/a/b". >From the system level, I understand this... but again, would encourage a design that incorporates this specific authorization into a more comprehensive picture about how installing a new NDN app would look to the end-user, and the level of knowledge expected from them to answer the questions presented. One way to do this would be to storyboard an end-user's entire interaction with NDN security infrastructure, starting from a freshly installed OS and a new identity and publishing prefix assignment from a service provider, and ending with application installation questions such as this. At the nfd side, nfd will check: a) whether the requested prefix matches the signing key name. b) whether the key "/localhost/prefix-register-key/a/b/[keyId]" is signed by a trusted user. c) whether the user is allowed to register the prefix. The request will be accepted only if all the three conditions are satisfied. If an app finds that the prefix-register-key it needs has already existed in the user's keystore (e.g., created by some other apps), it will ask the user for the access to the existing key. This access control feature is already supported by the osx-keychain based keystore. ================ Yingdi _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From jefft0 at remap.ucla.edu Mon Sep 8 09:13:40 2014 From: jefft0 at remap.ucla.edu (Thompson, Jeff) Date: Mon, 8 Sep 2014 16:13:40 +0000 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: > currently. NDN-CCL provides the MemoryContentCache, per JeffT's e-mail, as a result of this pattern showing up repeatedly in our applications. An indeed, the CCL implementation of ChronoSync2013 uses a MemoryContentCache and doesn't use the back door. https://github.com/named-data/ndn-cpp/blob/7f95422eeb278f83c901d556e1a06df65c0510dd/include/ndn-cpp/sync/chrono-sync2013.hpp#L351 From: , Jeff Burke > Date: Monday, September 8, 2014 9:04 AM To: Yingdi Yu >, nfd-dev >, NDN App > Subject: Re: [Nfd-dev] Close NFD backdoor Hi Yingdi, Some comments below. Thanks, Jeff From: Yingdi Yu > Date: Sun, 7 Sep 2014 21:41:57 -0700 To: ">" >, "ndn-app at lists.cs.ucla.edu" > Subject: [Nfd-dev] Close NFD backdoor Hi all, Beichuan and I had a discussion about a backdoor of nfd which allows apps to dump unsolicited data packets into content store of local nfd and how to close the backdoor. Here is a summary of the discussion, we would like to hear your comments. If people agree with it, we will start the implementation soon. While it makes sense to address this in the implementation soon, I would really encourage that this be considered as part of an evolving but comprehensive design and description about "how NDN applications interact with the (local) forwarder" and "how end-users interact with NDN security." A little more on this below. ================ 1. NFD backdoor Current NFD implementation allows local applications to dump unsolicited data packets into content store. This backdoor enables malware to cache poison local content store. To close the backdoor, nfd should 1) reject unsolicited data packets and 2) authenticate prefix registration on local nfd. Solicitation = having an Interest in the PIT, right? 2. In-app storage Some existing applications (such as ChronoChat) are using the backdoor to abuse local content store as an in-memory storage of data packets, because current library does not provide in-app storage. This is only the case in ndn-cxx, currently. NDN-CCL provides the MemoryContentCache, per JeffT's e-mail, as a result of this pattern showing up repeatedly in our applications. The differences between content store and in-app storage include: a) content store is a cache, it is not guaranteed that a data packet will always be there. In contrast, the existence of a data packet in an in-app storage is completely under the app's control. Ilya has previously proposed the notion of an API that would enable applications to request local storage from the content store. This still seems like an interesting notion to consider long-term. As AlexA has pointed out, in-application stores are likely redundant uses of memory, to some extent. Is there a reason that we are not considering a content store API and providing it in the library instead? b) content store is aware of FreshnessPeriod of data packets, but an in-app storage should not. Not sure what this means. FreshnessPeriod can have application-defined semantics, depending on how the app uses MustBeFresh... ? The working flow of in-app storage is: an app creates its own in-app storage. If the app generates some unsolicited data packets and insert them in the in-app storage. When receiving an interest, the app will first try to see if there is a matched data packet in the in-app storage and then process the interest if there is no matched data. With the in-app storage, apps do not need to abuse content store, and we can eliminate the source of unsolicited data in the localhost environment. For now, we need to provide two types of in-app storage: a) explicit-controlled storage: data packets need to be explicitly deleted; when storage is full, all data insertions must fail until app explicitly increases the capacity of the storage. b) FIFO: capacity is fixed, the first inserted data will be automatically deleted when the storage is full. The implementation of the in-app storage could be more flexible to support other strategies, but the two above seems to be sufficient for now. I generally agree but what use cases were considered to come up with these two approaches? Another possibility could be object-size based, # of request-based, etc. I also assume these would be configure per-namespace rather than per-application? (e.g., an application might handle /foo/archive differently than /foo/latest) 3. Authentication of prefix registration on local nfd Currently all apps are using the system-default key to sign prefix registration command. There are several drawbacks of this mechanism: a) every app get the access to the system-default private key. b) the system-default key is trusted to register any prefix. A better model should be: users should have per-prefix registration keys (the privilege of the key is expressed in the key name), and authorize applications to get the access of key for particular prefixes. This resembles the network control model of current os, e.g., when a user starts a new app which will listen to some ports, the os will ask user to give the permission to the app. This is where I think we should consider this within a more comprehensive model about how end-users interact with NDN security. (e.g., what are all the things that an application has to configure/ask/do when first installed? how much real transparency can be given to its use? and so on) The current way that most applications and operating systems interact with users about security and privacy should not necessarily be our model. :) Perhaps this is a topic for some background research - I haven't looked at the literature for a bit, but consider papers like: Dourish, Paul, et al. "Security in the wild: user strategies for managing security as an everyday, practical problem." Personal and Ubiquitous Computing 8.6 (2004): 391-401. [Dourish is at UCI] Smetters, Diana K., and Rebecca E. Grinter. "Moving from the design of usable security technologies to the design of useful secure applications." Proceedings of the 2002 workshop on New security paradigms. ACM, 2002. [Diana worked with Van on CCNx] I'm on a slow connection so can't do a search, but would guess there are also some papers on interaction design / user experience for OS security settings. (Once we get a student aboard on Katie's next ViD grant, perhaps they could contribute to this design effort.) More specifically, when an app wants to register a prefix "/a/b", it will first look for a key "/localhost/prefix-register-key/a/b/[keyId]" in the user's keystore. If there is no such key, the app will create one and ask for user's permission. User authorizes that by signing the prefix-register-key using user's own key (whose privilege is pre-configured at nfd side). After that the app will use key "/localhost/prefix-register-key/a/b/[keyId]" to sign request for prefix "/a/b". >From the system level, I understand this... but again, would encourage a design that incorporates this specific authorization into a more comprehensive picture about how installing a new NDN app would look to the end-user, and the level of knowledge expected from them to answer the questions presented. One way to do this would be to storyboard an end-user's entire interaction with NDN security infrastructure, starting from a freshly installed OS and a new identity and publishing prefix assignment from a service provider, and ending with application installation questions such as this. At the nfd side, nfd will check: a) whether the requested prefix matches the signing key name. b) whether the key "/localhost/prefix-register-key/a/b/[keyId]" is signed by a trusted user. c) whether the user is allowed to register the prefix. The request will be accepted only if all the three conditions are satisfied. If an app finds that the prefix-register-key it needs has already existed in the user's keystore (e.g., created by some other apps), it will ask the user for the access to the existing key. This access control feature is already supported by the osx-keychain based keystore. ================ Yingdi _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From davide.pesavento at lip6.fr Mon Sep 8 09:29:33 2014 From: davide.pesavento at lip6.fr (Davide Pesavento) Date: Mon, 8 Sep 2014 18:29:33 +0200 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: Hi Yingdi, On Mon, Sep 8, 2014 at 6:41 AM, Yingdi Yu wrote: > Hi all, > > Beichuan and I had a discussion about a backdoor of nfd which allows apps to > dump unsolicited data packets into content store of local nfd and how to > close the backdoor. Here is a summary of the discussion, we would like to > hear your comments. If people agree with it, we will start the > implementation soon. > > ================ > 1. NFD backdoor > Current NFD implementation allows local applications to dump unsolicited > data packets into content store. This backdoor enables malware to cache > poison local content store. To close the backdoor, nfd should 1) reject > unsolicited data packets and 2) authenticate prefix registration on local > nfd. > Can you explain how this "backdoor" works exactly and why it's exploitable only by local applications? Thanks, Davide From yingdi at CS.UCLA.EDU Mon Sep 8 11:29:56 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 8 Sep 2014 11:29:56 -0700 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: Hi JeffT, On Sep 8, 2014, at 2:53 AM, Thompson, Jeff wrote: > The NDN-RTC video conferencing app uses the MemoryContentCache in-app storage. It keeps data in the cache based on the data packet's FreshnessPeriod. > http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html > > You say "content store is aware of FreshnessPeriod of data packets, but an in-app storage should not." Why should the in-app storage not be aware of the FreshnessPeriod? In the NDN-TLV spec, the definition of FreshnessPeriod says "Note that the stale data is still valid data; the expiration of FreshnessPeriod only means that the producer may have produced newer data." My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store. The end producer is the only one who knows and can decide whether a data packet is valid or not. And the in-memory storage contains data packets that are treated as valid by end producer. For example, producer A generates a data packet "/a/b/c" with 10-second FreshnessPeriod. If "/a/b/c" is put into the MemoryContentCache, the packet will be removed from the cache after 10 seconds, and the producer A will have to generate the same data packet again even if the producer A still takes "/a/b/c" as valid. With the in-memory storage, the packet will never be removed until the producer generates a new valid data packet to replace "/a/b/c". And "/a/b/c" will be obsolete by the new data packet. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From oran at cisco.com Mon Sep 8 12:58:33 2014 From: oran at cisco.com (Dave Oran (oran)) Date: Mon, 8 Sep 2014 19:58:33 +0000 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com> I don?t mean to hijack this thread, but it seems that the architecture needs two cache timers, not one. One is an Expiry timer after which the data is no longer considered valid (and hence should not be returned from the cache if an interest arrives), the other is a desired caching period timer, which can be used by the cache as one of the inputs to its replacement algorithm. On Sep 8, 2014, at 2:29 PM, Yingdi Yu wrote: > Hi JeffT, > > On Sep 8, 2014, at 2:53 AM, Thompson, Jeff wrote: > >> The NDN-RTC video conferencing app uses the MemoryContentCache in-app storage. It keeps data in the cache based on the data packet's FreshnessPeriod. >> http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html >> >> You say "content store is aware of FreshnessPeriod of data packets, but an in-app storage should not." Why should the in-app storage not be aware of the FreshnessPeriod? > > In the NDN-TLV spec, the definition of FreshnessPeriod says "Note that the stale data is still valid data; the expiration of FreshnessPeriod only means that the producer may have produced newer data." My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store. The end producer is the only one who knows and can decide whether a data packet is valid or not. And the in-memory storage contains data packets that are treated as valid by end producer. > > For example, producer A generates a data packet "/a/b/c" with 10-second FreshnessPeriod. If "/a/b/c" is put into the MemoryContentCache, the packet will be removed from the cache after 10 seconds, and the producer A will have to generate the same data packet again even if the producer A still takes "/a/b/c" as valid. With the in-memory storage, the packet will never be removed until the producer generates a new valid data packet to replace "/a/b/c". And "/a/b/c" will be obsolete by the new data packet. > > Yingdi > > _______________________________________________ > Ndn-app mailing list > Ndn-app at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-app -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From yingdi at CS.UCLA.EDU Mon Sep 8 15:02:26 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 8 Sep 2014 15:02:26 -0700 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: References: Message-ID: <3C0DB2DF-F315-43FF-A518-E7ED3A1EAA7F@cs.ucla.edu> Hi Jeff, On Sep 8, 2014, at 9:04 AM, Burke, Jeff wrote: >> Beichuan and I had a discussion about a backdoor of nfd which allows apps to dump unsolicited data packets into content store of local nfd and how to close the backdoor. Here is a summary of the discussion, we would like to hear your comments. If people agree with it, we will start the implementation soon. > > > While it makes sense to address this in the implementation soon, I would really encourage that this be considered as part of an evolving but comprehensive design and description about "how NDN applications interact with the (local) forwarder" and "how end-users interact with NDN security." A little more on this below. Yes. I think through this process, we can nail down a standard mechanism for NDN apps to talk to local fowarder. >> ================ >> 1. NFD backdoor >> Current NFD implementation allows local applications to dump unsolicited data packets into content store. This backdoor enables malware to cache poison local content store. To close the backdoor, nfd should 1) reject unsolicited data packets and 2) authenticate prefix registration on local nfd. > > > Solicitation = having an Interest in the PIT, right? Not only having an interest in the PIT, but also that the incoming data packet must be received from one of the faces to which the corresponding interest has been forwarded. >> 2. In-app storage >> Some existing applications (such as ChronoChat) are using the backdoor to abuse local content store as an in-memory storage of data packets, because current library does not provide in-app storage. > > > This is only the case in ndn-cxx, currently. NDN-CCL provides the MemoryContentCache, per JeffT's e-mail, as a result of this pattern showing up repeatedly in our applications. It is good to hear that NDN-CCL has already supported this feature. ndn-cxx needs to catch up! :) But what we want to do in ndn-cxx is a little bit more than MemoryContentCache. MemoryContentCache relies on the FreshnessPeriod, but in some use cases, apps may want to retain a data packet longer than that. I am making a wiki page for this in-memory storage, trying to collect a set of use cases of that. I will send the link to the wiki page later. >> The differences between content store and in-app storage include: >> a) content store is a cache, it is not guaranteed that a data packet will always be there. In contrast, the existence of a data packet in an in-app storage is completely under the app's control. > > > Ilya has previously proposed the notion of an API that would enable applications to request local storage from the content store. This still seems like an interesting notion to consider long-term. As AlexA has pointed out, in-application stores are likely redundant uses of memory, to some extent. Is there a reason that we are not considering a content store API and providing it in the library instead? The differences is that the content store does not provide guaranteed storage. And content store really has no clue about whether the data is valid or obsoleted. So if the local forwarder gets a interest with MustBeFresh=true while the data in content store is marked as stale, the local forwarder will forward the interest to the app. But a stale data packet does not necessarily mean that the data packet is obsoleted from the perspective of the producer. Without having a copy of the data packet in the app, the app will have to generate the data packet (and sign it) again. I think such on-demand data generation is unnecessary. >> b) content store is aware of FreshnessPeriod of data packets, but an in-app storage should not. > > > Not sure what this means. FreshnessPeriod can have application-defined semantics, depending on how the app uses MustBeFresh... ? I think FreshnessPeriod is meaningful to intermediate routers and end-consumers. For data producer, FreshnessPeriod is only a suggestion that the producer gives to routers and consumers. >> The working flow of in-app storage is: an app creates its own in-app storage. If the app generates some unsolicited data packets and insert them in the in-app storage. >> When receiving an interest, the app will first try to see if there is a matched data packet in the in-app storage and then process the interest if there is no matched data. With the in-app storage, apps do not need to abuse content store, and we can eliminate the source of unsolicited data in the localhost environment. >> >> For now, we need to provide two types of in-app storage: >> a) explicit-controlled storage: data packets need to be explicitly deleted; when storage is full, all data insertions must fail until app explicitly increases the capacity of the storage. >> b) FIFO: capacity is fixed, the first inserted data will be automatically deleted when the storage is full. >> The implementation of the in-app storage could be more flexible to support other strategies, but the two above seems to be sufficient for now. > > > I generally agree but what use cases were considered to come up with these two approaches? I will send a link to a wiki page which describe the use cases of it. > Another possibility could be object-size based, # of request-based, etc. Just curious, is there any use case for these policy? > I also assume these would be configure per-namespace rather than per-application? (e.g., an application might handle /foo/archive differently than /foo/latest) I am a little bit confused here. This in-memory storage is not an independent service or program, it is a part of the application. And an app does not have to keep only one in-memory storage. >> 3. Authentication of prefix registration on local nfd >> Currently all apps are using the system-default key to sign prefix registration command. There are several drawbacks of this mechanism: >> a) every app get the access to the system-default private key. >> b) the system-default key is trusted to register any prefix. >> >> A better model should be: users should have per-prefix registration keys (the privilege of the key is expressed in the key name), and authorize applications to get the access of key for particular prefixes. >> This resembles the network control model of current os, e.g., when a user starts a new app which will listen to some ports, the os will ask user to give the permission to the app. > > > This is where I think we should consider this within a more comprehensive model about how end-users interact with NDN security. (e.g., what are all the things that an application has to configure/ask/do when first installed? how much real transparency can be given to its use? and so on) > > The current way that most applications and operating systems interact with users about security and privacy should not necessarily be our model. :) Perhaps this is a topic for some background research - I haven't looked at the literature for a bit, but consider papers like: > >> Dourish, Paul, et al. "Security in the wild: user strategies for managing security as an everyday, practical problem." Personal and Ubiquitous Computing 8.6 (2004): 391-401. [Dourish is at UCI] >> >> Smetters, Diana K., and Rebecca E. Grinter. "Moving from the design of usable security technologies to the design of useful secure applications." Proceedings of the 2002 workshop on New security paradigms. ACM, 2002. [Diana worked with Van on CCNx] > > > I'm on a slow connection so can't do a search, but would guess there are also some papers on interaction design / user experience for OS security settings. (Once we get a student aboard on Katie's next ViD grant, perhaps they could contribute to this design effort.) > >> >> More specifically, when an app wants to register a prefix "/a/b", it will first look for a key "/localhost/prefix-register-key/a/b/[keyId]" in the user's keystore. If there is no such key, the app will create one and ask for user's permission. >> User authorizes that by signing the prefix-register-key using user's own key (whose privilege is pre-configured at nfd side). After that the app will use key "/localhost/prefix-register-key/a/b/[keyId]" to sign request for prefix "/a/b". > > > From the system level, I understand this... but again, would encourage a design that incorporates this specific authorization into a more comprehensive picture about how installing a new NDN app would look to the end-user, and the level of knowledge expected from them to answer the questions presented. > > One way to do this would be to storyboard an end-user's entire interaction with NDN security infrastructure, starting from a freshly installed OS and a new identity and publishing prefix assignment from a service provider, and ending with application installation questions such as this. yes, the purpose is to make prefix registration as simple and straightforward as possible. This specific authorization mechanism is at least not worse than the current OS security model, but we definitely need to think about more appropriate solution. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From yingdi at CS.UCLA.EDU Mon Sep 8 15:13:50 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 8 Sep 2014 15:13:50 -0700 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: Hi Davide On Sep 8, 2014, at 9:29 AM, Davide Pesavento wrote: > Can you explain how this "backdoor" works exactly and why it's > exploitable only by local applications? Ideally, nfd should only accept a data packet from an interface to which a matched pending interest has been forwarded, but current implementation accepts data packets from any local interface (I am not sure if remote interface is the same, Alex and Junxiao can correct me). Then a local application can basically dump any data packet into the content store of the local forwarder. Although the data consumer can detect the false data packet through signature validation, the overhead of fetching the correct data unnecessarily increases. In the worse cases, a malware can DoS many applications running on the same host. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From yingdi at CS.UCLA.EDU Mon Sep 8 15:21:46 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 8 Sep 2014 15:21:46 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com> References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com> Message-ID: On Sep 8, 2014, at 12:58 PM, Dave Oran (oran) wrote: > I don?t mean to hijack this thread, but it seems that the architecture needs two cache timers, not one. > > One is an Expiry timer after which the data is no longer considered valid (and hence should not be returned from the cache if an interest arrives) I am not sure if it is always possible for a data producer to predict the time when the data will become invalid. Take DNS as an example, the authoritative server may never know when one of the records will be changed. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jefft0 at remap.ucla.edu Mon Sep 8 16:27:04 2014 From: jefft0 at remap.ucla.edu (Thompson, Jeff) Date: Mon, 8 Sep 2014 23:27:04 +0000 Subject: [Nfd-dev] Help needed with PyNDN2 In-Reply-To: References: Message-ID: Hi Lan, The PyNDN2 default constructor for Face makes a Unix socket connection to the local NFD. Therefore, if NFD crashes, the next attempt by the consumer application to send an interest to NFD gets "socket.error: [Errno 32] Broken pipe". This might be easy to detect. However, the producer application makes a connection and waits for NFD to send an interest to the application. If NFD crashes, it simply never gets an interest or an error. - Jeff T From: "Lan Wang (lanwang)" > Date: Wednesday, September 3, 2014 2:26 PM To: Junxiao Shi > Cc: "Ashlesh Gawande (agawande)" >, Jeff Thompson > Subject: Re: [Nfd-dev] Help needed with PyNDN2 Junxiao: Thank you for the explanation. Jeff: what's the behavior of PyNDN2 when nfd crashes? Lan On Sep 2, 2014, at 7:25 PM, Junxiao Shi > wrote: Hi Ashlesh I don't known PyNDN so I cannot answer the specific question. The answer below is applicable to library design in general. A client library can choose what to do when the forwarder has crashed. In most cases, the library would crash or raise an error if the connection to the server is lost. This is common among popular client libraries such as libmysql. ndn-cxx chooses to have this behavior, because ndn-cxx developers believe that an application is useless when the forwarder is down. This means, when NFD is closing, all ndn-cxx applications connected to NFD will crash as well. An external monitoring script can detect this, and restart NFD, followed by applications. This behavior is acceptable on servers for those applications that don't have much internal states. However, it's undesirable to let a user-facing application crash when NFD is closing. The application can catch the error raised by the library, and periodically attempt to connect the Face until a new connection is established, and then register the prefixes. Alternatively, the library can choose to detect the problem, and attempt to recover. If a library chooses to do this, it should still inform the application through the event or callback system when forwarder becomes unavailable and available again. In both cases, the application may need to recover network-visible states after forwarder recovery (eg. ChronoSync entity may need to rejoin the sync-group). Yours, Junxiao On Tue, Sep 2, 2014 at 5:08 PM, Ashlesh Gawande (agawande) > wrote: Hi I was working with PyNDN2 library which uses NFD. In PyNDN2 Is there a way to check if NFD has gone down (or should there be a way to that in the library) ? If so how can it be done? (I have added the conversation I had with Jeff T) Thanks Ashlesh _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Mon Sep 8 17:36:41 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Mon, 8 Sep 2014 17:36:41 -0700 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Hi Lan Bug 1961 has been fixed. Code is merged to master branch. Please test in your internal testbed. The correct behavior shall be: 1. P1 is forwarded to UCLA, and could be forwarded to CSU and other nexthops as well 2. UCLA responds first and remains as best face; other responses are received but ignored 3. P2 should be forwarded to UCLA This bug is not critical to end users. I don't plan to backport to v0.2. It won't be deployed on testbed routers until v0.3 release. Yours, Junxiao On Tue, Sep 2, 2014 at 10:26 AM, Junxiao Shi wrote: > Hi Lan > > Thanks for the report. This shall be a bug of NCC strategy. > > In ccnd, > > 1. When first Data comes back that satisfies the PIT entry, the > incoming face of that Data is remembered. > 2. The PIT entry is deleted as soon as it's satisfied. > 3. When second Data comes back, it's discarded as unsolicited. > > In NFD, > > 1. When first Data comes back that satisfies the PIT entry, the > incoming face of that Data is remembered. > 2. The PIT entry is kept, and will be deleted after straggler timer. > 3. When second Data comes back, the strategy is notified, and > remembers the incoming face. > > NFD's NCC strategy incorrectly assumes the incoming Data is from the > fastest upstream, causing the bug. > > I'll report the bug soon. > > Yours, Junxiao > > > On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) > wrote: > >> Junxiao, >> >> We observed something strange in one of our experiments when we used the >> ncc strategy (and link state routing). In some of the pings from Arizona >> to PKU, the ndnping delays were really large (>350ms) even though the >> shortest path delay is only around 200ms. We looked at the ndnping data, >> and found where the delay jumped from ~200ms to ~350ms. It turns out that >> instead of going through the shortest path (Arizona->UCLA->PKU), it is >> going through a much longer path (Arizona->CSU->?->NEU->PKU). >> >> To explain what happened, I'll call the ping before the change P1 and the >> ping after the change P2. Further looking into the nfd logs, it seems for >> P1 Arizona first sent the Interest to UCLA (which we expected), but while >> waiting for the data to come back, it also tried CSU and some other >> neighbors. The ping Data did come back from the shortest path >> (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes >> back from the other much longer paths. The problem is that Arizona seems >> to remember the most recent successful paths instead of the previously >> successful shorter paths. Next time when P2 came, it directly sent the >> Interest to CSU instead of UCLA. >> >> Is this expected behavior of ncc? I want to emphasize that the shorter >> path did return the same Ping data (and much earlier than the longer path). >> >> Lan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oran at cisco.com Tue Sep 9 04:17:39 2014 From: oran at cisco.com (Dave Oran (oran)) Date: Tue, 9 Sep 2014 11:17:39 +0000 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com>, Message-ID: ___________________________ iDevice - please excuse typos. On Sep 8, 2014, at 6:21 PM, Yingdi Yu > wrote: On Sep 8, 2014, at 12:58 PM, Dave Oran (oran) > wrote: I don?t mean to hijack this thread, but it seems that the architecture needs two cache timers, not one. One is an Expiry timer after which the data is no longer considered valid (and hence should not be returned from the cache if an interest arrives) I am not sure if it is always possible for a data producer to predict the time when the data will become invalid. If not the producer, then who? All data kept outside an application's private (and possibly permanent) store has a validity period, if not shorter then at least no longer than the lifetime of the key that signed it. Take DNS as an example, the authoritative server may never know when one of the records will be changed. Wrong analogy. A DNS Authoritative server is like a repo, a producer is in effect the application that updated the DNS. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Tue Sep 9 08:39:08 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Tue, 9 Sep 2014 08:39:08 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com>, Message-ID: On Sep 9, 2014, at 4:17 AM, Dave Oran (oran) wrote: > On Sep 8, 2014, at 6:21 PM, Yingdi Yu wrote: > >> >> On Sep 8, 2014, at 12:58 PM, Dave Oran (oran) wrote: >> >>> I don?t mean to hijack this thread, but it seems that the architecture needs two cache timers, not one. >>> >>> One is an Expiry timer after which the data is no longer considered valid (and hence should not be returned from the cache if an interest arrives) >> >> I am not sure if it is always possible for a data producer to predict the time when the data will become invalid. > If not the producer, then who? In many cases, no one knows. For example, you write an article, do you know when you will find some problem in it and change it? > All data kept outside an application's private (and possibly permanent) store has a validity period, if not shorter then at least no longer than the lifetime of the key that signed it. Yes, it is no longer than the lifetime of the key, but it could be shorter than that. As a producer, can you tell how shorter that could be? >> Take DNS as an example, the authoritative server may never know when one of the records will be changed. >> > Wrong analogy. A DNS Authoritative server is like a repo, a producer is in effect the application that updated the DNS. Ok, then for the "producer" in your mind, can he predict when his DNS record will be changed? I never know when my own domain's A record will be changed, because my ISP occasionally change my home IP address. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From lanwang at memphis.edu Tue Sep 9 10:20:46 2014 From: lanwang at memphis.edu (Lan Wang (lanwang)) Date: Tue, 9 Sep 2014 17:20:46 +0000 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: <821910A6-94F8-47D1-B485-686C7F62465E@memphis.edu> Junxiao, Thank you very much. Obaid will try the new code and verify the behavior. Lan On Sep 8, 2014, at 7:36 PM, Junxiao Shi > wrote: Hi Lan Bug 1961 has been fixed. Code is merged to master branch. Please test in your internal testbed. The correct behavior shall be: 1. P1 is forwarded to UCLA, and could be forwarded to CSU and other nexthops as well 2. UCLA responds first and remains as best face; other responses are received but ignored 3. P2 should be forwarded to UCLA This bug is not critical to end users. I don't plan to backport to v0.2. It won't be deployed on testbed routers until v0.3 release. Yours, Junxiao On Tue, Sep 2, 2014 at 10:26 AM, Junxiao Shi > wrote: Hi Lan Thanks for the report. This shall be a bug of NCC strategy. In ccnd, 1. When first Data comes back that satisfies the PIT entry, the incoming face of that Data is remembered. 2. The PIT entry is deleted as soon as it's satisfied. 3. When second Data comes back, it's discarded as unsolicited. In NFD, 1. When first Data comes back that satisfies the PIT entry, the incoming face of that Data is remembered. 2. The PIT entry is kept, and will be deleted after straggler timer. 3. When second Data comes back, the strategy is notified, and remembers the incoming face. NFD's NCC strategy incorrectly assumes the incoming Data is from the fastest upstream, causing the bug. I'll report the bug soon. Yours, Junxiao On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) > wrote: Junxiao, We observed something strange in one of our experiments when we used the ncc strategy (and link state routing). In some of the pings from Arizona to PKU, the ndnping delays were really large (>350ms) even though the shortest path delay is only around 200ms. We looked at the ndnping data, and found where the delay jumped from ~200ms to ~350ms. It turns out that instead of going through the shortest path (Arizona->UCLA->PKU), it is going through a much longer path (Arizona->CSU->?->NEU->PKU). To explain what happened, I'll call the ping before the change P1 and the ping after the change P2. Further looking into the nfd logs, it seems for P1 Arizona first sent the Interest to UCLA (which we expected), but while waiting for the data to come back, it also tried CSU and some other neighbors. The ping Data did come back from the shortest path (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes back from the other much longer paths. The problem is that Arizona seems to remember the most recent successful paths instead of the previously successful shorter paths. Next time when P2 came, it directly sent the Interest to CSU instead of UCLA. Is this expected behavior of ncc? I want to emphasize that the shorter path did return the same Ping data (and much earlier than the longer path). Lan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lanwang at memphis.edu Tue Sep 9 10:37:24 2014 From: lanwang at memphis.edu (Lan Wang (lanwang)) Date: Tue, 9 Sep 2014 17:37:24 +0000 Subject: [Nfd-dev] Help needed with PyNDN2 In-Reply-To: References: Message-ID: Jeff, Thank you very much. We are wondering if it is possible for the library to inform the producer when nfd crashes. Right now we'll have to rely on periodically getting the nfd-status to see if the prefix is still registered. Lan On Sep 8, 2014, at 6:27 PM, "Thompson, Jeff" > wrote: Hi Lan, The PyNDN2 default constructor for Face makes a Unix socket connection to the local NFD. Therefore, if NFD crashes, the next attempt by the consumer application to send an interest to NFD gets "socket.error: [Errno 32] Broken pipe". This might be easy to detect. However, the producer application makes a connection and waits for NFD to send an interest to the application. If NFD crashes, it simply never gets an interest or an error. - Jeff T From: "Lan Wang (lanwang)" > Date: Wednesday, September 3, 2014 2:26 PM To: Junxiao Shi > Cc: "Ashlesh Gawande (agawande)" >, Jeff Thompson > Subject: Re: [Nfd-dev] Help needed with PyNDN2 Junxiao: Thank you for the explanation. Jeff: what's the behavior of PyNDN2 when nfd crashes? Lan On Sep 2, 2014, at 7:25 PM, Junxiao Shi > wrote: Hi Ashlesh I don't known PyNDN so I cannot answer the specific question. The answer below is applicable to library design in general. A client library can choose what to do when the forwarder has crashed. In most cases, the library would crash or raise an error if the connection to the server is lost. This is common among popular client libraries such as libmysql. ndn-cxx chooses to have this behavior, because ndn-cxx developers believe that an application is useless when the forwarder is down. This means, when NFD is closing, all ndn-cxx applications connected to NFD will crash as well. An external monitoring script can detect this, and restart NFD, followed by applications. This behavior is acceptable on servers for those applications that don't have much internal states. However, it's undesirable to let a user-facing application crash when NFD is closing. The application can catch the error raised by the library, and periodically attempt to connect the Face until a new connection is established, and then register the prefixes. Alternatively, the library can choose to detect the problem, and attempt to recover. If a library chooses to do this, it should still inform the application through the event or callback system when forwarder becomes unavailable and available again. In both cases, the application may need to recover network-visible states after forwarder recovery (eg. ChronoSync entity may need to rejoin the sync-group). Yours, Junxiao On Tue, Sep 2, 2014 at 5:08 PM, Ashlesh Gawande (agawande) > wrote: Hi I was working with PyNDN2 library which uses NFD. In PyNDN2 Is there a way to check if NFD has gone down (or should there be a way to that in the library) ? If so how can it be done? (I have added the conversation I had with Jeff T) Thanks Ashlesh _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From oran at cisco.com Tue Sep 9 11:54:01 2014 From: oran at cisco.com (Dave Oran (oran)) Date: Tue, 9 Sep 2014 18:54:01 +0000 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com>, Message-ID: <7F186435-045E-4C15-8832-B24A8D567E48@cisco.com> On Sep 9, 2014, at 11:39 AM, Yingdi Yu wrote: On Sep 9, 2014, at 4:17 AM, Dave Oran (oran) wrote: On Sep 8, 2014, at 6:21 PM, Yingdi Yu wrote: On Sep 8, 2014, at 12:58 PM, Dave Oran (oran) wrote: I don?t mean to hijack this thread, but it seems that the architecture needs two cache timers, not one. One is an Expiry timer after which the data is no longer considered valid (and hence should not be returned from the cache if an interest arrives) I am not sure if it is always possible for a data producer to predict the time when the data will become invalid. If not the producer, then who? In many cases, no one knows. For example, you write an article, do you know when you will find some problem in it and change it? All data kept outside an application's private (and possibly permanent) store has a validity period, if not shorter then at least no longer than the lifetime of the key that signed it. Yes, it is no longer than the lifetime of the key, but it could be shorter than that. As a producer, can you tell how shorter that could be? Sometimes, no. Often, yes. I can give you lots of examples, if you want. Take DNS as an example, the authoritative server may never know when one of the records will be changed. Wrong analogy. A DNS Authoritative server is like a repo, a producer is in effect the application that updated the DNS. Ok, then for the "producer" in your mind, can he predict when his DNS record will be changed? I never know when my own domain's A record will be changed, because my ISP occasionally change my home IP address. Actually, yes, for example for IPv6 prefix delegations or DHCP leases. Yingdi -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From shijunxiao at email.arizona.edu Tue Sep 9 13:45:19 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 9 Sep 2014 13:45:19 -0700 Subject: [Nfd-dev] ndn-cxx devguide outline Message-ID: Dear folks Here is a draft outline for ndn-cxx Application Developer Guide. The target audience is developers who intend to develop NDN applications with ndn-cxx library. The document should briefly explain ndn-cxx architecture, and provide some guidance of designing producer and consumer applications. Please provide your comments on what should go into the document, and what should be emphasized. You may reply-all to this message, or post a note on ndn-cxx Task 1581 . Markdown source file can be found on ndn-cxx Task 1581 attachment. Yours, Junxiao ndn-cxx Application Developer Guide This document describes *ndn-cxx: NDN C++ library with eXperimental eXtensions*, and how to develop an NDN application using the ndn-cxx library. Introduction - history of ndn-cxx, comparison with ndn-ccl-api - what *should* be included in ndn-cxx - what *should not* be included in ndn-cxx Application Design This section describes how to *design* an application that can work well with NDN architecture and ndn-cxx library. Data Naming - fundamental difference between NDN content distribution model and IP communication model - same content should have same Name - Data is immutable: version component - segmenting Interests - how Interest matches Data - discover names with Interest Selectors, then retrieve with predictable names - avoid abusing Selectors Security - Data carries signature - signed Interest for command - communication channel is not secured: secrets should be protected by encryption (not signed Interest) - trust management: identity, public key, certificate (more details in Security section) Application Implementation This section describes how to *implement* an application with ndn-cxx library. Basics - start a project: obtain build flags with pkg-config, include ndn-cxx headers, C++ namespace consideration - Name, Interest, Data classes - Face class, asynchronous operation, event loop Producer - prefix registration and Interest filter - in-app storage and its difference from ContentStore Consumer - Interest pipeling for segmented dataset - retransmissions Security - signing with KeyChain - verification with ValidatorConfig (more details in Security section) Integration with Other Components - Boost.Asio event loop - scheduler Interacting with NFD Management - nfd::Controller class and control commands - status dataset - notification stream - local control header Security This section explains more about security features. Concepts - identity, public key, certificate - TPM - PIB and PIB service - KeyChain - Validator Credential Management - select PIB and TPM with client.conf - ndnsec commands - publish certificates using PIB service Trust Model - hierarchical trust model, and how to validate with ValidatorConfig - web-of-trust trust model, and how to validate with ValidatorConfig - considerations on designing a custom trust model Library Internals This section describes internals of the ndn-cxx library, useful for those who want to extend the library. TLV encoding - constants in ndn::tlv namespace - Block and Buffer - EncodingBuffer and EncodingEstimator - how to implement a TLV abstraction type Transport - interface between Face and Transport - select transport with Face constructor - select transport with client.conf - how to implement a transport -------------- next part -------------- An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Tue Sep 9 15:17:57 2014 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Tue, 9 Sep 2014 22:17:57 +0000 Subject: [Nfd-dev] ndn-cxx devguide outline In-Reply-To: Message-ID: Hi Junxiao, Thanks for this. I'd propose that the specification of how all libraries must/should operate be separated from the specifics of developing using ndn-cxx. This would help immensely in NDN-CCL development, for example, where APIs, data types, and even application interaction may differ from ndn-cxx, but certain important functionality must be the same. I'm not exactly sure where the line should be drawn between spec and ndn-cxx approach, but have highlighted a few things below that I'd suggest could mostly described in a spec and then referenced / expanded upon in the developer guide. As an aside, now that NFD has reached its first major milestone, I'd also like to encourage the many developers engaged for that effort to consider contributing functionality and documentation to NDN-CCL as well. Jeff From: Junxiao Shi > Date: Tue, 9 Sep 2014 13:45:19 -0700 To: ">" > Subject: [Nfd-dev] ndn-cxx devguide outline Dear folks Here is a draft outline for ndn-cxx Application Developer Guide. The target audience is developers who intend to develop NDN applications with ndn-cxx library. The document should briefly explain ndn-cxx architecture, and provide some guidance of designing producer and consumer applications. Please provide your comments on what should go into the document, and what should be emphasized. You may reply-all to this message, or post a note on ndn-cxx Task 1581. Markdown source file can be found on ndn-cxx Task 1581 attachment. Yours, Junxiao ndn-cxx Application Developer Guide This document describes ndn-cxx: NDN C++ library with eXperimental eXtensions, and how to develop an NDN application using the ndn-cxx library. Introduction * history of ndn-cxx, comparison with ndn-ccl-api * what should be included in ndn-cxx * what should not be included in ndn-cxx Application Design This section describes how to design an application that can work well with NDN architecture and ndn-cxx library. None of these sections are specific to ndn-cxx, and should probably not be written in a library specific manner. Data Naming * fundamental difference between NDN content distribution model and IP communication model * same content should have same Name * Data is immutable: version component * segmenting Interests * how Interest matches Data * discover names with Interest Selectors, then retrieve with predictable names * avoid abusing Selectors Security * Data carries signature * signed Interest for command * communication channel is not secured: secrets should be protected by encryption (not signed Interest) * trust management: identity, public key, certificate (more details in Security section) Application Implementation This section describes how to implement an application with ndn-cxx library. Basics * start a project: obtain build flags with pkg-config, include ndn-cxx headers, C++ namespace consideration * Name, Interest, Data classes * Face class, asynchronous operation, event loop Producer * prefix registration and Interest filter * in-app storage and its difference from ContentStore The techniques here could be in a recommendations sections of a spec or some other type of design focused document? Consumer * Interest pipeling for segmented dataset * retransmissions The techniques here could be in a recommendations sections of a spec or some other type of design focused document? Security * signing with KeyChain * verification with ValidatorConfig (more details in Security section) Integration with Other Components * Boost.Asio event loop * scheduler Interacting with NFD Management * nfd::Controller class and control commands * status dataset * notification stream * local control header The APIs to NFD should be documented separately in a specification for any library to be written to. Security This section explains more about security features. Concepts * identity, public key, certificate * TPM * PIB and PIB service * KeyChain * Validator I could go both ways on these ? if these are considered "architectural" components, they need to be in a spec that is not ndn-cxx specific. Credential Management * select PIB and TPM with client.conf * ndnsec commands * publish certificates using PIB service Trust Model * hierarchical trust model, and how to validate with ValidatorConfig * web-of-trust trust model, and how to validate with ValidatorConfig * considerations on designing a custom trust model Library Internals This section describes internals of the ndn-cxx library, useful for those who want to extend the library. TLV encoding * constants in ndn::tlv namespace * Block and Buffer * EncodingBuffer and EncodingEstimator * how to implement a TLV abstraction type Transport * interface between Face and Transport * select transport with Face constructor * select transport with client.conf * how to implement a transport _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From gpau at CS.UCLA.EDU Tue Sep 9 17:32:47 2014 From: gpau at CS.UCLA.EDU (Giovanni Pau) Date: Tue, 9 Sep 2014 17:32:47 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor -- PLEAE LEAVE IT OPEN In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com>, Message-ID: Dear All, This for the V2V case represents a problem, we relay a lot on caching unsolicited content to achieve higher delivery rate. The V2V environment is so harsh that you can?t miss any opportunity to grab a content that may be useful in the future. I would be more incline to find a compromise based on a configuration parameter (i.e. this becomes a configuration issue per face) or to enable unsolicited content caching if there are some trust mechanism, however for us is really essential to keep this feature alive. Thanks Giovanni ========================== It had long since come to my attention that people of accomplishment rarely sat back and let things happen to them. They went out and happened to things. - Leonardo da Vinci ========================== On Sep 9, 2014, at 4:17 AM, Dave Oran (oran) wrote: > > > ___________________________ > iDevice - please excuse typos. > > On Sep 8, 2014, at 6:21 PM, Yingdi Yu wrote: > >> >> On Sep 8, 2014, at 12:58 PM, Dave Oran (oran) wrote: >> >>> I don?t mean to hijack this thread, but it seems that the architecture needs two cache timers, not one. >>> >>> One is an Expiry timer after which the data is no longer considered valid (and hence should not be returned from the cache if an interest arrives) >> >> I am not sure if it is always possible for a data producer to predict the time when the data will become invalid. > If not the producer, then who? All data kept outside an application's private (and possibly permanent) store has a validity period, if not shorter then at least no longer than the lifetime of the key that signed it. > >> Take DNS as an example, the authoritative server may never know when one of the records will be changed. >> > Wrong analogy. A DNS Authoritative server is like a repo, a producer is in effect the application that updated the DNS. > >> Yingdi > _______________________________________________ > Ndn-app mailing list > Ndn-app at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-app From shijunxiao at email.arizona.edu Tue Sep 9 21:50:18 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 9 Sep 2014 21:50:18 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: Dear folks Below is a summary of the problem, and a comparison of three solutions. "In-Forwarder Storage" is my proposed solution. Yours, Junxiao BackgroundContentStore concept ContentStore is an element of NFD which serves as an *opportunistic cache*. Incoming Data packets have the opportunity to be admitted into the ContentStore. Data packets are not persistent in ContentStore: they can be evicted when ContentStore is full. Admission to the ContentStore is a two step process: 1. Forwarding pipelines must decide whether a Data packet *SHOULD* be admitted. A Data packet should be admitted if it's believed to be legitimate (ie. not part of an attack). 2. ContentStore must decide whether there is space for this Data packet. Data packets in the ContentStore can be used to satisfy incoming Interests. When an incoming Interest contains *MustBeFresh=yes* selector, only *non-stale* Data packets can be returned. A Data packet is *stale* if it has been in the ContentStore for longer than*FreshnessPeriod*; note that if a Data packet is admitted while the same Data packet already exists in the ContentStore, the admission timestamp is reset, so this is equivalent to deleting the old Data packet and admitting the new one. Retention in the ContentStore is not guaranteed. The ContentStore is permitted to *evict* any Data packet at any time, regardless of its origin, staleness, admission time, etc. ContentStore in NFD 0.2.0 *Admission*: *Incoming Data pipeline* determines whether a Data packet *SHOULD* be admitted. If a Data packet satisfies one or more PIT entries, it should be admitted; this is regardless of whether an Interest has been forwarded to the origin of this Data packet. Otherwise, the Data packet is said to be *unsolicited*, and decision is given to *Data unsolicited pipeline*. Generally, unsolicited Data needs to be dropped as it poses security risks to the forwarder. However, there are cases when unsolicited Data packets needs to be accepted to the ContentStore. In particular, the current implementation allows any unsolicited Data packet to be cached if this Data packet arrives from a local Face. Once forwarding pipelines decide a Data should be admitted, ContentStore always admits it. If the ContentStore is full and a new Data packet needs to be admitted, one old Data packet is evicted. *Eviction*: The ContentStore logically maintains three FIFO queues: 1. unsolicited Data 2. stale Data 3. fresh Data The front Data packet in the first non-empty queue is evicted. Data Generation in ApplicationData Generation Misaligned with Interest Arrival NDN content distribution is receiver-driven. The producer cannot push Data packets into the network; it must wait for consumer to express Interests, before responding with Data packets. However, *Data generation can happen before Interest arrival*. - a video camera captures a consequtive video stream, and generates Data packets as a frame is captured - a NFD StatusDataset publisher generates all segments of a StatusDataset in response to a request, not one segment at a time - a ChronoChat client generates a Data packet when the user enters a message, before other users discover the existence of this Data packet (through synchronization) and retrieve it - a NLSR daemon generates a Data packet for a Link State Announcement, before other routers discover the existence of this Data packet (through synchronization) and retrieve it Producer is not allowed to send those Data packets into the network before Interest arrival. Therefore, those Data packets must be stored *somewhere*. Latest Version and Short Freshness Period The most useful Data packets are those representing the latest version of some information. The producer has the final authority: when an incoming Interest asks for the latest version, the producer would respond with the latest version. *FreshnessPeriod* field is an indication about how long the Data packet may remain the latest version. Typically, the producer sets this field to a duration in which it believes a newer version won't be generated. However, two situations cause the producer to prefer a shorter *FreshnessPeriod*: - *FreshnessPeriod* is per network node: if a Data packet is near the end of its freshness period at one node, and it's delivered to another node, the admitted Data packet on the second node would remain *non-stale* for the entire *FreshnessPeriod*. This means, if there are N hops between consumer and producer, in the worst case, the Data may appear fresh to consumer for up to N times*FreshnessPeriod*. - The producer is sometimes unable to predict how long a version can remain to be the latest version. Problem The "NFD backdoor" is a *feature*: NFD allows unsolicited Data from a local face to be admitted to the ContentStore. This feature allows NDN applications to "pre-publish" Data packets when future Interests are anticipated. At the other end, the NDN applications themselves do not retain the Data packet. The drawback of this approach is: *retention is not guaranteed*. Although pre-published Data packets are admitted to the ContentStore, they are tagged as unsolicited. Unsolicited Data packets are the first to be evicted from ContentStore when ContentStore is full. Therefore, when there are enough solicited Data packets in the ContentStore, unsolicited Data packets can hardly be retained in the ContentStore. If a Data packet is evicted, the Interest would be forwarded to the application. The application either has to generate the Data again, or is unable to respond at all. Another problem of application not storing generated Data packets is: *a Data packet may remain to be the latest version beyond FreshnessPeriod*. The ContentStore can only use *non-stale* Data to satisfy an Interest with *MustBeFresh=yes*. If the producer sets a short *FreshnessPeriod*, the Interest would be forwarded to the application, even if the Data packet is still the latest version. The application would have to generate (and sign) the Data again. Necessity of Application-Controllable Data Storage Both problems have a common solution: a Data storage controlled by application. This Data storage should have the following characteristics: - Data packets *useful* to a producer application are stored. A Data packet is *useful*, if the producer application would generate the same again when an Interest asks for it. - Producer can explicitly delete Data packets that are no longer *useful*. The most common time to delete Data packets is when a new version is generated. - Producer should delete Data packets when the Data storage is close to its capacity. Such deletions can either be explicit, or be delegated to an eviction policy. - The Data storage is not a cache. Solutions The application-controllable Data storage can either be placed in the application, or be placed in the forwarder. In-App Storage In-App Storage provides an application-controllable Data storage within the application. The module is typically provided by the library. Benefits: - keep the forwarder simple - application has a chance to inspect the Data again before using it to satisfy incoming Interest Drawbacks: - duplication: when a Data packet is used to satisfy an Interest, it would have copies in both In-App Storage and ContentStore - risk of Data loss: when a Data packet is used to satisfy an Interest, if application chooses to delete the copy in In-App Storage, the Data would be lost when the copy in ContentStore is evicted Managed ContentStore Managed ContentStore provides an application-controllable Data storage as part of the ContentStore. A Data packet is explicitly inserted to the ContentStore, and the ContentStore guarantees to retain the Data packet until it becomes *stale* or the application requests to delete it. Benefits: - no duplication Drawbacks: - ContentStore is no longer a cache; eviction policy becomes more complex - application must refresh the stored Data packet, if it remains the latest version beyond *FreshnessPeriod*; setting a large *FreshnessPeriod* is still infeasible because in-network caches may retain the Data packet for too long, this means refreshing would be very frequent - higher deletion cost: deletion needs a command - inability to enumerate: unlike In-App Storage, the application is unable to enumerate stored Data packets In-Forwarder Storage An application-controllable Data storage can be placed in the forwarder, but out of the ContentStore. - The application sends a command to create an application-controllable Data storage in the forwarder. - The Data storage is logically attached to the face of this application. Its capacity and replacement policy are determined when it's created. Its destroyed when face is closing. - To insert a Data packet to the Data storage, the application must explicitly set a field in the LocalControlHeader. - Insertion to the Data storage is independant from forwarding pipelines, but the application can elect to deliver the same Data packet into forwarding pipelines. - *Data unsolicited pipeline* would not admit unsolicited Data packets to the ContentStore, except in special environments such as vehicle networks. - The application can scheduled the deletion of a Data packet during insertion. - The deletion timer can be set differently from *FreshnessPeriod*. - The application can delete Data packets by sending a command. - Command parameter is a Name prefix. All stored Data packets under this prefix are deleted. - Stored Data packets are considered only if an Interest is to be forwarded to the application. - When an Interest is to be forwarded to the application, the Data storage attached to the face is queried. If a matching Data packet is found, the Interest is not sent, and the Data packet is treated as if it's received from the application. Benefits: - no duplication: if a Data packet appears in both application-controllable Data storage and the ContentStore, it has only one in-memory copy, with references from two places - less inter-process communication and encoding/decoding, when most stored Data packets are requested more than once - no risk of cache poisoning: the application-controllable Data storage is not considered until an Interest is to be forwarded to the application - no refreshing needed Drawbacks: - higher deletion cost: deletion needs a command - inability to enumerate: unlike In-App Storage, the application is unable to enumerate stored Data packets - address space consumption: stored Data packets are mapped in forwarder's address space, and address space is limited on 32-bit platforms -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: contentstore-appstorage_20140909.md Type: application/octet-stream Size: 10882 bytes Desc: not available URL: From jburke at remap.UCLA.EDU Tue Sep 9 22:29:21 2014 From: jburke at remap.UCLA.EDU (Burke, Jeff) Date: Wed, 10 Sep 2014 05:29:21 +0000 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: Message-ID: One could imagine different configurable strategies for the MemoryContentCache depending on the way the application uses FreshnessPeriod. I'm not sure exactly the end goal of this discussion? I haven't been closely following all the messages but I think Dave Oran alluded to the fact that just having FreshnessPeriod may not be enough. This has come up in the past in other contexts and may be worth discussing. You state "My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store." This is fair, but imagine three different consumer applications for the same ndnrtc streams, all of which have different opinions about what content from a single publisher is relevant to be in a content store - 1. "real-time" live-playback (current app) - only needs freshest data (within some buffer time period) 2. Fixed ten-second expletive-removing delay consumer ? can have a fetching buffer that's significantly larger (though not the full seven seconds) than the above, but otherwise is "real-time" 3. digital video recorder style playback ? may have much larger delays, and fetch content that may be minutes, days, hours older than the freshest content while other consumers are also watching the freshest content. Given these three consumers for a single publisher of live video, how should FreshnessPeriod be used to the best effect / appropriately? J. From: Yingdi Yu > Date: Mon, 8 Sep 2014 11:29:56 -0700 To: Jeff Thompson > Cc: ">" >, "ndn-app at lists.cs.ucla.edu" > Subject: Re: [Nfd-dev] Close NFD backdoor Hi JeffT, On Sep 8, 2014, at 2:53 AM, Thompson, Jeff > wrote: The NDN-RTC video conferencing app uses the MemoryContentCache in-app storage. It keeps data in the cache based on the data packet's FreshnessPeriod. http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html You say "content store is aware of FreshnessPeriod of data packets, but an in-app storage should not." Why should the in-app storage not be aware of the FreshnessPeriod? In the NDN-TLV spec, the definition of FreshnessPeriod says "Note that the stale data is still valid data; the expiration of FreshnessPeriod only means that the producer may have produced newer data." My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store. The end producer is the only one who knows and can decide whether a data packet is valid or not. And the in-memory storage contains data packets that are treated as valid by end producer. For example, producer A generates a data packet "/a/b/c" with 10-second FreshnessPeriod. If "/a/b/c" is put into the MemoryContentCache, the packet will be removed from the cache after 10 seconds, and the producer A will have to generate the same data packet again even if the producer A still takes "/a/b/c" as valid. With the in-memory storage, the packet will never be removed until the producer generates a new valid data packet to replace "/a/b/c". And "/a/b/c" will be obsolete by the new data packet. Yingdi _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzhang at cs.ARIZONA.EDU Tue Sep 9 22:49:04 2014 From: bzhang at cs.ARIZONA.EDU (Beichuan Zhang) Date: Tue, 9 Sep 2014 22:49:04 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: I feel things are becoming too complicated. Here?s my opinion: The backdoor, i.e., caching unsolicited data from local apps, was a hack to make some apps work quickly. It is not the proper way to write the app nor the forwarder. Apps should take care of their own data (think e2e argument). They should work with or without local CS, or with any size of local CS. In other words, they shouldn?t rely on the backdoor. The most effective way to take care of one?s data is to store and manage them in the app itself. So an in-app storage mechanism such as MemoryContentCache is the way to go. An app can also use a database, the file system, or any other way to manage its data. If data duplication, one copy in app and one copy in CS, is a concern, the CS can be made not to cache data coming from local apps. Caching such data doesn?t improve user-perceived performance much anyway. Solving app?s data management problem in CS will only make the forwarder unnecessarily complicated, and I don?t see any clear gain from doing that. Beichuan On Sep 9, 2014, at 9:50 PM, Junxiao Shi wrote: > Dear folks > > Below is a summary of the problem, and a comparison of three solutions. > "In-Forwarder Storage" is my proposed solution. > > Yours, Junxiao > > Background > > ContentStore concept > > ContentStore is an element of NFD which serves as an opportunistic cache. Incoming Data packets have the opportunity to be admitted into the ContentStore. Data packets are not persistent in ContentStore: they can be evicted when ContentStore is full. > > Admission to the ContentStore is a two step process: > > Forwarding pipelines must decide whether a Data packet SHOULD be admitted. A Data packet should be admitted if it's believed to be legitimate (ie. not part of an attack). > ContentStore must decide whether there is space for this Data packet. > Data packets in the ContentStore can be used to satisfy incoming Interests. > When an incoming Interest contains MustBeFresh=yes selector, only non-stale Data packets can be returned. A Data packet is stale if it has been in the ContentStore for longer thanFreshnessPeriod; note that if a Data packet is admitted while the same Data packet already exists in the ContentStore, the admission timestamp is reset, so this is equivalent to deleting the old Data packet and admitting the new one. > > Retention in the ContentStore is not guaranteed. The ContentStore is permitted to evict any Data packet at any time, regardless of its origin, staleness, admission time, etc. > > ContentStore in NFD 0.2.0 > > Admission: Incoming Data pipeline determines whether a Data packet SHOULD be admitted. If a Data packet satisfies one or more PIT entries, it should be admitted; this is regardless of whether an Interest has been forwarded to the origin of this Data packet. Otherwise, the Data packet is said to be unsolicited, and decision is given to Data unsolicited pipeline. > > Generally, unsolicited Data needs to be dropped as it poses security risks to the forwarder. However, there are cases when unsolicited Data packets needs to be accepted to the ContentStore. In particular, the current implementation allows any unsolicited Data packet to be cached if this Data packet arrives from a local Face. > > Once forwarding pipelines decide a Data should be admitted, ContentStore always admits it. If the ContentStore is full and a new Data packet needs to be admitted, one old Data packet is evicted. > > Eviction: The ContentStore logically maintains three FIFO queues: > > unsolicited Data > stale Data > fresh Data > The front Data packet in the first non-empty queue is evicted. > > Data Generation in Application > > Data Generation Misaligned with Interest Arrival > > NDN content distribution is receiver-driven. The producer cannot push Data packets into the network; it must wait for consumer to express Interests, before responding with Data packets. > > However, Data generation can happen before Interest arrival. > > a video camera captures a consequtive video stream, and generates Data packets as a frame is captured > a NFD StatusDataset publisher generates all segments of a StatusDataset in response to a request, not one segment at a time > a ChronoChat client generates a Data packet when the user enters a message, before other users discover the existence of this Data packet (through synchronization) and retrieve it > a NLSR daemon generates a Data packet for a Link State Announcement, before other routers discover the existence of this Data packet (through synchronization) and retrieve it > Producer is not allowed to send those Data packets into the network before Interest arrival. Therefore, those Data packets must be stored somewhere. > > Latest Version and Short Freshness Period > > The most useful Data packets are those representing the latest version of some information. The producer has the final authority: when an incoming Interest asks for the latest version, the producer would respond with the latest version. > > FreshnessPeriod field is an indication about how long the Data packet may remain the latest version. Typically, the producer sets this field to a duration in which it believes a newer version won't be generated. However, two situations cause the producer to prefer a shorter FreshnessPeriod: > > FreshnessPeriod is per network node: if a Data packet is near the end of its freshness period at one node, and it's delivered to another node, the admitted Data packet on the second node would remain non-stale for the entire FreshnessPeriod. This means, if there are N hops between consumer and producer, in the worst case, the Data may appear fresh to consumer for up to N timesFreshnessPeriod. > The producer is sometimes unable to predict how long a version can remain to be the latest version. > Problem > > The "NFD backdoor" is a feature: NFD allows unsolicited Data from a local face to be admitted to the ContentStore. > > This feature allows NDN applications to "pre-publish" Data packets when future Interests are anticipated. At the other end, the NDN applications themselves do not retain the Data packet. > > The drawback of this approach is: retention is not guaranteed. > Although pre-published Data packets are admitted to the ContentStore, they are tagged as unsolicited. Unsolicited Data packets are the first to be evicted from ContentStore when ContentStore is full. Therefore, when there are enough solicited Data packets in the ContentStore, unsolicited Data packets can hardly be retained in the ContentStore. > If a Data packet is evicted, the Interest would be forwarded to the application. The application either has to generate the Data again, or is unable to respond at all. > > Another problem of application not storing generated Data packets is: a Data packet may remain to be the latest version beyond FreshnessPeriod. > The ContentStore can only use non-stale Data to satisfy an Interest with MustBeFresh=yes. If the producer sets a short FreshnessPeriod, the Interest would be forwarded to the application, even if the Data packet is still the latest version. > The application would have to generate (and sign) the Data again. > > Necessity of Application-Controllable Data Storage > > Both problems have a common solution: a Data storage controlled by application. > > This Data storage should have the following characteristics: > > Data packets useful to a producer application are stored. A Data packet is useful, if the producer application would generate the same again when an Interest asks for it. > Producer can explicitly delete Data packets that are no longer useful. The most common time to delete Data packets is when a new version is generated. > Producer should delete Data packets when the Data storage is close to its capacity. Such deletions can either be explicit, or be delegated to an eviction policy. > The Data storage is not a cache. > Solutions > > The application-controllable Data storage can either be placed in the application, or be placed in the forwarder. > > In-App Storage > > In-App Storage provides an application-controllable Data storage within the application. The module is typically provided by the library. > > Benefits: > > keep the forwarder simple > application has a chance to inspect the Data again before using it to satisfy incoming Interest > Drawbacks: > > duplication: when a Data packet is used to satisfy an Interest, it would have copies in both In-App Storage and ContentStore > risk of Data loss: when a Data packet is used to satisfy an Interest, if application chooses to delete the copy in In-App Storage, the Data would be lost when the copy in ContentStore is evicted > Managed ContentStore > > Managed ContentStore provides an application-controllable Data storage as part of the ContentStore. A Data packet is explicitly inserted to the ContentStore, and the ContentStore guarantees to retain the Data packet until it becomes stale or the application requests to delete it. > > Benefits: > > no duplication > Drawbacks: > > ContentStore is no longer a cache; eviction policy becomes more complex > application must refresh the stored Data packet, if it remains the latest version beyond FreshnessPeriod; setting a large FreshnessPeriod is still infeasible because in-network caches may retain the Data packet for too long, this means refreshing would be very frequent > higher deletion cost: deletion needs a command > inability to enumerate: unlike In-App Storage, the application is unable to enumerate stored Data packets > In-Forwarder Storage > > An application-controllable Data storage can be placed in the forwarder, but out of the ContentStore. > > The application sends a command to create an application-controllable Data storage in the forwarder. > The Data storage is logically attached to the face of this application. Its capacity and replacement policy are determined when it's created. Its destroyed when face is closing. > To insert a Data packet to the Data storage, the application must explicitly set a field in the LocalControlHeader. > Insertion to the Data storage is independant from forwarding pipelines, but the application can elect to deliver the same Data packet into forwarding pipelines. > Data unsolicited pipeline would not admit unsolicited Data packets to the ContentStore, except in special environments such as vehicle networks. > The application can scheduled the deletion of a Data packet during insertion. > The deletion timer can be set differently from FreshnessPeriod. > The application can delete Data packets by sending a command. > Command parameter is a Name prefix. All stored Data packets under this prefix are deleted. > Stored Data packets are considered only if an Interest is to be forwarded to the application. > When an Interest is to be forwarded to the application, the Data storage attached to the face is queried. If a matching Data packet is found, the Interest is not sent, and the Data packet is treated as if it's received from the application. > Benefits: > > no duplication: if a Data packet appears in both application-controllable Data storage and the ContentStore, it has only one in-memory copy, with references from two places > less inter-process communication and encoding/decoding, when most stored Data packets are requested more than once > no risk of cache poisoning: the application-controllable Data storage is not considered until an Interest is to be forwarded to the application > no refreshing needed > Drawbacks: > > higher deletion cost: deletion needs a command > inability to enumerate: unlike In-App Storage, the application is unable to enumerate stored Data packets > address space consumption: stored Data packets are mapped in forwarder's address space, and address space is limited on 32-bit platforms > _______________________________________________ > Ndn-app mailing list > Ndn-app at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-app -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzhang at cs.ARIZONA.EDU Tue Sep 9 22:55:39 2014 From: bzhang at cs.ARIZONA.EDU (Beichuan Zhang) Date: Tue, 9 Sep 2014 22:55:39 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor -- PLEAE LEAVE IT OPEN In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com>, Message-ID: We?re referring to unsolicited data received from apps on the same node. What you need, to my understanding, is caching unsolicited data from external broadcast faces such as WiFi, which is not part of this ?backdoor? and can be supported. Beichuan On Sep 9, 2014, at 5:32 PM, Giovanni Pau wrote: > Dear All, > > This for the V2V case represents a problem, we relay a lot on caching unsolicited content to achieve higher delivery rate. The V2V environment is so harsh that you can?t miss any opportunity to grab a content that may be useful in the future. > > I would be more incline to find a compromise based on a configuration parameter (i.e. this becomes a configuration issue per face) or to enable unsolicited content caching if there are some trust mechanism, however for us is really essential to keep this feature alive. > > Thanks > Giovanni -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Tue Sep 9 23:02:34 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 9 Sep 2014 23:02:34 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: Hi Beichuan My proposal does not store application's Data packets into the ContentStore. The In-Forwarder Data Storage is separate from ContentStore, although packets can be shared via shared_ptr. Using In-App Storage and not caching Data in ContentStore is compared to my proposal as follows. Benefits: * less memory usage by forwarder, which means less memory management overhead in forwarder process and thread * app is able to enumerate stored Data packets * deletion is easier Drawbacks: * more inter-process communication and encoding/decoding overhead Same: * no duplication * PIT states are generated. Note: in NFD, PIT entry is created even if Interest can be satisfied by ContentStore, because this is needed for loop detection and measurements. Not caching in ContentStore doesn't increase PIT states. I agree that using In-App Storage and not caching Data in ContentStore is better than In-Forwarder Data Storage. Yours, Junxiao On Sep 9, 2014 10:49 PM, "Beichuan Zhang" wrote: > I feel things are becoming too complicated. Here?s my opinion: > > The backdoor, i.e., caching unsolicited data from local apps, was a hack > to make some apps work quickly. It is not the proper way to write the app > nor the forwarder. Apps should take care of their own data (think e2e > argument). They should work with or without local CS, or with any size of > local CS. In other words, they shouldn?t rely on the backdoor. > > The most effective way to take care of one?s data is to store and manage > them in the app itself. So an in-app storage mechanism such as > MemoryContentCache is the way to go. An app can also use a database, the > file system, or any other way to manage its data. > > If data duplication, one copy in app and one copy in CS, is a concern, the > CS can be made not to cache data coming from local apps. Caching such data > doesn?t improve user-perceived performance much anyway. > > Solving app?s data management problem in CS will only make the forwarder > unnecessarily complicated, and I don?t see any clear gain from doing that. > > Beichuan > > > On Sep 9, 2014, at 9:50 PM, Junxiao Shi > wrote: > > Dear folks > > Below is a summary of the problem, and a comparison of three solutions. > "In-Forwarder Storage" is my proposed solution. > > Yours, Junxiao > > BackgroundContentStore concept > > ContentStore is an element of NFD which serves as an *opportunistic cache*. > Incoming Data packets have the opportunity to be admitted into the > ContentStore. Data packets are not persistent in ContentStore: they can be > evicted when ContentStore is full. > > Admission to the ContentStore is a two step process: > > 1. Forwarding pipelines must decide whether a Data packet *SHOULD* be > admitted. A Data packet should be admitted if it's believed to be > legitimate (ie. not part of an attack). > 2. ContentStore must decide whether there is space for this Data > packet. > > Data packets in the ContentStore can be used to satisfy incoming > Interests. > When an incoming Interest contains *MustBeFresh=yes* selector, only > *non-stale* Data packets can be returned. A Data packet is *stale* if it > has been in the ContentStore for longer than*FreshnessPeriod*; note that > if a Data packet is admitted while the same Data packet already exists in > the ContentStore, the admission timestamp is reset, so this is equivalent > to deleting the old Data packet and admitting the new one. > > Retention in the ContentStore is not guaranteed. The ContentStore is > permitted to *evict* any Data packet at any time, regardless of its > origin, staleness, admission time, etc. > ContentStore in NFD 0.2.0 > > *Admission*: *Incoming Data pipeline* determines whether a Data packet > *SHOULD* be admitted. If a Data packet satisfies one or more PIT entries, > it should be admitted; this is regardless of whether an Interest has been > forwarded to the origin of this Data packet. Otherwise, the Data packet is > said to be *unsolicited*, and decision is given to *Data unsolicited > pipeline*. > > Generally, unsolicited Data needs to be dropped as it poses security risks > to the forwarder. However, there are cases when unsolicited Data packets > needs to be accepted to the ContentStore. In particular, the current > implementation allows any unsolicited Data packet to be cached if this Data > packet arrives from a local Face. > > Once forwarding pipelines decide a Data should be admitted, ContentStore > always admits it. If the ContentStore is full and a new Data packet needs > to be admitted, one old Data packet is evicted. > > *Eviction*: The ContentStore logically maintains three FIFO queues: > > 1. unsolicited Data > 2. stale Data > 3. fresh Data > > The front Data packet in the first non-empty queue is evicted. > Data Generation in ApplicationData Generation Misaligned with Interest > Arrival > > NDN content distribution is receiver-driven. The producer cannot push Data > packets into the network; it must wait for consumer to express Interests, > before responding with Data packets. > > However, *Data generation can happen before Interest arrival*. > > - a video camera captures a consequtive video stream, and generates > Data packets as a frame is captured > - a NFD StatusDataset > publisher > generates all segments of a StatusDataset in response to a request, not one > segment at a time > - a ChronoChat client > generates a Data packet when the user enters a message, before other users > discover the existence of this Data packet (through synchronization) and > retrieve it > - a NLSR daemon generates a Data > packet for a Link State Announcement, before other routers discover the > existence of this Data packet (through synchronization) and retrieve it > > Producer is not allowed to send those Data packets into the network before > Interest arrival. Therefore, those Data packets must be stored *somewhere* > . > Latest Version and Short Freshness Period > > The most useful Data packets are those representing the latest version of > some information. The producer has the final authority: when an incoming > Interest asks for the latest version, the producer would respond with the > latest version. > > *FreshnessPeriod* field is an indication about how long the Data packet > may remain the latest version. Typically, the producer sets this field to a > duration in which it believes a newer version won't be generated. However, > two situations cause the producer to prefer a shorter *FreshnessPeriod*: > > - *FreshnessPeriod* is per network node: if a Data packet is near the > end of its freshness period at one node, and it's delivered to another > node, the admitted Data packet on the second node would remain > *non-stale* for the entire *FreshnessPeriod*. This means, if there are > N hops between consumer and producer, in the worst case, the Data may > appear fresh to consumer for up to N times*FreshnessPeriod*. > - The producer is sometimes unable to predict how long a version can > remain to be the latest version. > > Problem > > The "NFD backdoor" is a *feature*: NFD allows unsolicited Data from a > local face to be admitted to the ContentStore. > > This feature allows NDN applications to "pre-publish" Data packets when > future Interests are anticipated. At the other end, the NDN applications > themselves do not retain the Data packet. > > The drawback of this approach is: *retention is not guaranteed*. > Although pre-published Data packets are admitted to the ContentStore, they > are tagged as unsolicited. Unsolicited Data packets are the first to be > evicted from ContentStore when ContentStore is full. Therefore, when there > are enough solicited Data packets in the ContentStore, unsolicited Data > packets can hardly be retained in the ContentStore. > If a Data packet is evicted, the Interest would be forwarded to the > application. The application either has to generate the Data again, or is > unable to respond at all. > > Another problem of application not storing generated Data packets is: *a > Data packet may remain to be the latest version beyond FreshnessPeriod*. > The ContentStore can only use *non-stale* Data to satisfy an Interest > with *MustBeFresh=yes*. If the producer sets a short *FreshnessPeriod*, > the Interest would be forwarded to the application, even if the Data packet > is still the latest version. > The application would have to generate (and sign) the Data again. > Necessity of Application-Controllable Data Storage > > Both problems have a common solution: a Data storage controlled by > application. > > This Data storage should have the following characteristics: > > - Data packets *useful* to a producer application are stored. A Data > packet is *useful*, if the producer application would generate the > same again when an Interest asks for it. > - Producer can explicitly delete Data packets that are no longer > *useful*. The most common time to delete Data packets is when a new > version is generated. > - Producer should delete Data packets when the Data storage is close > to its capacity. Such deletions can either be explicit, or be delegated to > an eviction policy. > - The Data storage is not a cache. > > Solutions > > The application-controllable Data storage can either be placed in the > application, or be placed in the forwarder. > In-App Storage > > In-App Storage > provides > an application-controllable Data storage within the application. The module > is typically provided by the library. > > Benefits: > > - keep the forwarder simple > - application has a chance to inspect the Data again before using it > to satisfy incoming Interest > > Drawbacks: > > - duplication: when a Data packet is used to satisfy an Interest, it > would have copies in both In-App Storage and ContentStore > - risk of Data loss: when a Data packet is used to satisfy an > Interest, if application chooses to delete the copy in In-App Storage, the > Data would be lost when the copy in ContentStore is evicted > > Managed ContentStore > > Managed ContentStore > provides > an application-controllable Data storage as part of the ContentStore. A > Data packet is explicitly inserted to the ContentStore, and the > ContentStore guarantees to retain the Data packet until it becomes *stale* or > the application requests to delete it. > > Benefits: > > - no duplication > > Drawbacks: > > - ContentStore is no longer a cache; eviction policy becomes more > complex > - application must refresh the stored Data packet, if it remains the > latest version beyond *FreshnessPeriod*; setting a large > *FreshnessPeriod* is still infeasible because in-network caches may > retain the Data packet for too long, this means refreshing would be very > frequent > - higher deletion cost: deletion needs a command > - inability to enumerate: unlike In-App Storage, the application is > unable to enumerate stored Data packets > > In-Forwarder Storage > > An application-controllable Data storage can be placed in the forwarder, > but out of the ContentStore. > > - The application sends a command to create an > application-controllable Data storage in the forwarder. > - The Data storage is logically attached to the face of this > application. Its capacity and replacement policy are determined when it's > created. Its destroyed when face is closing. > - To insert a Data packet to the Data storage, the application must > explicitly set a field in the LocalControlHeader. > - Insertion to the Data storage is independant from forwarding > pipelines, but the application can elect to deliver the same Data packet > into forwarding pipelines. > - *Data unsolicited pipeline* would not admit unsolicited Data > packets to the ContentStore, except in special environments such as vehicle > networks. > - The application can scheduled the deletion of a Data packet during > insertion. > - The deletion timer can be set differently from *FreshnessPeriod*. > - The application can delete Data packets by sending a command. > - Command parameter is a Name prefix. All stored Data packets under > this prefix are deleted. > - Stored Data packets are considered only if an Interest is to be > forwarded to the application. > - When an Interest is to be forwarded to the application, the Data > storage attached to the face is queried. If a matching Data packet is > found, the Interest is not sent, and the Data packet is treated as if it's > received from the application. > > Benefits: > > - no duplication: if a Data packet appears in both > application-controllable Data storage and the ContentStore, it has only one > in-memory copy, with references from two places > - less inter-process communication and encoding/decoding, when most > stored Data packets are requested more than once > - no risk of cache poisoning: the application-controllable Data > storage is not considered until an Interest is to be forwarded to the > application > - no refreshing needed > > Drawbacks: > > - higher deletion cost: deletion needs a command > - inability to enumerate: unlike In-App Storage, the application is > unable to enumerate stored Data packets > - address space consumption: stored Data packets are mapped in > forwarder's address space, and address space is limited on 32-bit platforms > > > _______________________________________________ > Ndn-app mailing list > Ndn-app at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-app > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Tue Sep 9 23:06:42 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Tue, 9 Sep 2014 23:06:42 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor -- PLEAE LEAVE IT OPEN In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> <78B7DB56-44E2-40D5-B12F-6F1B73BE3558@cisco.com> Message-ID: Dear folks Admitting unsolicited Data from local applications, and admitted all Data from wireless face, are two different features. But they are related, and they are both controlled by a policy in *Data unsolicited pipeline*. The current policy is admitting Data from local faces and dropping others. This can be changed to fit the characteristics of vehicle networks. Yours, Junxiao On Sep 9, 2014 10:56 PM, "Beichuan Zhang" wrote: > > We?re referring to unsolicited data received from apps on the same node. What you need, to my understanding, is caching unsolicited data from external broadcast faces such as WiFi, which is not part of this ?backdoor? and can be supported. > > Beichuan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Tue Sep 9 23:20:45 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Tue, 9 Sep 2014 23:20:45 -0700 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: References: Message-ID: <261DE30F-DEDA-4470-A58F-61E0CE2ABB87@cs.ucla.edu> Hi Jeff, On Sep 9, 2014, at 10:29 PM, Burke, Jeff wrote: > One could imagine different configurable strategies for the MemoryContentCache depending on the way the application uses FreshnessPeriod. I'm not sure exactly the end goal of this discussion? I did not mean MemoryContentCache is wrong. In my own opinion, it is generally good to have app manage its own data packets. About how to manage the data, I think it is up to the application. > You state "My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store." > > This is fair, but imagine three different consumer applications for the same ndnrtc streams, all of which have different opinions about what content from a single publisher is relevant to be in a content store - > 1. "real-time" live-playback (current app) - only needs freshest data (within some buffer time period) > 2. Fixed ten-second expletive-removing delay consumer ? can have a fetching buffer that's significantly larger (though not the full seven seconds) than the above, but otherwise is "real-time" > 3. digital video recorder style playback ? may have much larger delays, and fetch content that may be minutes, days, hours older than the freshest content while other consumers are also watching the freshest content. > > Given these three consumers for a single publisher of live video, how should FreshnessPeriod be used to the best effect / appropriately? I am not trying to define how to specify FreshnessPeriod. What I want to push for is an in-app data storage which can be explicitly managed by the app. Since the app is the producer of the data packet, so it should be able to determine the validity of a data packet without relying on FreshnessPeriod. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From yingdi at CS.UCLA.EDU Tue Sep 9 23:34:28 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Tue, 9 Sep 2014 23:34:28 -0700 Subject: [Nfd-dev] [Ndn-app] Close NFD backdoor In-Reply-To: References: <1DBD3AF1-6A51-4E11-8996-764CA74D4893@cs.ucla.edu> Message-ID: <9B05181A-02FD-47A6-A2A5-08B63D88EDAE@cs.ucla.edu> Hi Junxiao, Just two quick comments about in-app storage. On Sep 9, 2014, at 9:50 PM, Junxiao Shi wrote: > In-App Storage > > In-App Storage provides an application-controllable Data storage within the application. The module is typically provided by the library. > > Benefits: > > keep the forwarder simple > application has a chance to inspect the Data again before using it to satisfy incoming Interest > Drawbacks: > > duplication: when a Data packet is used to satisfy an Interest, it would have copies in both In-App Storage and ContentStore This can be solved if app can tell nfd not to cache the data. Given this only happens on a localhost, it can be done by extending the local control header. Although there are some encoding/decoding overhead, we do not have change the architecture. > risk of Data loss: when a Data packet is used to satisfy an Interest, if application chooses to delete the copy in In-App Storage, the Data would be lost when the copy in ContentStore is evicted As the producer of the data packet, the app should be responsible for taking care of its own data. If the app wants to keep the data persistently available, it should look for some persistent storage service for that purpose, rather than relying on nfd. If the producer decides to discard a data packet, then there is no point for local ContentStore to keep it. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jburke at remap.ucla.edu Tue Sep 9 23:36:27 2014 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Wed, 10 Sep 2014 06:36:27 +0000 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: <261DE30F-DEDA-4470-A58F-61E0CE2ABB87@cs.ucla.edu> Message-ID: From: Yingdi Yu > Date: Tue, 9 Sep 2014 23:20:45 -0700 To: Jeff Burke > Cc: Jeff Thompson >, ">" >, "ndn-app at lists.cs.ucla.edu" > Subject: Re: [Nfd-dev] Close NFD backdoor Hi Jeff, On Sep 9, 2014, at 10:29 PM, Burke, Jeff > wrote: One could imagine different configurable strategies for the MemoryContentCache depending on the way the application uses FreshnessPeriod. I'm not sure exactly the end goal of this discussion? I did not mean MemoryContentCache is wrong. In my own opinion, it is generally good to have app manage its own data packets. About how to manage the data, I think it is up to the application. You state "My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store." This is fair, but imagine three different consumer applications for the same ndnrtc streams, all of which have different opinions about what content from a single publisher is relevant to be in a content store - 1. "real-time" live-playback (current app) - only needs freshest data (within some buffer time period) 2. Fixed ten-second expletive-removing delay consumer ? can have a fetching buffer that's significantly larger (though not the full seven seconds) than the above, but otherwise is "real-time" 3. digital video recorder style playback ? may have much larger delays, and fetch content that may be minutes, days, hours older than the freshest content while other consumers are also watching the freshest content. Given these three consumers for a single publisher of live video, how should FreshnessPeriod be used to the best effect / appropriately? I am not trying to define how to specify FreshnessPeriod. What I want to push for is an in-app data storage which can be explicitly managed by the app. Since the app is the producer of the data packet, so it should be able to determine the validity of a data packet without relying on FreshnessPeriod. Yes, ok, I think we agree on that. But I am still interested in figuring out FreshnessPeriod, too, separately. :) Jeff Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: From obaidasyed at gmail.com Wed Sep 10 09:09:40 2014 From: obaidasyed at gmail.com (Syed Obaid Amin) Date: Wed, 10 Sep 2014 11:09:40 -0500 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Hi Junxiao, Thanks for the fix. The patch improved the results significantly. For e.g., have a look at the following results: Old strategy: http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-oldncc_nolog/arizona-caida-converge.png Patched: http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-ncc_nolog/arizona-caida-converge.png The topology I used for these tests can be found here: http://netwisdom.cs.memphis.edu/imgs/topology.png Regards, Obaid On Mon, Sep 8, 2014 at 7:36 PM, Junxiao Shi wrote: > Hi Lan > > Bug 1961 has been fixed. Code is merged to master branch. > Please test in your internal testbed. > > The correct behavior shall be: > > 1. P1 is forwarded to UCLA, and could be forwarded to CSU and other > nexthops as well > 2. UCLA responds first and remains as best face; other responses are > received but ignored > 3. P2 should be forwarded to UCLA > > > This bug is not critical to end users. I don't plan to backport to v0.2. > It won't be deployed on testbed routers until v0.3 release. > > Yours, Junxiao > > > On Tue, Sep 2, 2014 at 10:26 AM, Junxiao Shi > wrote: > >> Hi Lan >> >> Thanks for the report. This shall be a bug of NCC strategy. >> >> In ccnd, >> >> 1. When first Data comes back that satisfies the PIT entry, the >> incoming face of that Data is remembered. >> 2. The PIT entry is deleted as soon as it's satisfied. >> 3. When second Data comes back, it's discarded as unsolicited. >> >> In NFD, >> >> 1. When first Data comes back that satisfies the PIT entry, the >> incoming face of that Data is remembered. >> 2. The PIT entry is kept, and will be deleted after straggler timer. >> 3. When second Data comes back, the strategy is notified, and >> remembers the incoming face. >> >> NFD's NCC strategy incorrectly assumes the incoming Data is from the >> fastest upstream, causing the bug. >> >> I'll report the bug soon. >> >> Yours, Junxiao >> >> >> On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) >> wrote: >> >>> Junxiao, >>> >>> We observed something strange in one of our experiments when we used the >>> ncc strategy (and link state routing). In some of the pings from Arizona >>> to PKU, the ndnping delays were really large (>350ms) even though the >>> shortest path delay is only around 200ms. We looked at the ndnping data, >>> and found where the delay jumped from ~200ms to ~350ms. It turns out that >>> instead of going through the shortest path (Arizona->UCLA->PKU), it is >>> going through a much longer path (Arizona->CSU->?->NEU->PKU). >>> >>> To explain what happened, I'll call the ping before the change P1 and >>> the ping after the change P2. Further looking into the nfd logs, it seems >>> for P1 Arizona first sent the Interest to UCLA (which we expected), but >>> while waiting for the data to come back, it also tried CSU and some other >>> neighbors. The ping Data did come back from the shortest path >>> (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes >>> back from the other much longer paths. The problem is that Arizona seems >>> to remember the most recent successful paths instead of the previously >>> successful shorter paths. Next time when P2 came, it directly sent the >>> Interest to CSU instead of UCLA. >>> >>> Is this expected behavior of ncc? I want to emphasize that the shorter >>> path did return the same Ping data (and much earlier than the longer path). >>> >>> Lan >> >> > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lanwang at memphis.edu Wed Sep 10 09:17:50 2014 From: lanwang at memphis.edu (Lan Wang (lanwang)) Date: Wed, 10 Sep 2014 16:17:50 +0000 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Can you give a detailed explanation of the observed behavior and how/why the patch improved the results? Also why didn't Arizona switch back to the faster path when CSU came back? Lan On Sep 10, 2014, at 11:09 AM, Syed Obaid Amin > wrote: Hi Junxiao, Thanks for the fix. The patch improved the results significantly. For e.g., have a look at the following results: Old strategy: http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-oldncc_nolog/arizona-caida-converge.png Patched: http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-ncc_nolog/arizona-caida-converge.png The topology I used for these tests can be found here: http://netwisdom.cs.memphis.edu/imgs/topology.png Regards, Obaid On Mon, Sep 8, 2014 at 7:36 PM, Junxiao Shi > wrote: Hi Lan Bug 1961 has been fixed. Code is merged to master branch. Please test in your internal testbed. The correct behavior shall be: 1. P1 is forwarded to UCLA, and could be forwarded to CSU and other nexthops as well 2. UCLA responds first and remains as best face; other responses are received but ignored 3. P2 should be forwarded to UCLA This bug is not critical to end users. I don't plan to backport to v0.2. It won't be deployed on testbed routers until v0.3 release. Yours, Junxiao On Tue, Sep 2, 2014 at 10:26 AM, Junxiao Shi > wrote: Hi Lan Thanks for the report. This shall be a bug of NCC strategy. In ccnd, 1. When first Data comes back that satisfies the PIT entry, the incoming face of that Data is remembered. 2. The PIT entry is deleted as soon as it's satisfied. 3. When second Data comes back, it's discarded as unsolicited. In NFD, 1. When first Data comes back that satisfies the PIT entry, the incoming face of that Data is remembered. 2. The PIT entry is kept, and will be deleted after straggler timer. 3. When second Data comes back, the strategy is notified, and remembers the incoming face. NFD's NCC strategy incorrectly assumes the incoming Data is from the fastest upstream, causing the bug. I'll report the bug soon. Yours, Junxiao On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) > wrote: Junxiao, We observed something strange in one of our experiments when we used the ncc strategy (and link state routing). In some of the pings from Arizona to PKU, the ndnping delays were really large (>350ms) even though the shortest path delay is only around 200ms. We looked at the ndnping data, and found where the delay jumped from ~200ms to ~350ms. It turns out that instead of going through the shortest path (Arizona->UCLA->PKU), it is going through a much longer path (Arizona->CSU->?->NEU->PKU). To explain what happened, I'll call the ping before the change P1 and the ping after the change P2. Further looking into the nfd logs, it seems for P1 Arizona first sent the Interest to UCLA (which we expected), but while waiting for the data to come back, it also tried CSU and some other neighbors. The ping Data did come back from the shortest path (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes back from the other much longer paths. The problem is that Arizona seems to remember the most recent successful paths instead of the previously successful shorter paths. Next time when P2 came, it directly sent the Interest to CSU instead of UCLA. Is this expected behavior of ncc? I want to emphasize that the shorter path did return the same Ping data (and much earlier than the longer path). Lan _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.afanasyev at ucla.edu Wed Sep 10 09:51:16 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Wed, 10 Sep 2014 09:51:16 -0700 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: References: Message-ID: <4B6F267A-48D9-4E06-AECA-AF7CFF77F982@ucla.edu> On Sep 9, 2014, at 10:29 PM, Burke, Jeff wrote: > > One could imagine different configurable strategies for the MemoryContentCache depending on the way the application uses FreshnessPeriod. I'm not sure exactly the end goal of this discussion? > > I haven't been closely following all the messages but I think Dave Oran alluded to the fact that just having FreshnessPeriod may not be enough. This has come up in the past in other contexts and may be worth discussing. > > You state "My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store." > > This is fair, but imagine three different consumer applications for the same ndnrtc streams, all of which have different opinions about what content from a single publisher is relevant to be in a content store - > 1. "real-time" live-playback (current app) - only needs freshest data (within some buffer time period) > 2. Fixed ten-second expletive-removing delay consumer ? can have a fetching buffer that's significantly larger (though not the full seven seconds) than the above, but otherwise is "real-time" > 3. digital video recorder style playback ? may have much larger delays, and fetch content that may be minutes, days, hours older than the freshest content while other consumers are also watching the freshest content. > > Given these three consumers for a single publisher of live video, how should FreshnessPeriod be used to the best effect / appropriately? In my mind, the only purpose for FreshnessPeriod is to control on producer-side how often it wants to see request for "fresh" data. Also, consumers should send requests for "fresh" data only when they really need to get up to date knowledge about "current" data. For me, this implies that FreshnessPeriod (at least for some key frames or meta-data) should be set to something that producer thinks it can handle and it is an acceptable "real-time delay" (say 10-100ms). "Normal" data should be requested without any selectors set and it is perfectly acceptable for any content store to serve "stale" data. This applies to all content stores on the path and content store on a producer machine/in-app store. So, I don't see any complication in the above three cases. For the first two "real-time" cases, consumers would from time to time request fresh data to synchronize playback, but majority requests would need not to be related to freshness (no MustBeFresh selector). For the last case, none of the interests should have the selector and FreshnessPeriod is completely irrelevant for those. --- Alex > J. > > From: Yingdi Yu > Date: Mon, 8 Sep 2014 11:29:56 -0700 > To: Jeff Thompson > Cc: "" , "ndn-app at lists.cs.ucla.edu" > Subject: Re: [Nfd-dev] Close NFD backdoor > >> Hi JeffT, >> >> On Sep 8, 2014, at 2:53 AM, Thompson, Jeff wrote: >> >>> The NDN-RTC video conferencing app uses the MemoryContentCache in-app storage. It keeps data in the cache based on the data packet's FreshnessPeriod. >>> http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html >>> >>> You say "content store is aware of FreshnessPeriod of data packets, but an in-app storage should not." Why should the in-app storage not be aware of the FreshnessPeriod? >> >> In the NDN-TLV spec, the definition of FreshnessPeriod says "Note that the stale data is still valid data; the expiration of FreshnessPeriod only means that the producer may have produced newer data." My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store. The end producer is the only one who knows and can decide whether a data packet is valid or not. And the in-memory storage contains data packets that are treated as valid by end producer. >> >> For example, producer A generates a data packet "/a/b/c" with 10-second FreshnessPeriod. If "/a/b/c" is put into the MemoryContentCache, the packet will be removed from the cache after 10 seconds, and the producer A will have to generate the same data packet again even if the producer A still takes "/a/b/c" as valid. With the in-memory storage, the packet will never be removed until the producer generates a new valid data packet to replace "/a/b/c". And "/a/b/c" will be obsolete by the new data packet. >> >> Yingdi >> >> _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Wed Sep 10 11:24:13 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Wed, 10 Sep 2014 11:24:13 -0700 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Hi Lan The fix for Bug 1961 improve the results because it ensures an upstream who isn't first to respond won't be incorrectly recorded as best face. The patched figure shows two links are used. ARIZONA-CSU-CAIDA has RTT 72ms; ARIZONA-MEMPHIS-CAIDA has RTT 156ms. CSU node fails at message 60, and recovers at message 180. The experiment ends at message 300. *NCC strategy fails to switch back to CSU after its recovery. This is either a weakness of ccnd strategy, or a misunderstand when it's ported to NFD.* This is not an experiment problem, and cannot be solved by running experiment for longer time. Detail analysis follows. NCC strategy maintains an RTT prediction for the best face of each Name prefix. The granularity is at Interest Name minus the last component (usually a segment number). - RTT prediction starts at 8.192ms. - If Data comes within prediction, prediction is adjusted down by 1/128, and minimum is 0.127ms. - If Data doesn't come within prediction, prediction is adjusted up by 1/8, and maximum is 160ms. - The effect of above two point is: ultimately, prediction oscillates around real RTT (if real RTT is between 0.127ms and 160ms). - If Data doesn't come within prediction, other faces will be tried, starting from the end of original prediction. The previous best face, along with one other face who has lowest routing cost, are tried first. Other faces are tried one by one, in a short period of time. Two important points are: - All except the best face are propagated to, after the prediction. This means, the Interest is forwarded to CSU 154ms (=156*127/128) later than forwarding to MEMPHIS. - Incoming face of the first Data that satisfies the Interest becomes the best face. If the MEMPHIS link does not fail, the Data would always arrive first, because the Interest is forwarded 154ms earlier. Therefore, *CSU link can never become the best face, unless MEMPHIS link fails, or until the traffic stops so that measurements are wiped out*. This is confirmed by my calculation in Excel. Questions: - Does the routing protocol uninstall and re-install the Route during this experiment? If yes, please give the time point at which the FIB update is applied (a time point shall be given as "just before sending Interest for message X" so that I can correlate to messages). - If you have results of a similar experiment with ccnd or ndnd, please share them, so that I can confirm whether it's a design weakness or implementation bug. Yours, Junxiao On Wed, Sep 10, 2014 at 9:17 AM, Lan Wang (lanwang) wrote: > Can you give a detailed explanation of the observed behavior and how/why > the patch improved the results? > > Also why didn't Arizona switch back to the faster path when CSU came > back? > > Lan > On Sep 10, 2014, at 11:09 AM, Syed Obaid Amin > wrote: > > Hi Junxiao, > > Thanks for the fix. The patch improved the results significantly. For > e.g., have a look at the following results: > > Old strategy: > > http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-oldncc_nolog/arizona-caida-converge.png > > Patched: > > http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-ncc_nolog/arizona-caida-converge.png > > The topology I used for these tests can be found here: > http://netwisdom.cs.memphis.edu/imgs/topology.png > > Regards, > Obaid > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From obaidasyed at gmail.com Wed Sep 10 11:24:37 2014 From: obaidasyed at gmail.com (Syed Obaid Amin) Date: Wed, 10 Sep 2014 13:24:37 -0500 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: I cannot say if the patch fixed the Bug 1961 completely, as I don't have detailed nfd logs with me and without them I cannot trace packets. But it does improve the results, in terms of RTT. I am going to run the experiment with DEBUG log level now and will update you about it. @Junxiao, I've a few questions regarding NCC can you please explain? 1. The result of old ncc kind of contradicts what you explained earlier. If an RTT of duplicate data is not getting discarded in old ncc, then why packet coming via Memphis didn't update the face priorities. What it means that after bringing down CSU node, the RTT shouldn't stay to 350ms for more than 2 mins? 2. Did the patch fixed other issues too? As now with the new patch the strategy doesn't use the best path once it is back again? 3. How face priorities are updated, does the strategy update the the priorities as soon as the Data packet arrives or it waits for sometime, if so for how long it waits? 4. Is there any refresh period for reprioritizing the faces? Thanks, Obaid On Wed, Sep 10, 2014 at 11:17 AM, Lan Wang (lanwang) wrote: > Can you give a detailed explanation of the observed behavior and how/why > the patch improved the results? > > Also why didn't Arizona switch back to the faster path when CSU came > back? > > Lan > On Sep 10, 2014, at 11:09 AM, Syed Obaid Amin > wrote: > > Hi Junxiao, > > Thanks for the fix. The patch improved the results significantly. For > e.g., have a look at the following results: > > Old strategy: > > http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-oldncc_nolog/arizona-caida-converge.png > > Patched: > > http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-ncc_nolog/arizona-caida-converge.png > > The topology I used for these tests can be found here: > http://netwisdom.cs.memphis.edu/imgs/topology.png > > Regards, > Obaid > > > > On Mon, Sep 8, 2014 at 7:36 PM, Junxiao Shi > wrote: > >> Hi Lan >> >> Bug 1961 has been fixed. Code is merged to master branch. >> Please test in your internal testbed. >> >> The correct behavior shall be: >> >> 1. P1 is forwarded to UCLA, and could be forwarded to CSU and other >> nexthops as well >> 2. UCLA responds first and remains as best face; other responses are >> received but ignored >> 3. P2 should be forwarded to UCLA >> >> >> This bug is not critical to end users. I don't plan to backport to >> v0.2. It won't be deployed on testbed routers until v0.3 release. >> >> Yours, Junxiao >> >> >> On Tue, Sep 2, 2014 at 10:26 AM, Junxiao Shi < >> shijunxiao at email.arizona.edu> wrote: >> >>> Hi Lan >>> >>> Thanks for the report. This shall be a bug of NCC strategy. >>> >>> In ccnd, >>> >>> 1. When first Data comes back that satisfies the PIT entry, the >>> incoming face of that Data is remembered. >>> 2. The PIT entry is deleted as soon as it's satisfied. >>> 3. When second Data comes back, it's discarded as unsolicited. >>> >>> In NFD, >>> >>> 1. When first Data comes back that satisfies the PIT entry, the >>> incoming face of that Data is remembered. >>> 2. The PIT entry is kept, and will be deleted after straggler timer. >>> 3. When second Data comes back, the strategy is notified, and >>> remembers the incoming face. >>> >>> NFD's NCC strategy incorrectly assumes the incoming Data is from the >>> fastest upstream, causing the bug. >>> >>> I'll report the bug soon. >>> >>> Yours, Junxiao >>> >>> >>> On Tue, Sep 2, 2014 at 10:03 AM, Lan Wang (lanwang) >> > wrote: >>> >>>> Junxiao, >>>> >>>> We observed something strange in one of our experiments when we used >>>> the ncc strategy (and link state routing). In some of the pings from >>>> Arizona to PKU, the ndnping delays were really large (>350ms) even though >>>> the shortest path delay is only around 200ms. We looked at the ndnping >>>> data, and found where the delay jumped from ~200ms to ~350ms. It turns out >>>> that instead of going through the shortest path (Arizona->UCLA->PKU), it is >>>> going through a much longer path (Arizona->CSU->?->NEU->PKU). >>>> >>>> To explain what happened, I'll call the ping before the change P1 and >>>> the ping after the change P2. Further looking into the nfd logs, it seems >>>> for P1 Arizona first sent the Interest to UCLA (which we expected), but >>>> while waiting for the data to come back, it also tried CSU and some other >>>> neighbors. The ping Data did come back from the shortest path >>>> (PKU->UCLA->Arizona) as expected. But afterwards, the ping Data also comes >>>> back from the other much longer paths. The problem is that Arizona seems >>>> to remember the most recent successful paths instead of the previously >>>> successful shorter paths. Next time when P2 came, it directly sent the >>>> Interest to CSU instead of UCLA. >>>> >>>> Is this expected behavior of ncc? I want to emphasize that the shorter >>>> path did return the same Ping data (and much earlier than the longer path). >>>> >>>> Lan >>> >>> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From obaidasyed at gmail.com Wed Sep 10 12:19:13 2014 From: obaidasyed at gmail.com (Syed Obaid Amin) Date: Wed, 10 Sep 2014 14:19:13 -0500 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: On Wed, Sep 10, 2014 at 1:24 PM, Junxiao Shi wrote: > Hi Lan > > The fix for Bug 1961 improve the results because it ensures an upstream > who isn't first to respond won't be incorrectly recorded as best face. > > The patched figure shows two links are used. ARIZONA-CSU-CAIDA has RTT > 72ms; ARIZONA-MEMPHIS-CAIDA has RTT 156ms. > CSU node fails at message 60, and recovers at message 180. > The experiment ends at message 300. > > *NCC strategy fails to switch back to CSU after its recovery. This is > either a weakness of ccnd strategy, or a misunderstand when it's ported to > NFD.* This is not an experiment problem, and cannot be solved by running > experiment for longer time. > Detail analysis follows. > > NCC strategy maintains an RTT prediction for the best face of each Name > prefix. The granularity is at Interest Name minus the last component > (usually a segment number). > > - RTT prediction starts at 8.192ms. > - If Data comes within prediction, prediction is adjusted down by > 1/128, and minimum is 0.127ms. > - If Data doesn't come within prediction, prediction is adjusted up by > 1/8, and maximum is 160ms. > - The effect of above two point is: ultimately, prediction oscillates > around real RTT (if real RTT is between 0.127ms and 160ms). > - If Data doesn't come within prediction, other faces will be tried, > starting from the end of original prediction. > The previous best face, along with one other face who has lowest > routing cost, are tried first. > Other faces are tried one by one, in a short period of time. > > Two important points are: > > - All except the best face are propagated to, after the prediction. > This means, the Interest is forwarded to CSU 154ms (=156*127/128) later > than forwarding to MEMPHIS. > - Incoming face of the first Data that satisfies the Interest becomes > the best face. If the MEMPHIS link does not fail, the Data would always > arrive first, because the Interest is forwarded 154ms earlier. Therefore, *CSU > link can never become the best face, unless MEMPHIS link fails, or until > the traffic stops so that measurements are wiped out*. This is > confirmed by my calculation in Excel. > > Questions: > > - Does the routing protocol uninstall and re-install the Route during > this experiment? If yes, please give the time point at which the FIB update > is applied (a time point shall be given as "just before sending Interest > for message X" so that I can correlate to messages). > > Not sure what do you mean here. At 60th sec, the CSU node was brought down, as a result all other nodes update their routing tables. But we don't have any timestamps as the logging was not on. > > - If you have results of a similar experiment with ccnd or ndnd, > please share them, so that I can confirm whether it's a design weakness or > implementation bug. > > I've data from ccnx, but that experiment ran for 180 secs only. I am attaching here the file. If seems useful I can re-run the experiments with ccnx. But it may take little time for exact setup. Regards, Obaid > > - > > > Yours, Junxiao > > On Wed, Sep 10, 2014 at 9:17 AM, Lan Wang (lanwang) > wrote: > >> Can you give a detailed explanation of the observed behavior and how/why >> the patch improved the results? >> >> Also why didn't Arizona switch back to the faster path when CSU came >> back? >> >> Lan >> On Sep 10, 2014, at 11:09 AM, Syed Obaid Amin >> wrote: >> >> Hi Junxiao, >> >> Thanks for the fix. The patch improved the results significantly. For >> e.g., have a look at the following results: >> >> Old strategy: >> >> http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-oldncc_nolog/arizona-caida-converge.png >> >> Patched: >> >> http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-ncc_nolog/arizona-caida-converge.png >> >> The topology I used for these tests can be found here: >> http://netwisdom.cs.memphis.edu/imgs/topology.png >> >> Regards, >> Obaid >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: arizona-caida-converge.png Type: image/png Size: 28193 bytes Desc: not available URL: From shijunxiao at email.arizona.edu Wed Sep 10 13:29:28 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Wed, 10 Sep 2014 13:29:28 -0700 Subject: [Nfd-dev] ncc strategy In-Reply-To: References: Message-ID: Hi Obaid The experiment results from ccnd shows that ccnd is also unable to detect the recovery of the best path. It would stick with the second path until it fails, or until traffic stops so measurements are gone. Therefore, it's a weakness of ccnd strategy, and not a bug of NCC strategy. I'll certain consider this scenario when designing a new strategy for routers (Task 1901). Yours, Junxiao On Wed, Sep 10, 2014 at 12:19 PM, Syed Obaid Amin wrote: > > > On Wed, Sep 10, 2014 at 1:24 PM, Junxiao Shi > wrote: > >> Hi Lan >> >> The fix for Bug 1961 improve the results because it ensures an upstream >> who isn't first to respond won't be incorrectly recorded as best face. >> >> The patched figure shows two links are used. ARIZONA-CSU-CAIDA has RTT >> 72ms; ARIZONA-MEMPHIS-CAIDA has RTT 156ms. >> CSU node fails at message 60, and recovers at message 180. >> The experiment ends at message 300. >> >> *NCC strategy fails to switch back to CSU after its recovery. This is >> either a weakness of ccnd strategy, or a misunderstand when it's ported to >> NFD.* This is not an experiment problem, and cannot be solved by running >> experiment for longer time. >> Detail analysis follows. >> >> NCC strategy maintains an RTT prediction for the best face of each Name >> prefix. The granularity is at Interest Name minus the last component >> (usually a segment number). >> >> - RTT prediction starts at 8.192ms. >> - If Data comes within prediction, prediction is adjusted down by >> 1/128, and minimum is 0.127ms. >> - If Data doesn't come within prediction, prediction is adjusted up >> by 1/8, and maximum is 160ms. >> - The effect of above two point is: ultimately, prediction oscillates >> around real RTT (if real RTT is between 0.127ms and 160ms). >> - If Data doesn't come within prediction, other faces will be tried, >> starting from the end of original prediction. >> The previous best face, along with one other face who has lowest >> routing cost, are tried first. >> Other faces are tried one by one, in a short period of time. >> >> Two important points are: >> >> - All except the best face are propagated to, after the prediction. >> This means, the Interest is forwarded to CSU 154ms (=156*127/128) later >> than forwarding to MEMPHIS. >> - Incoming face of the first Data that satisfies the Interest becomes >> the best face. If the MEMPHIS link does not fail, the Data would always >> arrive first, because the Interest is forwarded 154ms earlier. Therefore, *CSU >> link can never become the best face, unless MEMPHIS link fails, or until >> the traffic stops so that measurements are wiped out*. This is >> confirmed by my calculation in Excel. >> >> Questions: >> >> - Does the routing protocol uninstall and re-install the Route during >> this experiment? If yes, please give the time point at which the FIB update >> is applied (a time point shall be given as "just before sending Interest >> for message X" so that I can correlate to messages). >> >> Not sure what do you mean here. At 60th sec, the CSU node was brought > down, as a result all other nodes update their routing tables. But we don't > have any timestamps as the logging was not on. > > >> >> - If you have results of a similar experiment with ccnd or ndnd, >> please share them, so that I can confirm whether it's a design weakness or >> implementation bug. >> >> I've data from ccnx, but that experiment ran for 180 secs only. I am > attaching here the file. If seems useful I can re-run the experiments with > ccnx. But it may take little time for exact setup. > > Regards, > Obaid > > >> >> - >> >> >> Yours, Junxiao >> >> On Wed, Sep 10, 2014 at 9:17 AM, Lan Wang (lanwang) >> wrote: >> >>> Can you give a detailed explanation of the observed behavior and >>> how/why the patch improved the results? >>> >>> Also why didn't Arizona switch back to the faster path when CSU came >>> back? >>> >>> Lan >>> On Sep 10, 2014, at 11:09 AM, Syed Obaid Amin >>> wrote: >>> >>> Hi Junxiao, >>> >>> Thanks for the fix. The patch improved the results significantly. For >>> e.g., have a look at the following results: >>> >>> Old strategy: >>> >>> http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-oldncc_nolog/arizona-caida-converge.png >>> >>> Patched: >>> >>> http://netwisdom.cs.memphis.edu/hr_evaluation/tests-withfailure-ncc_nolog/arizona-caida-converge.png >>> >>> The topology I used for these tests can be found here: >>> http://netwisdom.cs.memphis.edu/imgs/topology.png >>> >>> Regards, >>> Obaid >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jburke at remap.UCLA.EDU Wed Sep 10 15:48:10 2014 From: jburke at remap.UCLA.EDU (Burke, Jeff) Date: Wed, 10 Sep 2014 22:48:10 +0000 Subject: [Nfd-dev] Close NFD backdoor In-Reply-To: <4B6F267A-48D9-4E06-AECA-AF7CFF77F982@ucla.edu> Message-ID: From: Alex Afanasyev > Date: Wed, 10 Sep 2014 09:51:16 -0700 To: Jeff Burke > Cc: Yingdi Yu >, Jeff Thompson >, ">" >, "ndn-app at lists.cs.ucla.edu" > Subject: Re: [Nfd-dev] Close NFD backdoor On Sep 9, 2014, at 10:29 PM, Burke, Jeff > wrote: One could imagine different configurable strategies for the MemoryContentCache depending on the way the application uses FreshnessPeriod. I'm not sure exactly the end goal of this discussion? I haven't been closely following all the messages but I think Dave Oran alluded to the fact that just having FreshnessPeriod may not be enough. This has come up in the past in other contexts and may be worth discussing. You state "My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store." This is fair, but imagine three different consumer applications for the same ndnrtc streams, all of which have different opinions about what content from a single publisher is relevant to be in a content store - 1. "real-time" live-playback (current app) - only needs freshest data (within some buffer time period) 2. Fixed ten-second expletive-removing delay consumer ? can have a fetching buffer that's significantly larger (though not the full seven seconds) than the above, but otherwise is "real-time" 3. digital video recorder style playback ? may have much larger delays, and fetch content that may be minutes, days, hours older than the freshest content while other consumers are also watching the freshest content. Given these three consumers for a single publisher of live video, how should FreshnessPeriod be used to the best effect / appropriately? In my mind, the only purpose for FreshnessPeriod is to control on producer-side how often it wants to see request for "fresh" data. This is fair enough, but it's not what (I thought) Yingdi said, "FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store." Also, consumers should send requests for "fresh" data only when they really need to get up to date knowledge about "current" data. Makes sense, but this definition of what "fresh" means needs to be made a little clearer. What makes something current? For me, this implies that FreshnessPeriod (at least for some key frames or meta-data) should be set to something that producer thinks it can handle and it is an acceptable "real-time delay" (say 10-100ms). "Normal" data should be requested without any selectors set and it is perfectly acceptable for any content store to serve "stale" data. This applies to all content stores on the path and content store on a producer machine/in-app store. Yes ? same question as above ? Yingdi has a different interpretation. So, I don't see any complication in the above three cases. For the first two "real-time" cases, consumers would from time to time request fresh data to synchronize playback, but majority requests would need not to be related to freshness (no MustBeFresh selector). For the last case, none of the interests should have the selector and FreshnessPeriod is completely irrelevant for those. Ok. But that relies on content stores ignoring (or at least not only obeying) the FreshnessPeriod, right? Jeff --- Alex J. From: Yingdi Yu > Date: Mon, 8 Sep 2014 11:29:56 -0700 To: Jeff Thompson > Cc: ">" >, "ndn-app at lists.cs.ucla.edu" > Subject: Re: [Nfd-dev] Close NFD backdoor Hi JeffT, On Sep 8, 2014, at 2:53 AM, Thompson, Jeff > wrote: The NDN-RTC video conferencing app uses the MemoryContentCache in-app storage. It keeps data in the cache based on the data packet's FreshnessPeriod. http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html You say "content store is aware of FreshnessPeriod of data packets, but an in-app storage should not." Why should the in-app storage not be aware of the FreshnessPeriod? In the NDN-TLV spec, the definition of FreshnessPeriod says "Note that the stale data is still valid data; the expiration of FreshnessPeriod only means that the producer may have produced newer data." My understanding about this is that FreshnessPeriod is used by end producer to tell intermediate routers (including local nfd) how long a data packet should stay in a content store. The end producer is the only one who knows and can decide whether a data packet is valid or not. And the in-memory storage contains data packets that are treated as valid by end producer. For example, producer A generates a data packet "/a/b/c" with 10-second FreshnessPeriod. If "/a/b/c" is put into the MemoryContentCache, the packet will be removed from the cache after 10 seconds, and the producer A will have to generate the same data packet again even if the producer A still takes "/a/b/c" as valid. With the in-memory storage, the packet will never be removed until the producer generates a new valid data packet to replace "/a/b/c". And "/a/b/c" will be obsolete by the new data packet. Yingdi _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.afanasyev at ucla.edu Fri Sep 12 16:40:07 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Fri, 12 Sep 2014 16:40:07 -0700 Subject: [Nfd-dev] LINK spec discussion Message-ID: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Hi! I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). Currently we preliminary defined three elements: a) link for redirection: signA("nameA", content: "nameB") b) link for delegation signA("nameA", content: signB("nameB", content: "prefixOfNameA")) c) "link" for encapsulation sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). --- Alex From shijunxiao at email.arizona.edu Fri Sep 12 21:53:08 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Fri, 12 Sep 2014 21:53:08 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: Dear folks In the non-routable web hosting scenario, to achieve the same level of security as IP DNS, we only need redirection and encapsulation LINKs. /ndnsim doesn't need /att's permission to host the website under /att/alex-ndnsim. This is similar to IP DNS: to setup an A/AAAA record to that IP address, you don't need IP address owner's permission. The procedure of accessing /ndnsim contents is: 1. perform a NDNS lookup for /ndnsim, which would return a redirection LINK sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") 2. replace /ndnsim with /att/alex-ndnsim in all Interests - to access /ndnsim/index.html, express Interest /att/alex-ndnsim/index.html 3. some router near end host or the end host itself or the producer application, must replace /att/alex-ndnsim back to /ndnsim - to execute signed Interest /ndnsim/action.cgi/[sig], Interest Name across the network is /att/alex-ndnsim/action.cgi/[sig], but the signature is generated based on /ndnsim/action.cgi 4. Data is encapsulated in encapsulation LINK sha256(name=/att/alex-ndnsim/index.html/segment0, content={sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") | sign[ndnsim](name=/ndnsim/index.html/segment0, content=payload)}) If the owner of /ndnsim points the NDNS record to some bogus location such as /verizon/alex-ndnsim, those Interests would be dropped after reaching /verizon due to lack of Route. Adding delegation to the mix could prevent an attack in IP DNS. In IP DNS, if the owner of a large website points his A record to a smaller website, the large amount of traffic would cause denial of service in the smaller website, although doing so would make the large website itself inaccessible. (This happened once on BAIDU, a top search engine in China; an attacker hijacked BAIDU's Nameserver and pointed A record to a small server). If the owner of (large website) /ndnsim points the NDNS record to another user's namespace such as /verizon/john-victim (small website), it's the same attack as described above. Storing the delegation record instead of redirection record in NDNS would prevent this attack. A drawback of delegation record is: it promotes content control by ISP. In IP DNS, user has the freedom to host any website on the IP address (unless ISP deploys layer-7 filtering). If a delegation record is needed, ISP could refuse to sign a delegation record if they don't want user to host a certain website. Yours, Junxiao On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev < alexander.afanasyev at ucla.edu> wrote: > Hi! > > I want continue our discussion about LINK specification ( > named-data.net/downloads/memos/link.pdf). > > Currently we preliminary defined three elements: > > a) link for redirection: > > signA("nameA", content: "nameB") > > b) link for delegation > > signA("nameA", content: signB("nameB", content: "prefixOfNameA")) > > c) "link" for encapsulation > > sha256("nameB" | "nameA", content: ("delegation or redirection link", > data("nameA")) > > > The option (b) is the latest addition and I have some reservations about > it. In particular, what exactly this delegation achieves? What security > problem we are solving with it. > > Let me give some example of (b). I own a "non-routable" namespace /ndnsim > and my site is connected to /att and /ucla networks. So, the (b) option > would require: > > - I will have to obtain a "redirection" links from /att and /ucla to > /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim > site > > - Then I will create a delegation packet(s) that I can put in NDNS to tell > others that my site is currently available through /att and /ucla. > > The process looks ok, but the question I have is what significance is in > the first permission. What is the point of me of requesting permission > from my provider to host a website? What I would gain and what provider > itself would gain? > > I personally, do not yet see a clear benefit from this process, only that > it would complicate the delegation process. Only me is able to tell others > that my site is available at different places, so there is no question > about others claiming that my site is available somewhere else. If I > mistakenly put a wrong link, then I myself will be at loss: i wouldn't be > able to serve my content and somebody else would receive my interests and > could reply to them with some junk. > > If I "maliciously" put somebody else's network as a link, then interests > for my data would go there. But what is the harm? If nobody replies to > them, then routers can start ignoring/pushing back such interests. If > somebody replying, then somebody serves data and from the network point of > view nothing bad is happening. > > > In short, I want us have a deep discussion on what exactly we are > protecting and from whom. For me, even with only one signature (option a), > only the owner of the original namespace (/ndnsim) is able to create a > legitimate link to somewhere else. Additional signature could protect > "provider" (destination of the link), but is it really a problem? Anybody > could send any number of interests to that provider even without any link > (erroneous or malicious). > > > --- > Alex > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wentaoshang at gmail.com Sun Sep 14 14:55:15 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Sun, 14 Sep 2014 14:55:15 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: I was involved in the previous discussion and I have a quick question: the document says the LINK type will be a "sub-type of NDN DATA packet". Is the "sub-type" referring to implementation mechanism (i.e., Link class is inherited from Data class) or do we actually have some way to encode the "sub-type" relationship in the TLV format? I remember the type field is using flat number so not sure whether the latter is possible. If it is the former case, I don't think specifying implementation detail in the document is a good idea (what if the programming language doesn't support inheritance?). Wentao On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev < alexander.afanasyev at ucla.edu> wrote: > Hi! > > I want continue our discussion about LINK specification ( > named-data.net/downloads/memos/link.pdf). > > Currently we preliminary defined three elements: > > a) link for redirection: > > signA("nameA", content: "nameB") > > b) link for delegation > > signA("nameA", content: signB("nameB", content: "prefixOfNameA")) > > c) "link" for encapsulation > > sha256("nameB" | "nameA", content: ("delegation or redirection link", > data("nameA")) > > > The option (b) is the latest addition and I have some reservations about > it. In particular, what exactly this delegation achieves? What security > problem we are solving with it. > > Let me give some example of (b). I own a "non-routable" namespace /ndnsim > and my site is connected to /att and /ucla networks. So, the (b) option > would require: > > - I will have to obtain a "redirection" links from /att and /ucla to > /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim > site > > - Then I will create a delegation packet(s) that I can put in NDNS to tell > others that my site is currently available through /att and /ucla. > > The process looks ok, but the question I have is what significance is in > the first permission. What is the point of me of requesting permission > from my provider to host a website? What I would gain and what provider > itself would gain? > > I personally, do not yet see a clear benefit from this process, only that > it would complicate the delegation process. Only me is able to tell others > that my site is available at different places, so there is no question > about others claiming that my site is available somewhere else. If I > mistakenly put a wrong link, then I myself will be at loss: i wouldn't be > able to serve my content and somebody else would receive my interests and > could reply to them with some junk. > > If I "maliciously" put somebody else's network as a link, then interests > for my data would go there. But what is the harm? If nobody replies to > them, then routers can start ignoring/pushing back such interests. If > somebody replying, then somebody serves data and from the network point of > view nothing bad is happening. > > > In short, I want us have a deep discussion on what exactly we are > protecting and from whom. For me, even with only one signature (option a), > only the owner of the original namespace (/ndnsim) is able to create a > legitimate link to somewhere else. Additional signature could protect > "provider" (destination of the link), but is it really a problem? Anybody > could send any number of interests to that provider even without any link > (erroneous or malicious). > > > --- > Alex > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From wentaoshang at gmail.com Sun Sep 14 14:56:05 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Sun, 14 Sep 2014 14:56:05 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: On Sun, Sep 14, 2014 at 2:55 PM, Wentao Shang wrote: > I was involved in the previous discussion and I have a quick question: > "was -> wasn't". Sorry about typo :P Wentao > the document says the LINK type will be a "sub-type of NDN DATA packet". > Is the "sub-type" referring to implementation mechanism (i.e., Link class > is inherited from Data class) or do we actually have some way to encode the > "sub-type" relationship in the TLV format? > > I remember the type field is using flat number so not sure whether the > latter is possible. If it is the former case, I don't think specifying > implementation detail in the document is a good idea (what if the > programming language doesn't support inheritance?). > > Wentao > > On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev < > alexander.afanasyev at ucla.edu> wrote: > >> Hi! >> >> I want continue our discussion about LINK specification ( >> named-data.net/downloads/memos/link.pdf). >> >> Currently we preliminary defined three elements: >> >> a) link for redirection: >> >> signA("nameA", content: "nameB") >> >> b) link for delegation >> >> signA("nameA", content: signB("nameB", content: "prefixOfNameA")) >> >> c) "link" for encapsulation >> >> sha256("nameB" | "nameA", content: ("delegation or redirection link", >> data("nameA")) >> >> >> The option (b) is the latest addition and I have some reservations about >> it. In particular, what exactly this delegation achieves? What security >> problem we are solving with it. >> >> Let me give some example of (b). I own a "non-routable" namespace >> /ndnsim and my site is connected to /att and /ucla networks. So, the (b) >> option would require: >> >> - I will have to obtain a "redirection" links from /att and /ucla to >> /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim >> site >> >> - Then I will create a delegation packet(s) that I can put in NDNS to >> tell others that my site is currently available through /att and /ucla. >> >> The process looks ok, but the question I have is what significance is in >> the first permission. What is the point of me of requesting permission >> from my provider to host a website? What I would gain and what provider >> itself would gain? >> >> I personally, do not yet see a clear benefit from this process, only that >> it would complicate the delegation process. Only me is able to tell others >> that my site is available at different places, so there is no question >> about others claiming that my site is available somewhere else. If I >> mistakenly put a wrong link, then I myself will be at loss: i wouldn't be >> able to serve my content and somebody else would receive my interests and >> could reply to them with some junk. >> >> If I "maliciously" put somebody else's network as a link, then interests >> for my data would go there. But what is the harm? If nobody replies to >> them, then routers can start ignoring/pushing back such interests. If >> somebody replying, then somebody serves data and from the network point of >> view nothing bad is happening. >> >> >> In short, I want us have a deep discussion on what exactly we are >> protecting and from whom. For me, even with only one signature (option a), >> only the owner of the original namespace (/ndnsim) is able to create a >> legitimate link to somewhere else. Additional signature could protect >> "provider" (destination of the link), but is it really a problem? Anybody >> could send any number of interests to that provider even without any link >> (erroneous or malicious). >> >> >> --- >> Alex >> >> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev >> > > > > -- > PhD @ IRL, CSD, UCLA > -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Sun Sep 14 15:06:14 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Sun, 14 Sep 2014 15:06:14 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: Hi Wentao "sub-type of Data packet" refers to a Data packet with a specific ContentType. http://named-data.net/doc/ndn-tlv/data.html#contenttype It's more accurate to say: We define LINK as a Data packet with LINK as its ContentType. Yours, Junxiao On Sun, Sep 14, 2014 at 2:55 PM, Wentao Shang wrote: > I wasn't involved in the previous discussion and I have a quick question: > the document says the LINK type will be a "sub-type of NDN DATA packet". Is > the "sub-type" referring to implementation mechanism (i.e., Link class is > inherited from Data class) or do we actually have some way to encode the > "sub-type" relationship in the TLV format? > > I remember the type field is using flat number so not sure whether the > latter is possible. If it is the former case, I don't think specifying > implementation detail in the document is a good idea (what if the > programming language doesn't support inheritance?). > > Wentao > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wentaoshang at gmail.com Sun Sep 14 15:09:23 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Sun, 14 Sep 2014 15:09:23 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: On Sun, Sep 14, 2014 at 3:06 PM, Junxiao Shi wrote: > Hi Wentao > > "sub-type of Data packet" refers to a Data packet with a specific > ContentType. > http://named-data.net/doc/ndn-tlv/data.html#contenttype > > It's more accurate to say: We define LINK as a Data packet with LINK as > its ContentType. > Yes, that's a better way to phrase it. Thanks for clarification. Wentao > > Yours, Junxiao > > On Sun, Sep 14, 2014 at 2:55 PM, Wentao Shang > wrote: > >> I wasn't involved in the previous discussion and I have a quick question: >> the document says the LINK type will be a "sub-type of NDN DATA packet". Is >> the "sub-type" referring to implementation mechanism (i.e., Link class is >> inherited from Data class) or do we actually have some way to encode the >> "sub-type" relationship in the TLV format? >> >> I remember the type field is using flat number so not sure whether the >> latter is possible. If it is the former case, I don't think specifying >> implementation detail in the document is a good idea (what if the >> programming language doesn't support inheritance?). >> >> Wentao >> >> -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.afanasyev at ucla.edu Sun Sep 14 15:11:02 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Sun, 14 Sep 2014 15:11:02 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: > On Sep 14, 2014, at 2:55 PM, Wentao Shang wrote: > > I was involved in the previous discussion and I have a quick question: the document says the LINK type will be a "sub-type of NDN DATA packet". Is the "sub-type" referring to implementation mechanism (i.e., Link class is inherited from Data class) or do we actually have some way to encode the "sub-type" relationship in the TLV format? > > I remember the type field is using flat number so not sure whether the latter is possible. If it is the former case, I don't think specifying implementation detail in the document is a good idea (what if the programming language doesn't support inheritance?). We should have been more specific. This "subtype" has nothing to do with inheritance or any implementation details. It is just a "type" of data packet as specified in MetaInfo part of the Data packet itself, nothing more. By default (type = 0) is generic "blob", other types may determine the specific common encoding of the content part and the link spec is just defining a couple of new types/content part encodings. --- Alex > Wentao > >> On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev wrote: >> Hi! >> >> I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). >> >> Currently we preliminary defined three elements: >> >> a) link for redirection: >> >> signA("nameA", content: "nameB") >> >> b) link for delegation >> >> signA("nameA", content: signB("nameB", content: "prefixOfNameA")) >> >> c) "link" for encapsulation >> >> sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) >> >> >> The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. >> >> Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: >> >> - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site >> >> - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. >> >> The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? >> >> I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. >> >> If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. >> >> >> In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). >> >> >> --- >> Alex >> >> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > > > -- > PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Sun Sep 14 15:11:25 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Sun, 14 Sep 2014 15:11:25 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: <76421C1B-B66D-460F-BD67-5EAD0DC47A84@cs.ucla.edu> On Sep 12, 2014, at 9:53 PM, Junxiao Shi wrote: > Dear folks > > In the non-routable web hosting scenario, to achieve the same level of security as IP DNS, we only need redirection and encapsulation LINKs. > /ndnsim doesn't need /att's permission to host the website under /att/alex-ndnsim. This is similar to IP DNS: to setup an A/AAAA record to that IP address, you don't need IP address owner's permission. > > The procedure of accessing /ndnsim contents is: > perform a NDNS lookup for /ndnsim, which would return a redirection LINK > sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") > replace /ndnsim with /att/alex-ndnsim in all Interests > to access /ndnsim/index.html, express Interest /att/alex-ndnsim/index.html I wonder what is the benefit of replacing some part of the non-routable name, instead of simply concatenating the routable-prefix and non-routable name? > some router near end host or the end host itself or the producer application, must replace /att/alex-ndnsim back to /ndnsim > to execute signed Interest /ndnsim/action.cgi/[sig], Interest Name across the network is /att/alex-ndnsim/action.cgi/[sig], but the signature is generated based on /ndnsim/action.cgi It seems that replacement make the router's life harder, because the router cannot do the replacement without knowing the Link object. > Adding delegation to the mix could prevent an attack in IP DNS. > In IP DNS, if the owner of a large website points his A record to a smaller website, the large amount of traffic would cause denial of service in the smaller website, although doing so would make the large website itself inaccessible. (This happened once on BAIDU, a top search engine in China; an attacker hijacked BAIDU's Nameserver and pointed A record to a small server). This attack may happen because DNSSEC is not enable. NDNS records are signed, and resolvers are required to validate that, so the worst case is that no one can do DNS resolution for Baidu, but the small web site should be fine. > If the owner of (large website) /ndnsim points the NDNS record to another user's namespace such as /verizon/john-victim (small website), it's the same attack as described above. First, I don't think any descent big website intends to do that. If they really want to attack some small websites, there are a lot of other easy ways to do so instead of sacrificing its own service. Second, if any attacker want to do that, they have to accumulate a lot of interests, which is not trivial at the first place. > Storing the delegation record instead of redirection record in NDNS would prevent this attack. As you mentioned before, if there is not route for those "mis-redirected" interests, the interests will be dropped. Doesn't this prevent the attack? > A drawback of delegation record is: it promotes content control by ISP. > In IP DNS, user has the freedom to host any website on the IP address (unless ISP deploys layer-7 filtering). > If a delegation record is needed, ISP could refuse to sign a delegation record if they don't want user to host a certain website. Agree. > > Yours, Junxiao > > On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev wrote: > Hi! > > I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). > > Currently we preliminary defined three elements: > > a) link for redirection: > > signA("nameA", content: "nameB") > > b) link for delegation > > signA("nameA", content: signB("nameB", content: "prefixOfNameA")) > > c) "link" for encapsulation > > sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) > > > The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. > > Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: > > - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site > > - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. > > The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? > > I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. > > If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. > > > In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). > > > --- > Alex > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From lanwang at memphis.edu Sun Sep 14 15:32:08 2014 From: lanwang at memphis.edu (Lan Wang (lanwang)) Date: Sun, 14 Sep 2014 22:32:08 +0000 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: <9B3794D1-C8FD-49D7-AF85-4F17FC5C7F34@memphis.edu> An alternative failure scenario which benefits from delegation is that the owner of /ndnsim misconfigured its link and put another ISP's name (maybe it used to be the provider) there or just forgot to remove the ISP's name when it switches to a new ISP. If we require that ISP explicitly delegates its subprefix to link to /ndnsim, then Interests won't flood to the wrong ISP, which is undesirable for both the ISP and /ndnsim. Lan On Sep 12, 2014, at 11:53 PM, Junxiao Shi > wrote: Dear folks In the non-routable web hosting scenario, to achieve the same level of security as IP DNS, we only need redirection and encapsulation LINKs. /ndnsim doesn't need /att's permission to host the website under /att/alex-ndnsim. This is similar to IP DNS: to setup an A/AAAA record to that IP address, you don't need IP address owner's permission. The procedure of accessing /ndnsim contents is: 1. perform a NDNS lookup for /ndnsim, which would return a redirection LINK sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") 2. replace /ndnsim with /att/alex-ndnsim in all Interests * to access /ndnsim/index.html, express Interest /att/alex-ndnsim/index.html 3. some router near end host or the end host itself or the producer application, must replace /att/alex-ndnsim back to /ndnsim * to execute signed Interest /ndnsim/action.cgi/[sig], Interest Name across the network is /att/alex-ndnsim/action.cgi/[sig], but the signature is generated based on /ndnsim/action.cgi 4. Data is encapsulated in encapsulation LINK sha256(name=/att/alex-ndnsim/index.html/segment0, content={sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") | sign[ndnsim](name=/ndnsim/index.html/segment0, content=payload)}) If the owner of /ndnsim points the NDNS record to some bogus location such as /verizon/alex-ndnsim, those Interests would be dropped after reaching /verizon due to lack of Route. Adding delegation to the mix could prevent an attack in IP DNS. In IP DNS, if the owner of a large website points his A record to a smaller website, the large amount of traffic would cause denial of service in the smaller website, although doing so would make the large website itself inaccessible. (This happened once on BAIDU, a top search engine in China; an attacker hijacked BAIDU's Nameserver and pointed A record to a small server). If the owner of (large website) /ndnsim points the NDNS record to another user's namespace such as /verizon/john-victim (small website), it's the same attack as described above. Storing the delegation record instead of redirection record in NDNS would prevent this attack. A drawback of delegation record is: it promotes content control by ISP. In IP DNS, user has the freedom to host any website on the IP address (unless ISP deploys layer-7 filtering). If a delegation record is needed, ISP could refuse to sign a delegation record if they don't want user to host a certain website. Yours, Junxiao On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev > wrote: Hi! I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). Currently we preliminary defined three elements: a) link for redirection: signA("nameA", content: "nameB") b) link for delegation signA("nameA", content: signB("nameB", content: "prefixOfNameA")) c) "link" for encapsulation sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). --- Alex _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev _______________________________________________ Nfd-dev mailing list Nfd-dev at lists.cs.ucla.edu http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Sun Sep 14 15:37:47 2014 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Sun, 14 Sep 2014 22:37:47 +0000 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: Hi, Maybe I am coming into the middle of another discussion, but I still have yet to understand the status (and the choice of name) for NDNS. Is this new component now a "given" in the architecture? Or, is it an example of where LINKs may come from? Junxiao uses the phrase "perform a NDNS lookup for /ndnsim" - is this different than just issuing an Interest for "/ndnsim"? Jeff On 9/12/14, 4:40 PM, "Alex Afanasyev" wrote: >Hi! > >I want continue our discussion about LINK specification >(named-data.net/downloads/memos/link.pdf). > >Currently we preliminary defined three elements: > >a) link for redirection: > > signA("nameA", content: "nameB") > >b) link for delegation > > signA("nameA", content: signB("nameB", content: "prefixOfNameA")) > >c) "link" for encapsulation > > sha256("nameB" | "nameA", content: ("delegation or redirection link", >data("nameA")) > > >The option (b) is the latest addition and I have some reservations about >it. In particular, what exactly this delegation achieves? What security >problem we are solving with it. > >Let me give some example of (b). I own a "non-routable" namespace >/ndnsim and my site is connected to /att and /ucla networks. So, the (b) >option would require: > >- I will have to obtain a "redirection" links from /att and /ucla to >/ndnsim (option (a)) = get a permission from /att and /ucla to host >/ndnsim site > >- Then I will create a delegation packet(s) that I can put in NDNS to >tell others that my site is currently available through /att and /ucla. > >The process looks ok, but the question I have is what significance is in >the first permission. What is the point of me of requesting permission >from my provider to host a website? What I would gain and what provider >itself would gain? > >I personally, do not yet see a clear benefit from this process, only that >it would complicate the delegation process. Only me is able to tell >others that my site is available at different places, so there is no >question about others claiming that my site is available somewhere else. > If I mistakenly put a wrong link, then I myself will be at loss: i >wouldn't be able to serve my content and somebody else would receive my >interests and could reply to them with some junk. > >If I "maliciously" put somebody else's network as a link, then interests >for my data would go there. But what is the harm? If nobody replies to >them, then routers can start ignoring/pushing back such interests. If >somebody replying, then somebody serves data and from the network point >of view nothing bad is happening. > > >In short, I want us have a deep discussion on what exactly we are >protecting and from whom. For me, even with only one signature (option >a), only the owner of the original namespace (/ndnsim) is able to create >a legitimate link to somewhere else. Additional signature could protect >"provider" (destination of the link), but is it really a problem? >Anybody could send any number of interests to that provider even without >any link (erroneous or malicious). > > >--- >Alex > > > >_______________________________________________ >Nfd-dev mailing list >Nfd-dev at lists.cs.ucla.edu >http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev From wentaoshang at gmail.com Sun Sep 14 15:38:44 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Sun, 14 Sep 2014 15:38:44 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <76421C1B-B66D-460F-BD67-5EAD0DC47A84@cs.ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <76421C1B-B66D-460F-BD67-5EAD0DC47A84@cs.ucla.edu> Message-ID: On Sun, Sep 14, 2014 at 3:11 PM, Yingdi Yu wrote: > > On Sep 12, 2014, at 9:53 PM, Junxiao Shi > wrote: > > Dear folks > > In the non-routable web hosting scenario, to achieve the same level of > security as IP DNS, we only need redirection and encapsulation LINKs. > /ndnsim doesn't need /att's permission to host the website under > /att/alex-ndnsim. This is similar to IP DNS: to setup an A/AAAA record to > that IP address, you don't need IP address owner's permission. > > The procedure of accessing /ndnsim contents is: > > 1. perform a NDNS lookup for /ndnsim, which would return a redirection > LINK > sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") > 2. replace /ndnsim with /att/alex-ndnsim in all Interests > - to access /ndnsim/index.html, express Interest > /att/alex-ndnsim/index.html > > I wonder what is the benefit of replacing some part of the non-routable > name, instead of simply concatenating the routable-prefix and non-routable > name? > > > 1. some router near end host or the end host itself or the producer > application, must replace /att/alex-ndnsim back to /ndnsim > - to execute signed Interest /ndnsim/action.cgi/[sig], Interest > Name across the network is /att/alex-ndnsim/action.cgi/[sig], but the > signature is generated based on /ndnsim/action.cgi > > It seems that replacement make the router's life harder, because the > router cannot do the replacement without knowing the Link object. > > Adding delegation to the mix could prevent an attack in IP DNS. > In IP DNS, if the owner of a large website points his A record to a > smaller website, the large amount of traffic would cause denial of service > in the smaller website, although doing so would make the large website > itself inaccessible. (This happened once on BAIDU, a top search engine in > China; an attacker hijacked BAIDU's Nameserver and pointed A record to a > small server). > > > This attack may happen because DNSSEC is not enable. NDNS records are > signed, and resolvers are required to validate that, so the worst case is > that no one can do DNS resolution for Baidu, but the small web site should > be fine. > A high-level comment without getting into DNS/DNSSEC details: a link object "A -> B" involves two parties, i.e., A and B. Unless A and B are actually the same party, you need to have two signatures, one from A and one from B, to indicate that both parties have agreed on this link relationship. A single signature (as in the LINK redirect object) is not enough to prevent such attacks as the one mentioned by Junxiao. Wentao > > If the owner of (large website) /ndnsim points the NDNS record to another > user's namespace such as /verizon/john-victim (small website), it's the > same attack as described above. > > > First, I don't think any descent big website intends to do that. If they > really want to attack some small websites, there are a lot of other easy > ways to do so instead of sacrificing its own service. > > Second, if any attacker want to do that, they have to accumulate a lot of > interests, which is not trivial at the first place. > > Storing the delegation record instead of redirection record in NDNS would > prevent this attack. > > > As you mentioned before, if there is not route for those "mis-redirected" > interests, the interests will be dropped. Doesn't this prevent the attack? > > A drawback of delegation record is: it promotes content control by ISP. > In IP DNS, user has the freedom to host any website on the IP address > (unless ISP deploys layer-7 filtering). > If a delegation record is needed, ISP could refuse to sign a delegation > record if they don't want user to host a certain website. > > > Agree. > > > Yours, Junxiao > > On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev < > alexander.afanasyev at ucla.edu> wrote: > >> Hi! >> >> I want continue our discussion about LINK specification ( >> named-data.net/downloads/memos/link.pdf). >> >> Currently we preliminary defined three elements: >> >> a) link for redirection: >> >> signA("nameA", content: "nameB") >> >> b) link for delegation >> >> signA("nameA", content: signB("nameB", content: "prefixOfNameA")) >> >> c) "link" for encapsulation >> >> sha256("nameB" | "nameA", content: ("delegation or redirection link", >> data("nameA")) >> >> >> The option (b) is the latest addition and I have some reservations about >> it. In particular, what exactly this delegation achieves? What security >> problem we are solving with it. >> >> Let me give some example of (b). I own a "non-routable" namespace >> /ndnsim and my site is connected to /att and /ucla networks. So, the (b) >> option would require: >> >> - I will have to obtain a "redirection" links from /att and /ucla to >> /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim >> site >> >> - Then I will create a delegation packet(s) that I can put in NDNS to >> tell others that my site is currently available through /att and /ucla. >> >> The process looks ok, but the question I have is what significance is in >> the first permission. What is the point of me of requesting permission >> from my provider to host a website? What I would gain and what provider >> itself would gain? >> >> I personally, do not yet see a clear benefit from this process, only that >> it would complicate the delegation process. Only me is able to tell others >> that my site is available at different places, so there is no question >> about others claiming that my site is available somewhere else. If I >> mistakenly put a wrong link, then I myself will be at loss: i wouldn't be >> able to serve my content and somebody else would receive my interests and >> could reply to them with some junk. >> >> If I "maliciously" put somebody else's network as a link, then interests >> for my data would go there. But what is the harm? If nobody replies to >> them, then routers can start ignoring/pushing back such interests. If >> somebody replying, then somebody serves data and from the network point of >> view nothing bad is happening. >> >> >> In short, I want us have a deep discussion on what exactly we are >> protecting and from whom. For me, even with only one signature (option a), >> only the owner of the original namespace (/ndnsim) is able to create a >> legitimate link to somewhere else. Additional signature could protect >> "provider" (destination of the link), but is it really a problem? Anybody >> could send any number of interests to that provider even without any link >> (erroneous or malicious). >> >> >> --- >> Alex >> >> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev >> > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.afanasyev at ucla.edu Sun Sep 14 15:39:35 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Sun, 14 Sep 2014 15:39:35 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <9B3794D1-C8FD-49D7-AF85-4F17FC5C7F34@memphis.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9B3794D1-C8FD-49D7-AF85-4F17FC5C7F34@memphis.edu> Message-ID: <58449AD5-6EF6-4B61-AD5F-68E0729CE26E@ucla.edu> Hi Lan, Why this is an alternative? This is exactly what this delegation is about and what I'm struggling with. why would I require permission to host my website in att and who is at harm if I wrongly put a link to third party network? How is it different from somebody just sending tons of interest to that third party network? --- Alex On Sep 14, 2014, at 3:32 PM, Lan Wang (lanwang) wrote: > An alternative failure scenario which benefits from delegation is that the owner of /ndnsim misconfigured its link and put another ISP's name (maybe it used to be the provider) there or just forgot to remove the ISP's name when it switches to a new ISP. If we require that ISP explicitly delegates its subprefix to link to /ndnsim, then Interests won't flood to the wrong ISP, which is undesirable for both the ISP and /ndnsim. > > Lan > > On Sep 12, 2014, at 11:53 PM, Junxiao Shi wrote: > >> Dear folks >> >> In the non-routable web hosting scenario, to achieve the same level of security as IP DNS, we only need redirection and encapsulation LINKs. >> /ndnsim doesn't need /att's permission to host the website under /att/alex-ndnsim. This is similar to IP DNS: to setup an A/AAAA record to that IP address, you don't need IP address owner's permission. >> >> The procedure of accessing /ndnsim contents is: >> perform a NDNS lookup for /ndnsim, which would return a redirection LINK >> sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") >> replace /ndnsim with /att/alex-ndnsim in all Interests >> to access /ndnsim/index.html, express Interest /att/alex-ndnsim/index.html >> some router near end host or the end host itself or the producer application, must replace /att/alex-ndnsim back to /ndnsim >> to execute signed Interest /ndnsim/action.cgi/[sig], Interest Name across the network is /att/alex-ndnsim/action.cgi/[sig], but the signature is generated based on /ndnsim/action.cgi >> Data is encapsulated in encapsulation LINK >> sha256(name=/att/alex-ndnsim/index.html/segment0, content={sign[ndnsim](name=/ndnsim, content="redirect=/att/alex-ndnsim") | sign[ndnsim](name=/ndnsim/index.html/segment0, content=payload)}) >> >> If the owner of /ndnsim points the NDNS record to some bogus location such as /verizon/alex-ndnsim, those Interests would be dropped after reaching /verizon due to lack of Route. >> >> >> Adding delegation to the mix could prevent an attack in IP DNS. >> In IP DNS, if the owner of a large website points his A record to a smaller website, the large amount of traffic would cause denial of service in the smaller website, although doing so would make the large website itself inaccessible. (This happened once on BAIDU, a top search engine in China; an attacker hijacked BAIDU's Nameserver and pointed A record to a small server). >> If the owner of (large website) /ndnsim points the NDNS record to another user's namespace such as /verizon/john-victim (small website), it's the same attack as described above. >> Storing the delegation record instead of redirection record in NDNS would prevent this attack. >> >> A drawback of delegation record is: it promotes content control by ISP. >> In IP DNS, user has the freedom to host any website on the IP address (unless ISP deploys layer-7 filtering). >> If a delegation record is needed, ISP could refuse to sign a delegation record if they don't want user to host a certain website. >> >> Yours, Junxiao >> >> On Fri, Sep 12, 2014 at 4:40 PM, Alex Afanasyev wrote: >> Hi! >> >> I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). >> >> Currently we preliminary defined three elements: >> >> a) link for redirection: >> >> signA("nameA", content: "nameB") >> >> b) link for delegation >> >> signA("nameA", content: signB("nameB", content: "prefixOfNameA")) >> >> c) "link" for encapsulation >> >> sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) >> >> >> The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. >> >> Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: >> >> - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site >> >> - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. >> >> The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? >> >> I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. >> >> If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. >> >> >> In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). >> >> >> --- >> Alex >> >> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Sun Sep 14 15:43:10 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Sun, 14 Sep 2014 15:43:10 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: It seems that there are only two cases in which the delegation LINK may be used: case a) NDNS resolution: in this case, data consumer will not send an interests without getting a valid delegation LINK from the NDNS response. It seems that we are trying to solve a problem that a network may receive some undesired interests. But my question is: Can this "delegation" or "authorization" solve this problem? I do not think it is the right direction, because no one can control what interest others can send. Attackers have a lot of ways to redirect interests to some victim networks, (e.g., in http, one can hide a URI in a very popular webpage, and many browsers viewing that webpage may send a pre-fetching request for that URI). Given there are so many ways to force end users to send interests to somewhere as the attacker wishes, I think we should think about how to prevent in some other ways (such as dropping no-route interest), rather than trying to control the data consumer's behavior. case b) from the encapsulation packet. the data consumer has already got the requested data and can validate the data directly, so the delegation LINK seems unnecessary in this case. Yingdi On Sep 12, 2014, at 4:40 PM, Alex Afanasyev wrote: > Hi! > > I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). > > Currently we preliminary defined three elements: > > a) link for redirection: > > signA("nameA", content: "nameB") > > b) link for delegation > > signA("nameA", content: signB("nameB", content: "prefixOfNameA")) > > c) "link" for encapsulation > > sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) > > > The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. > > Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: > > - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site > > - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. > > The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? > > I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. > > If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. > > > In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). > > > --- > Alex > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From yingdi at CS.UCLA.EDU Sun Sep 14 15:49:37 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Sun, 14 Sep 2014 15:49:37 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <76421C1B-B66D-460F-BD67-5EAD0DC47A84@cs.ucla.edu> Message-ID: On Sep 14, 2014, at 3:38 PM, Wentao Shang wrote: > On Sun, Sep 14, 2014 at 3:11 PM, Yingdi Yu wrote: > > On Sep 12, 2014, at 9:53 PM, Junxiao Shi wrote: > >> Adding delegation to the mix could prevent an attack in IP DNS. >> In IP DNS, if the owner of a large website points his A record to a smaller website, the large amount of traffic would cause denial of service in the smaller website, although doing so would make the large website itself inaccessible. (This happened once on BAIDU, a top search engine in China; an attacker hijacked BAIDU's Nameserver and pointed A record to a small server). > > This attack may happen because DNSSEC is not enable. NDNS records are signed, and resolvers are required to validate that, so the worst case is that no one can do DNS resolution for Baidu, but the small web site should be fine. > > A high-level comment without getting into DNS/DNSSEC details: > > a link object "A -> B" involves two parties, i.e., A and B. Unless A and B are actually the same party, you need to have two signatures, one from A and one from B, to indicate that both parties have agreed on this link relationship. A single signature (as in the LINK redirect object) is not enough to prevent such attacks as the one mentioned by Junxiao. In the attack above, a single signature is enough. The attacker can redirect the traffic because it returns a wrong DNS A record for Baidu, and dns resolvers simply accept the response. If dns resolver can validate the signature of the DNS A record (signed by Baidu's key), then they will tell that the returned A record is wrong, and they will not send anything to the IP address in the returned DNS A record. The whole process does not require any signature from the small website side. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alexander.afanasyev at ucla.edu Sun Sep 14 15:57:35 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Sun, 14 Sep 2014 15:57:35 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: Message-ID: On Sep 14, 2014, at 3:37 PM, Burke, Jeff wrote: > Hi, > > Maybe I am coming into the middle of another discussion, but I still have > yet to understand the status (and the choice of name) for NDNS. > > Is this new component now a "given" in the architecture? Or, is it an > example of where LINKs may come from? > > Junxiao uses the phrase "perform a NDNS lookup for /ndnsim" - is this > different than just issuing an Interest for "/ndnsim"? For this particular discussion, we can assume that there is some kind of resolution mechanism. It doesn't need to be NDNS, but it would provide such a service (I hope very soon). The question here is what we want such lookup service to return and what actions should happen after link is received. We can also discuss questions of who issues queries, but this is different type of discussion and I would prefer if we get to understanding on the basic elements. Answering the specific question about the lookup. In terms of NDNS, the lookup process is more complex than simply sending an interest (= sending a set of interests according to the protocol spec). For example, if not all NDNS authoritative servers can be directly accessed (/net/ndnsim's authoritative NDNS server in this example), one would need to discover first LINK for that server, after which it can send another query to discover link for the actual data item. In any case, I would like to defer this specific conversation until we have initial specification for updated the NDNS protocol (should be available some time this week). --- Alex > Jeff > > On 9/12/14, 4:40 PM, "Alex Afanasyev" wrote: > >> Hi! >> >> I want continue our discussion about LINK specification >> (named-data.net/downloads/memos/link.pdf). >> >> Currently we preliminary defined three elements: >> >> a) link for redirection: >> >> signA("nameA", content: "nameB") >> >> b) link for delegation >> >> signA("nameA", content: signB("nameB", content: "prefixOfNameA")) >> >> c) "link" for encapsulation >> >> sha256("nameB" | "nameA", content: ("delegation or redirection link", >> data("nameA")) >> >> >> The option (b) is the latest addition and I have some reservations about >> it. In particular, what exactly this delegation achieves? What security >> problem we are solving with it. >> >> Let me give some example of (b). I own a "non-routable" namespace >> /ndnsim and my site is connected to /att and /ucla networks. So, the (b) >> option would require: >> >> - I will have to obtain a "redirection" links from /att and /ucla to >> /ndnsim (option (a)) = get a permission from /att and /ucla to host >> /ndnsim site >> >> - Then I will create a delegation packet(s) that I can put in NDNS to >> tell others that my site is currently available through /att and /ucla. >> >> The process looks ok, but the question I have is what significance is in >> the first permission. What is the point of me of requesting permission >> from my provider to host a website? What I would gain and what provider >> itself would gain? >> >> I personally, do not yet see a clear benefit from this process, only that >> it would complicate the delegation process. Only me is able to tell >> others that my site is available at different places, so there is no >> question about others claiming that my site is available somewhere else. >> If I mistakenly put a wrong link, then I myself will be at loss: i >> wouldn't be able to serve my content and somebody else would receive my >> interests and could reply to them with some junk. >> >> If I "maliciously" put somebody else's network as a link, then interests >> for my data would go there. But what is the harm? If nobody replies to >> them, then routers can start ignoring/pushing back such interests. If >> somebody replying, then somebody serves data and from the network point >> of view nothing bad is happening. >> >> >> In short, I want us have a deep discussion on what exactly we are >> protecting and from whom. For me, even with only one signature (option >> a), only the owner of the original namespace (/ndnsim) is able to create >> a legitimate link to somewhere else. Additional signature could protect >> "provider" (destination of the link), but is it really a problem? >> Anybody could send any number of interests to that provider even without >> any link (erroneous or malicious). >> >> >> --- >> Alex >> >> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > From alexander.afanasyev at UCLA.EDU Mon Sep 15 10:53:20 2014 From: alexander.afanasyev at UCLA.EDU (Alex Afanasyev) Date: Mon, 15 Sep 2014 10:53:20 -0700 Subject: [Nfd-dev] do we need "application store"? In-Reply-To: References: <6092B762-0A3C-4F56-9FCF-C47129E49159@cs.ucla.edu> <1B6E83E5-5A23-4FFA-A2F0-481904E2933B@cs.arizona.edu> <010AE88E-0AEA-4F3C-BF85-C79E9F1C2948@cs.colostate.edu> <5CEC0D7B-D334-4458-8AD2-217E5770239F@cs.ucla.edu> <254FFD08-C66B-4592-A0AF-0B71A3BE87BE@cs.colostate.edu> <1FBC85FD-14A2-4D29-9A7C-04E182FEABF0@cs.ucla.edu> <6C0BC12F-9FD5-4118-8A44-3FF4FD8FDAEC@cs.ucla.edu> <48AAF5EF-D183-43AB-A451-FCFAD6EB4A57@cs.ucla.edu> Message-ID: <90DDE875-1182-4B22-A083-36C6E63A8D1B@ucla.edu> Hi Yingdi, I like the design. I would only suggest a couple of quick changes: - there is too much description about freshness. I think it would be enough to say that a primary objective of freshness is to guard producer from receiving too many interests. - I would avoid putting PIB service as a first example. Start with the existing and relatively known apps (chat/sync/nlsr). For PIB service, I wouldn't talk about "system default key", rather "current certificate for the key" and optionally "current key for the identity". - for the design goal, you should start with emphasizing that "common pattern" is to keep produced data packet for a while, so it can be served later. You already have all the description, I would just start with the "common pattern" statement and then go into more details. - "... packets are not backed up ..." this thing is commonly called serialization. Either "storage" is serialized on disk or each data packet in the storage is serialized on disk. - don't add save/load to the interface. This is completely unrelated to storage and, as you mentioned before, there could be tons of different ways to do it. If you simply provide `begin() const', `end() const' methods, I thing it would be enough to implement any kind of serialization. We can make a serialization helper, but it is still outside the scope of InMemoryStorage class. - Can you add a diagram/description on how applications would use this storage? I mean, description on how storage is connected within the application: app registers prefix, sets interest filter, receives interest, calls in memory storage, etc. --- Alex On Sep 13, 2014, at 10:57 PM, Yingdi Yu wrote: > Hi all, > > I have updated the design document of InMemoryStorage: http://redmine.named-data.net/projects/ndn-cxx/wiki/InMemoryStorage. I would like to get feedback from you, so that we can nail down the design details, and Jiewen Tan can keep working on it. Thanks! > > Yingdi > > > > > On Sep 10, 2014, at 11:17 AM, Yingdi Yu wrote: > >> On Sep 10, 2014, at 9:47 AM, Steve DiBenedetto wrote: >> >>> As for using the CS for an application's data storage: I want understand the role of the CS in the special, but common, case of end hosts/local nodes/whatever-you-want to-call-them where we have vastly more storage capabilities, both memory and disk, than a router. Is the CS still just the store-and-forwarder buffer or is it something else? >> >> Yes, I also hope we can clarify the role of CS. >> >>> I'm trying to view the end host CS as master storage controller (probably a bad term) that conceptually holds everything on the system and presents a API for interacting with named data. >> >> In this case, why not just run a repo on the system. It can provide more persistent storage than CS. >> >>> In other words, something that evolves from just our "store-and-forward buffer with named data" mentioned in the original paper towards NDNFS' idea(s) (unfortunately not familiar with the details). >> >> I am not sure about NDNFS, maybe repo is a better one to describe your idea? >> >>> Then, all data storage is using shared resources between the forwarder and applications. >> >> This is the thing I did not get through the whole discussion: what is the benefit of sharing resources? >> >>> That said, is the InMemoryStorage proposal just a temporary solution or are we also making a statement about what we anticipate non-duplicating data storage on end hosts to look like? >> >> I would not take InMemoryStorage as a temporary solution. It represents one of the data management models, that is, apps should take care of their data, rather than relying on forwarder. I am not saying this is only model, therefore apps can make their own choice whether to adopt this model or not. This is also why I always take the in-app storage as a helper class in the library. >> >>> I took Lixia's initial prompt to be a discussion of the latter, but it's very possible that I'm dragging the discussion to somewhere it was never intended to go. >>> >>> I was hoping to discuss the design a new/alternative CS residing within NFD that communicates with applications through a library API (hopefully similar to InMemoryStorage). The idea being to provide unified named data storage across the NDN app ecosystem that emulates some of the OS' storage functionality. This would obviously be *a lot* of work, but it makes a statement of "how things should be". >> >> One thing I do not agree with Lixia's argument is that this is not about resources, this is also about management. If there is enough resources, it does not matter where the data is stored. If there is not enough resources, neither putting data in CS nor in app cannot help. >> >> However, no matter where the data is stored, the holder should manage the data. The cost of managing apps' data in CS is much higher than asking apps to do it by themselves. >> >> Yingdi > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From lixia at CS.UCLA.EDU Mon Sep 15 12:54:19 2014 From: lixia at CS.UCLA.EDU (Lixia Zhang) Date: Mon, 15 Sep 2014 12:54:19 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> Message-ID: <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> top posting: sorry I'm falling behind email; Yingdi reminded me today that there is an exchange going on here. maybe I read the long msg too quick: I got confused with whether nameA=ndnSIM, or nameA=att in Alex's argument .... I also do not understand the format of (b) below... Further, what is "link for encapsulation"... My understanding says that an encapsulated Data packet includes 2 objects, a LINK and a Data, so what is "link for encapsulation" ? On Sep 12, 2014, at 4:40 PM, Alex Afanasyev wrote: > Hi! > > I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). > > Currently we preliminary defined three elements: > > a) link for redirection: > > signA("nameA", content: "nameB") > > b) link for delegation > > signA("nameA", content: signB("nameB", content: "prefixOfNameA")) > > c) "link" for encapsulation > > sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) > > > The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. > > Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: > > - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site > > - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. > > The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? > > I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. > > If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. > > > In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). > > > --- > Alex > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev From alexander.afanasyev at UCLA.EDU Mon Sep 15 13:09:40 2014 From: alexander.afanasyev at UCLA.EDU (Alex Afanasyev) Date: Mon, 15 Sep 2014 13:09:40 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> Message-ID: <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> On Sep 15, 2014, at 12:54 PM, Lixia Zhang wrote: > top posting: sorry I'm falling behind email; Yingdi reminded me today that there is an exchange going on here. > > maybe I read the long msg too quick: I got confused with whether nameA=ndnSIM, or nameA=att in Alex's argument .... > > I also do not understand the format of (b) below... > > Further, what is "link for encapsulation"... My understanding says that an encapsulated Data packet includes 2 objects, a LINK and a Data, so what is "link for encapsulation" ? Please see the named-data.net/downloads/memos/link.pdf spec. I intentionally did not fully described all the option and listed just a summary of what we currently have in the link.pdf. All naming is very preliminary, so it could be confusing. --- Alex > > On Sep 12, 2014, at 4:40 PM, Alex Afanasyev wrote: > >> Hi! >> >> I want continue our discussion about LINK specification (named-data.net/downloads/memos/link.pdf). >> >> Currently we preliminary defined three elements: >> >> a) link for redirection: >> >> signA("nameA", content: "nameB") >> >> b) link for delegation >> >> signA("nameA", content: signB("nameB", content: "prefixOfNameA")) >> >> c) "link" for encapsulation >> >> sha256("nameB" | "nameA", content: ("delegation or redirection link", data("nameA")) >> >> >> The option (b) is the latest addition and I have some reservations about it. In particular, what exactly this delegation achieves? What security problem we are solving with it. >> >> Let me give some example of (b). I own a "non-routable" namespace /ndnsim and my site is connected to /att and /ucla networks. So, the (b) option would require: >> >> - I will have to obtain a "redirection" links from /att and /ucla to /ndnsim (option (a)) = get a permission from /att and /ucla to host /ndnsim site >> >> - Then I will create a delegation packet(s) that I can put in NDNS to tell others that my site is currently available through /att and /ucla. >> >> The process looks ok, but the question I have is what significance is in the first permission. What is the point of me of requesting permission from my provider to host a website? What I would gain and what provider itself would gain? >> >> I personally, do not yet see a clear benefit from this process, only that it would complicate the delegation process. Only me is able to tell others that my site is available at different places, so there is no question about others claiming that my site is available somewhere else. If I mistakenly put a wrong link, then I myself will be at loss: i wouldn't be able to serve my content and somebody else would receive my interests and could reply to them with some junk. >> >> If I "maliciously" put somebody else's network as a link, then interests for my data would go there. But what is the harm? If nobody replies to them, then routers can start ignoring/pushing back such interests. If somebody replying, then somebody serves data and from the network point of view nothing bad is happening. >> >> >> In short, I want us have a deep discussion on what exactly we are protecting and from whom. For me, even with only one signature (option a), only the owner of the original namespace (/ndnsim) is able to create a legitimate link to somewhere else. Additional signature could protect "provider" (destination of the link), but is it really a problem? Anybody could send any number of interests to that provider even without any link (erroneous or malicious). >> >> >> --- >> Alex >> >> >> >> _______________________________________________ >> Nfd-dev mailing list >> Nfd-dev at lists.cs.ucla.edu >> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > From lixia at CS.UCLA.EDU Mon Sep 15 15:50:15 2014 From: lixia at CS.UCLA.EDU (Lixia Zhang) Date: Mon, 15 Sep 2014 15:50:15 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> Message-ID: <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> On Sep 15, 2014, at 1:09 PM, Alex Afanasyev wrote: > On Sep 15, 2014, at 12:54 PM, Lixia Zhang wrote: > >> top posting: sorry I'm falling behind email; Yingdi reminded me today that there is an exchange going on here. >> >> maybe I read the long msg too quick: I got confused with whether nameA=ndnSIM, or nameA=att in Alex's argument .... >> >> I also do not understand the format of (b) below... >> >> Further, what is "link for encapsulation"... My understanding says that an encapsulated Data packet includes 2 objects, a LINK and a Data, so what is "link for encapsulation" ? > > Please see the named-data.net/downloads/memos/link.pdf spec. I intentionally did not fully described all the option and listed just a summary of what we currently have in the link.pdf. All naming is very preliminary, so it could be confusing. > > --- > Alex still have not read the spec yet but did caught up all the msgs. Here is a summary of my summary/opinion: 1/ I agree with Wentao on the following: > a link object "A -> B" involves two parties, i.e., A and B. Unless A and B are actually the same party, you need to have two signatures, one from A and one from B, to indicate that both parties have agreed on this link relationship. I disagree with Junxiao's comment that > A drawback of delegation record is: it promotes content control by ISP. > In IP DNS, user has the freedom to host any website on the IP address (unless ISP deploys layer-7 filtering). > If a delegation record is needed, ISP could refuse to sign a delegation record if they don't want user to host a certain website. the ISP only signs the delegation of /att/lixia, not /lixia/foobar (the content name) that I may publish. 2/ I agree with Yingdi that it's desirable to have some simple concatenation rule to glue together the routable-prefix and non-routable name for encapsulated Data packet, a much better way to go than any partial rewrite/replacement of the name. 3/ The LINK object in an encapsulated Data packet should be signed, this enables whoever interested in checking the authentication of the LINK to do so (yes there is still the trust model question to answer, but without this signature, one cannot check). 4/ there is one detail I'd like to spell out explicitly: my understanding is that all the unroutable names that publish globally reachable contents should be globally unique, they are just not on the global routing table. 5/ to answer JeffB's question: I think the assumption here is only that a resolution system can be used to get the routable prefix for an unroutable name; NDNS can be a candidate here. No decision has been made anywhere at this time. We need a NDNS TR out ASAP, then an NDN seminar on the topic, as one of the next steps. my 2 cents, Lixia From alexander.afanasyev at ucla.edu Mon Sep 15 16:02:48 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Mon, 15 Sep 2014 16:02:48 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> Message-ID: <62875D2B-821C-4225-927B-66382C686397@ucla.edu> On Sep 15, 2014, at 3:50 PM, Lixia Zhang wrote: > > On Sep 15, 2014, at 1:09 PM, Alex Afanasyev wrote: > >> On Sep 15, 2014, at 12:54 PM, Lixia Zhang wrote: >> >>> top posting: sorry I'm falling behind email; Yingdi reminded me today that there is an exchange going on here. >>> >>> maybe I read the long msg too quick: I got confused with whether nameA=ndnSIM, or nameA=att in Alex's argument .... >>> >>> I also do not understand the format of (b) below... >>> >>> Further, what is "link for encapsulation"... My understanding says that an encapsulated Data packet includes 2 objects, a LINK and a Data, so what is "link for encapsulation" ? >> >> Please see the named-data.net/downloads/memos/link.pdf spec. I intentionally did not fully described all the option and listed just a summary of what we currently have in the link.pdf. All naming is very preliminary, so it could be confusing. >> >> --- >> Alex > > still have not read the spec yet but did caught up all the msgs. Here is a summary of my summary/opinion: > > 1/ I agree with Wentao on the following: > >> a link object "A -> B" involves two parties, i.e., A and B. Unless A and B are actually the same party, you need to have two signatures, one from A and one from B, to indicate that both parties have agreed on this link relationship. This is very general statement. On a surface, this could be a desired property. The whole question which goal is this property achieving? In my opinion, giving provider a tool to allow/deny hosting things inside the provider is much bigger harm then allowing anybody to express their desire for their client to try to request data from the specific provider. As I mentioned before, even if such property implemented in the LINK, how is it different from just sending of bunch of unsatisfiable Interests towards the provider? > I disagree with Junxiao's comment that > >> A drawback of delegation record is: it promotes content control by ISP. >> In IP DNS, user has the freedom to host any website on the IP address (unless ISP deploys layer-7 filtering). >> If a delegation record is needed, ISP could refuse to sign a delegation record if they don't want user to host a certain website. > > the ISP only signs the delegation of /att/lixia, not /lixia/foobar (the content name) that I may publish. I disagree, since this is not a fair example: it does not show all the "danger". You assume here that you "own" just /lixia namespace and you publish data only under this namespace, which somehow is related to your relationship with att. In more general, I could own/control a bunch of namespaces, which has nothing to do with me as ATT customer. Let's say we have /att (routable) and /net/ndnsim, /net/6watch, /net/named-data (all unroutable, but somewhere inside att network). ATT would need to sign all these delegations, essentially having "control" over which namespace can be served by ATT network and which not. > > 2/ I agree with Yingdi that it's desirable to have some simple concatenation rule to glue together the routable-prefix and non-routable name for encapsulated Data packet, a much better way to go than any partial rewrite/replacement of the name. > > 3/ The LINK object in an encapsulated Data packet should be signed, this enables whoever interested in checking the authentication of the LINK to do so (yes there is still the trust model question to answer, but without this signature, one cannot check). This one needs to be more specific. Which option you're talking about here? We have 3 "link objects" (we need to have a better terminology for that), each having different set of security parameters and meanings. --- Alex > 4/ there is one detail I'd like to spell out explicitly: my understanding is that all the unroutable names that publish globally reachable contents should be globally unique, they are just not on the global routing table. > > 5/ to answer JeffB's question: I think the assumption here is only that a resolution system can be used to get the routable prefix for an unroutable name; NDNS can be a candidate here. No decision has been made anywhere at this time. > We need a NDNS TR out ASAP, then an NDN seminar on the topic, as one of the next steps. > > my 2 cents, > Lixia > > From wentaoshang at gmail.com Mon Sep 15 17:20:23 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Mon, 15 Sep 2014 17:20:23 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <62875D2B-821C-4225-927B-66382C686397@ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> <62875D2B-821C-4225-927B-66382C686397@ucla.edu> Message-ID: On Mon, Sep 15, 2014 at 4:02 PM, Alex Afanasyev < alexander.afanasyev at ucla.edu> wrote: > > On Sep 15, 2014, at 3:50 PM, Lixia Zhang wrote: > > > > > On Sep 15, 2014, at 1:09 PM, Alex Afanasyev < > alexander.afanasyev at ucla.edu> wrote: > > > >> On Sep 15, 2014, at 12:54 PM, Lixia Zhang wrote: > >> > >>> top posting: sorry I'm falling behind email; Yingdi reminded me today > that there is an exchange going on here. > >>> > >>> maybe I read the long msg too quick: I got confused with whether > nameA=ndnSIM, or nameA=att in Alex's argument .... > >>> > >>> I also do not understand the format of (b) below... > >>> > >>> Further, what is "link for encapsulation"... My understanding says > that an encapsulated Data packet includes 2 objects, a LINK and a Data, so > what is "link for encapsulation" ? > >> > >> Please see the named-data.net/downloads/memos/link.pdf spec. I > intentionally did not fully described all the option and listed just a > summary of what we currently have in the link.pdf. All naming is very > preliminary, so it could be confusing. > >> > >> --- > >> Alex > > > > still have not read the spec yet but did caught up all the msgs. Here is > a summary of my summary/opinion: > > > > 1/ I agree with Wentao on the following: > > > >> a link object "A -> B" involves two parties, i.e., A and B. Unless A > and B are actually the same party, you need to have two signatures, one > from A and one from B, to indicate that both parties have agreed on this > link relationship. > > This is very general statement. On a surface, this could be a desired > property. The whole question which goal is this property achieving? In my > opinion, giving provider a tool to allow/deny hosting things inside the > provider is much bigger harm then allowing anybody to express their desire > for their client to try to request data from the specific provider. I don't quite understand this argument: if the provider doesn't want to host the client's content, the link relationship wouldn't have existed in the first place. The signature is only a technical means for the provider to express its approval of hosting. Whether the provide *will* host or not sounds more like a layer-10 issue to me. > As I mentioned before, even if such property implemented in the LINK, how > is it different from just sending of bunch of unsatisfiable Interests > towards the provider? > I'm not comfortable with the logic behind this argument: if there are two backdoors that can lead to the same security hole, can we simply say that we can leave the 2nd door open because the hackers are going to use the 1st door anyway? I think the key issue here is really the cost of having such complexity versus how much we can gain from it. For that I don't have a good answer. Current Web architecture has been running with uni-directional links for years without (??) major problems. So I feel we can use "redirect LINK object" as the default and keep the "delegation LINK object" as optional for the applications to choose. Wentao > > > > I disagree with Junxiao's comment that > > > >> A drawback of delegation record is: it promotes content control by ISP. > >> In IP DNS, user has the freedom to host any website on the IP address > (unless ISP deploys layer-7 filtering). > >> If a delegation record is needed, ISP could refuse to sign a delegation > record if they don't want user to host a certain website. > > > > the ISP only signs the delegation of /att/lixia, not /lixia/foobar (the > content name) that I may publish. > > I disagree, since this is not a fair example: it does not show all the > "danger". You assume here that you "own" just /lixia namespace and you > publish data only under this namespace, which somehow is related to your > relationship with att. > > In more general, I could own/control a bunch of namespaces, which has > nothing to do with me as ATT customer. Let's say we have /att (routable) > and /net/ndnsim, /net/6watch, /net/named-data (all unroutable, but > somewhere inside att network). > > ATT would need to sign all these delegations, essentially having "control" > over which namespace can be served by ATT network and which not. > > > > > 2/ I agree with Yingdi that it's desirable to have some simple > concatenation rule to glue together the routable-prefix and non-routable > name for encapsulated Data packet, a much better way to go than any partial > rewrite/replacement of the name. > > > > 3/ The LINK object in an encapsulated Data packet should be signed, this > enables whoever interested in checking the authentication of the LINK to do > so (yes there is still the trust model question to answer, but without this > signature, one cannot check). > > This one needs to be more specific. Which option you're talking about > here? We have 3 "link objects" (we need to have a better terminology for > that), each having different set of security parameters and meanings. > > --- > Alex > > > 4/ there is one detail I'd like to spell out explicitly: my > understanding is that all the unroutable names that publish globally > reachable contents should be globally unique, they are just not on the > global routing table. > > > > 5/ to answer JeffB's question: I think the assumption here is only that > a resolution system can be used to get the routable prefix for an > unroutable name; NDNS can be a candidate here. No decision has been made > anywhere at this time. > > We need a NDNS TR out ASAP, then an NDN seminar on the topic, as one of > the next steps. > > > > my 2 cents, > > Lixia > > > > > > > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev > -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Mon Sep 15 21:19:02 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 15 Sep 2014 21:19:02 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> <62875D2B-821C-4225-927B-66382C686397@ucla.edu> Message-ID: <4104A1A1-F9AD-4A1D-8507-A237302ECA75@cs.ucla.edu> On Sep 15, 2014, at 5:20 PM, Wentao Shang wrote: > > > On Mon, Sep 15, 2014 at 4:02 PM, Alex Afanasyev wrote: > > On Sep 15, 2014, at 3:50 PM, Lixia Zhang wrote: > > > 1/ I agree with Wentao on the following: > > > >> a link object "A -> B" involves two parties, i.e., A and B. Unless A and B are actually the same party, you need to have two signatures, one from A and one from B, to indicate that both parties have agreed on this link relationship. > > This is very general statement. On a surface, this could be a desired property. The whole question which goal is this property achieving? In my opinion, giving provider a tool to allow/deny hosting things inside the provider is much bigger harm then allowing anybody to express their desire for their client to try to request data from the specific provider. > > I don't quite understand this argument: if the provider doesn't want to host the client's content, the link relationship wouldn't have existed in the first place. No, it is not about hosting the client's content, the client still hosts its own data from its own machine, but the machine is connected to the internet through att's network. Basically, the client only bought the internet access from att. If a link object signed by att is required to publish the data, it implies that att can decide the type of data that the client can publish. I think this violates the network neutrality, and nobody would like that. > The signature is only a technical means for the provider to express its approval of hosting. Whether the provide *will* host or not sounds more like a layer-10 issue to me. > > As I mentioned before, even if such property implemented in the LINK, how is it different from just sending of bunch of unsatisfiable Interests towards the provider? > > I'm not comfortable with the logic behind this argument: if there are two backdoors that can lead to the same security hole, can we simply say that we can leave the 2nd door open because the hackers are going to use the 1st door anyway? No, the logic is: if one of the backdoor is easier to be exploited to launch an attack, what is the point of fixing the other backdoor (which is more difficult to be exploited)? Attackers are not fools. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alanwake at ucla.edu Mon Sep 15 21:39:22 2014 From: alanwake at ucla.edu (Jiewen Tan) Date: Mon, 15 Sep 2014 21:39:22 -0700 Subject: [Nfd-dev] do we need "application store"? In-Reply-To: <90DDE875-1182-4B22-A083-36C6E63A8D1B@ucla.edu> References: <6092B762-0A3C-4F56-9FCF-C47129E49159@cs.ucla.edu> <1B6E83E5-5A23-4FFA-A2F0-481904E2933B@cs.arizona.edu> <010AE88E-0AEA-4F3C-BF85-C79E9F1C2948@cs.colostate.edu> <5CEC0D7B-D334-4458-8AD2-217E5770239F@cs.ucla.edu> <254FFD08-C66B-4592-A0AF-0B71A3BE87BE@cs.colostate.edu> <1FBC85FD-14A2-4D29-9A7C-04E182FEABF0@cs.ucla.edu> <6C0BC12F-9FD5-4118-8A44-3FF4FD8FDAEC@cs.ucla.edu> <48AAF5EF-D183-43AB-A451-FCFAD6EB4A57@cs.ucla.edu> <90DDE875-1182-4B22-A083-36C6E63A8D1B@ucla.edu> Message-ID: <00fb01cfd168$309440e0$91bcc2a0$@ucla.edu> Hi Alex and Yingdi, I do agree with Alex that it might be more useful to make the save/load interfaces as virtual interfaces to provide more flexibility to the app developers. Yet I think it would be necessary to provide such interfaces, as some apps might need to save states when they exit and resume when they start. Although app developers can surely inherit the base in-memory storage class and do whatever they want in the derived one, it would be good to provide a base helper for lazy app developers and demonstrate an example to do serialization. Regards, Jiewen Tan From: Alexander Afanasyev [mailto:cawka1 at gmail.com] On Behalf Of Alex Afanasyev Sent: Monday, September 15, 2014 10:53 AM To: nfd-dev Cc: Yingdi Yu; Steve DiBenedetto; Lixia Zhang; Beichuan Zhang; Junxiao Shi; Jiewen Tan Subject: Re: do we need "application store"? Hi Yingdi, I like the design. I would only suggest a couple of quick changes: - there is too much description about freshness. I think it would be enough to say that a primary objective of freshness is to guard producer from receiving too many interests. - I would avoid putting PIB service as a first example. Start with the existing and relatively known apps (chat/sync/nlsr). For PIB service, I wouldn't talk about "system default key", rather "current certificate for the key" and optionally "current key for the identity". - for the design goal, you should start with emphasizing that "common pattern" is to keep produced data packet for a while, so it can be served later. You already have all the description, I would just start with the "common pattern" statement and then go into more details. - "... packets are not backed up ..." this thing is commonly called serialization. Either "storage" is serialized on disk or each data packet in the storage is serialized on disk. - don't add save/load to the interface. This is completely unrelated to storage and, as you mentioned before, there could be tons of different ways to do it. If you simply provide `begin() const', `end() const' methods, I thing it would be enough to implement any kind of serialization. We can make a serialization helper, but it is still outside the scope of InMemoryStorage class. - Can you add a diagram/description on how applications would use this storage? I mean, description on how storage is connected within the application: app registers prefix, sets interest filter, receives interest, calls in memory storage, etc. --- Alex On Sep 13, 2014, at 10:57 PM, Yingdi Yu wrote: Hi all, I have updated the design document of InMemoryStorage: http://redmine.named-data.net/projects/ndn-cxx/wiki/InMemoryStorage. I would like to get feedback from you, so that we can nail down the design details, and Jiewen Tan can keep working on it. Thanks! Yingdi On Sep 10, 2014, at 11:17 AM, Yingdi Yu wrote: On Sep 10, 2014, at 9:47 AM, Steve DiBenedetto wrote: As for using the CS for an application's data storage: I want understand the role of the CS in the special, but common, case of end hosts/local nodes/whatever-you-want to-call-them where we have vastly more storage capabilities, both memory and disk, than a router. Is the CS still just the store-and-forwarder buffer or is it something else? Yes, I also hope we can clarify the role of CS. I'm trying to view the end host CS as master storage controller (probably a bad term) that conceptually holds everything on the system and presents a API for interacting with named data. In this case, why not just run a repo on the system. It can provide more persistent storage than CS. In other words, something that evolves from just our "store-and-forward buffer with named data" mentioned in the original paper towards NDNFS' idea(s) (unfortunately not familiar with the details). I am not sure about NDNFS, maybe repo is a better one to describe your idea? Then, all data storage is using shared resources between the forwarder and applications. This is the thing I did not get through the whole discussion: what is the benefit of sharing resources? That said, is the InMemoryStorage proposal just a temporary solution or are we also making a statement about what we anticipate non-duplicating data storage on end hosts to look like? I would not take InMemoryStorage as a temporary solution. It represents one of the data management models, that is, apps should take care of their data, rather than relying on forwarder. I am not saying this is only model, therefore apps can make their own choice whether to adopt this model or not. This is also why I always take the in-app storage as a helper class in the library. I took Lixia's initial prompt to be a discussion of the latter, but it's very possible that I'm dragging the discussion to somewhere it was never intended to go. I was hoping to discuss the design a new/alternative CS residing within NFD that communicates with applications through a library API (hopefully similar to InMemoryStorage). The idea being to provide unified named data storage across the NDN app ecosystem that emulates some of the OS' storage functionality. This would obviously be *a lot* of work, but it makes a statement of "how things should be". One thing I do not agree with Lixia's argument is that this is not about resources, this is also about management. If there is enough resources, it does not matter where the data is stored. If there is not enough resources, neither putting data in CS nor in app cannot help. However, no matter where the data is stored, the holder should manage the data. The cost of managing apps' data in CS is much higher than asking apps to do it by themselves. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: From alanwake at ucla.edu Mon Sep 15 21:46:43 2014 From: alanwake at ucla.edu (Jiewen Tan) Date: Mon, 15 Sep 2014 21:46:43 -0700 Subject: [Nfd-dev] do we need "application store"? References: <6092B762-0A3C-4F56-9FCF-C47129E49159@cs.ucla.edu> <1B6E83E5-5A23-4FFA-A2F0-481904E2933B@cs.arizona.edu> <010AE88E-0AEA-4F3C-BF85-C79E9F1C2948@cs.colostate.edu> <5CEC0D7B-D334-4458-8AD2-217E5770239F@cs.ucla.edu> <254FFD08-C66B-4592-A0AF-0B71A3BE87BE@cs.colostate.edu> <1FBC85FD-14A2-4D29-9A7C-04E182FEABF0@cs.ucla.edu> <6C0BC12F-9FD5-4118-8A44-3FF4FD8FDAEC@cs.ucla.edu> <48AAF5EF-D183-43AB-A451-FCFAD6EB4A57@cs.ucla.edu> <90DDE875-1182-4B22-A083-36C6E63A8D1B@ucla.edu> Message-ID: <010b01cfd169$37830b60$a6892220$@ucla.edu> Hi All, Also, there is a need from Junxiao that is to provide an interface: size_t eraseByPrefix(const Name& prefix); to erase items stored in the in-memory storage by prefix. Regards, Jiewen Tan From: Jiewen Tan [mailto:alanwake at ucla.edu] Sent: Monday, September 15, 2014 9:39 PM To: 'Alex Afanasyev'; 'nfd-dev' Cc: 'Yingdi Yu'; 'Steve DiBenedetto'; 'Lixia Zhang'; 'Beichuan Zhang'; 'Junxiao Shi' Subject: RE: do we need "application store"? Hi Alex and Yingdi, I do agree with Alex that it might be more useful to make the save/load interfaces as virtual interfaces to provide more flexibility to the app developers. Yet I think it would be necessary to provide such interfaces, as some apps might need to save states when they exit and resume when they start. Although app developers can surely inherit the base in-memory storage class and do whatever they want in the derived one, it would be good to provide a base helper for lazy app developers and demonstrate an example to do serialization. Regards, Jiewen Tan From: Alexander Afanasyev [mailto:cawka1 at gmail.com] On Behalf Of Alex Afanasyev Sent: Monday, September 15, 2014 10:53 AM To: nfd-dev Cc: Yingdi Yu; Steve DiBenedetto; Lixia Zhang; Beichuan Zhang; Junxiao Shi; Jiewen Tan Subject: Re: do we need "application store"? Hi Yingdi, I like the design. I would only suggest a couple of quick changes: - there is too much description about freshness. I think it would be enough to say that a primary objective of freshness is to guard producer from receiving too many interests. - I would avoid putting PIB service as a first example. Start with the existing and relatively known apps (chat/sync/nlsr). For PIB service, I wouldn't talk about "system default key", rather "current certificate for the key" and optionally "current key for the identity". - for the design goal, you should start with emphasizing that "common pattern" is to keep produced data packet for a while, so it can be served later. You already have all the description, I would just start with the "common pattern" statement and then go into more details. - "... packets are not backed up ..." this thing is commonly called serialization. Either "storage" is serialized on disk or each data packet in the storage is serialized on disk. - don't add save/load to the interface. This is completely unrelated to storage and, as you mentioned before, there could be tons of different ways to do it. If you simply provide `begin() const', `end() const' methods, I thing it would be enough to implement any kind of serialization. We can make a serialization helper, but it is still outside the scope of InMemoryStorage class. - Can you add a diagram/description on how applications would use this storage? I mean, description on how storage is connected within the application: app registers prefix, sets interest filter, receives interest, calls in memory storage, etc. --- Alex On Sep 13, 2014, at 10:57 PM, Yingdi Yu wrote: Hi all, I have updated the design document of InMemoryStorage: http://redmine.named-data.net/projects/ndn-cxx/wiki/InMemoryStorage. I would like to get feedback from you, so that we can nail down the design details, and Jiewen Tan can keep working on it. Thanks! Yingdi On Sep 10, 2014, at 11:17 AM, Yingdi Yu wrote: On Sep 10, 2014, at 9:47 AM, Steve DiBenedetto wrote: As for using the CS for an application's data storage: I want understand the role of the CS in the special, but common, case of end hosts/local nodes/whatever-you-want to-call-them where we have vastly more storage capabilities, both memory and disk, than a router. Is the CS still just the store-and-forwarder buffer or is it something else? Yes, I also hope we can clarify the role of CS. I'm trying to view the end host CS as master storage controller (probably a bad term) that conceptually holds everything on the system and presents a API for interacting with named data. In this case, why not just run a repo on the system. It can provide more persistent storage than CS. In other words, something that evolves from just our "store-and-forward buffer with named data" mentioned in the original paper towards NDNFS' idea(s) (unfortunately not familiar with the details). I am not sure about NDNFS, maybe repo is a better one to describe your idea? Then, all data storage is using shared resources between the forwarder and applications. This is the thing I did not get through the whole discussion: what is the benefit of sharing resources? That said, is the InMemoryStorage proposal just a temporary solution or are we also making a statement about what we anticipate non-duplicating data storage on end hosts to look like? I would not take InMemoryStorage as a temporary solution. It represents one of the data management models, that is, apps should take care of their data, rather than relying on forwarder. I am not saying this is only model, therefore apps can make their own choice whether to adopt this model or not. This is also why I always take the in-app storage as a helper class in the library. I took Lixia's initial prompt to be a discussion of the latter, but it's very possible that I'm dragging the discussion to somewhere it was never intended to go. I was hoping to discuss the design a new/alternative CS residing within NFD that communicates with applications through a library API (hopefully similar to InMemoryStorage). The idea being to provide unified named data storage across the NDN app ecosystem that emulates some of the OS' storage functionality. This would obviously be *a lot* of work, but it makes a statement of "how things should be". One thing I do not agree with Lixia's argument is that this is not about resources, this is also about management. If there is enough resources, it does not matter where the data is stored. If there is not enough resources, neither putting data in CS nor in app cannot help. However, no matter where the data is stored, the holder should manage the data. The cost of managing apps' data in CS is much higher than asking apps to do it by themselves. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Mon Sep 15 21:49:57 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 15 Sep 2014 21:49:57 -0700 Subject: [Nfd-dev] do we need "application store"? In-Reply-To: <010b01cfd169$37830b60$a6892220$@ucla.edu> References: <6092B762-0A3C-4F56-9FCF-C47129E49159@cs.ucla.edu> <1B6E83E5-5A23-4FFA-A2F0-481904E2933B@cs.arizona.edu> <010AE88E-0AEA-4F3C-BF85-C79E9F1C2948@cs.colostate.edu> <5CEC0D7B-D334-4458-8AD2-217E5770239F@cs.ucla.edu> <254FFD08-C66B-4592-A0AF-0B71A3BE87BE@cs.colostate.edu> <1FBC85FD-14A2-4D29-9A7C-04E182FEABF0@cs.ucla.edu> <6C0BC12F-9FD5-4118-8A44-3FF4FD8FDAEC@cs.ucla.edu> <48AAF5EF-D183-43AB-A451-FCFAD6EB4A57@cs.ucla.edu> <90DDE875-1182-4B22-A083-36C6E63A8D1B@ucla.edu> <010b01cfd169$37830b60$a6892220$@ucla.edu> Message-ID: <906DB43C-F516-4996-8B82-420559080C30@cs.ucla.edu> On Sep 15, 2014, at 9:46 PM, Jiewen Tan wrote: > Also, there is a need from Junxiao that is to provide an interface: > size_t > eraseByPrefix(const Name& prefix); > to erase items stored in the in-memory storage by prefix. In the spec, the second method "erase(const Name& prefix)" is indeed the one Junxiao suggested. If we supply the full name of a data packet as the prefix, the method will delete the single data packet, therefore, there is no need to keep a separate erase method for single data packet. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From wentaoshang at gmail.com Mon Sep 15 21:54:19 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Mon, 15 Sep 2014 21:54:19 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <4104A1A1-F9AD-4A1D-8507-A237302ECA75@cs.ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> <62875D2B-821C-4225-927B-66382C686397@ucla.edu> <4104A1A1-F9AD-4A1D-8507-A237302ECA75@cs.ucla.edu> Message-ID: On Mon, Sep 15, 2014 at 9:19 PM, Yingdi Yu wrote: > > On Sep 15, 2014, at 5:20 PM, Wentao Shang wrote: > > > > On Mon, Sep 15, 2014 at 4:02 PM, Alex Afanasyev < > alexander.afanasyev at ucla.edu> wrote: > >> >> On Sep 15, 2014, at 3:50 PM, Lixia Zhang wrote: >> >> > 1/ I agree with Wentao on the following: >> > >> >> a link object "A -> B" involves two parties, i.e., A and B. Unless A >> and B are actually the same party, you need to have two signatures, one >> from A and one from B, to indicate that both parties have agreed on this >> link relationship. >> >> This is very general statement. On a surface, this could be a desired >> property. The whole question which goal is this property achieving? In my >> opinion, giving provider a tool to allow/deny hosting things inside the >> provider is much bigger harm then allowing anybody to express their desire >> for their client to try to request data from the specific provider. > > > I don't quite understand this argument: if the provider doesn't want to > host the client's content, the link relationship wouldn't have existed in > the first place. > > > No, it is not about hosting the client's content, the client still hosts > its own data from its own machine, but the machine is connected to the > internet through att's network. Basically, the client only bought the > internet access from att. If a link object signed by att is required to > publish the data, it implies that att can decide the type of data that the > client can publish. I think this violates the network neutrality, and > nobody would like that. > In this case, yes, double signature is not necessary because the link is pointing to the client's own data. As I said, double signature is only meaningful when the link tries to connect two different parties. > > The signature is only a technical means for the provider to express its > approval of hosting. Whether the provide *will* host or not sounds more > like a layer-10 issue to me. > > >> As I mentioned before, even if such property implemented in the LINK, how >> is it different from just sending of bunch of unsatisfiable Interests >> towards the provider? >> > > I'm not comfortable with the logic behind this argument: if there are two > backdoors that can lead to the same security hole, can we simply say that > we can leave the 2nd door open because the hackers are going to use the 1st > door anyway? > > > No, the logic is: if one of the backdoor is easier to be exploited to > launch an attack, what is the point of fixing the other backdoor (which is > more difficult to be exploited)? Attackers are not fools. > I don't have counter-argument to this one from technical perspective :) But at least this discussion should go into the "security considerations" section in the documentation. Wentao > > Yingdi > > -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.afanasyev at ucla.edu Mon Sep 15 21:54:47 2014 From: alexander.afanasyev at ucla.edu (Alex Afanasyev) Date: Mon, 15 Sep 2014 21:54:47 -0700 Subject: [Nfd-dev] do we need "application store"? In-Reply-To: <00fb01cfd168$309440e0$91bcc2a0$@ucla.edu> References: <6092B762-0A3C-4F56-9FCF-C47129E49159@cs.ucla.edu> <1B6E83E5-5A23-4FFA-A2F0-481904E2933B@cs.arizona.edu> <010AE88E-0AEA-4F3C-BF85-C79E9F1C2948@cs.colostate.edu> <5CEC0D7B-D334-4458-8AD2-217E5770239F@cs.ucla.edu> <254FFD08-C66B-4592-A0AF-0B71A3BE87BE@cs.colostate.edu> <1FBC85FD-14A2-4D29-9A7C-04E182FEABF0@cs.ucla.edu> <6C0BC12F-9FD5-4118-8A44-3FF4FD8FDAEC@cs.ucla.edu> <48AAF5EF-D183-43AB-A451-FCFAD6EB4A57@cs.ucla.edu> <90DDE875-1182-4B22-A083-36C6E63A8D1B@ucla.edu> <00fb01cfd168$309440e0$91bcc2a0$@ucla.edu> Message-ID: I did not say virtual class or method. These methods don't belong to the InMemoryStorage. What I meant is that this function is completely outside the scope of InMemoryStorage. There should be separate serialization hierarchy (there is boost serialization library, though I haven't used it at all). --- Alex On Sep 15, 2014, at 9:39 PM, Jiewen Tan wrote: > Hi Alex and Yingdi, > > I do agree with Alex that it might be more useful to make the save/load interfaces as virtual interfaces to provide more flexibility to the app developers. Yet I think it would be necessary to provide such interfaces, as some apps might need to save states when they exit and resume when they start. Although app developers can surely inherit the base in-memory storage class and do whatever they want in the derived one, it would be good to provide a base helper for lazy app developers and demonstrate an example to do serialization. > > Regards, > Jiewen Tan > > From: Alexander Afanasyev [mailto:cawka1 at gmail.com] On Behalf Of Alex Afanasyev > Sent: Monday, September 15, 2014 10:53 AM > To: nfd-dev > Cc: Yingdi Yu; Steve DiBenedetto; Lixia Zhang; Beichuan Zhang; Junxiao Shi; Jiewen Tan > Subject: Re: do we need "application store"? > > Hi Yingdi, > > I like the design. I would only suggest a couple of quick changes: > > - there is too much description about freshness. I think it would be enough to say that a primary objective of freshness is to guard producer from receiving too many interests. > > - I would avoid putting PIB service as a first example. Start with the existing and relatively known apps (chat/sync/nlsr). For PIB service, I wouldn't talk about "system default key", rather "current certificate for the key" and optionally "current key for the identity". > > - for the design goal, you should start with emphasizing that "common pattern" is to keep produced data packet for a while, so it can be served later. You already have all the description, I would just start with the "common pattern" statement and then go into more details. > > - "... packets are not backed up ..." this thing is commonly called serialization. Either "storage" is serialized on disk or each data packet in the storage is serialized on disk. > > - don't add save/load to the interface. This is completely unrelated to storage and, as you mentioned before, there could be tons of different ways to do it. If you simply provide `begin() const', `end() const' methods, I thing it would be enough to implement any kind of serialization. We can make a serialization helper, but it is still outside the scope of InMemoryStorage class. > > - Can you add a diagram/description on how applications would use this storage? I mean, description on how storage is connected within the application: app registers prefix, sets interest filter, receives interest, calls in memory storage, etc. > > --- > Alex > > On Sep 13, 2014, at 10:57 PM, Yingdi Yu wrote: > > > Hi all, > > I have updated the design document of InMemoryStorage: http://redmine.named-data.net/projects/ndn-cxx/wiki/InMemoryStorage. I would like to get feedback from you, so that we can nail down the design details, and Jiewen Tan can keep working on it. Thanks! > > Yingdi > > > > > On Sep 10, 2014, at 11:17 AM, Yingdi Yu wrote: > > > On Sep 10, 2014, at 9:47 AM, Steve DiBenedetto wrote: > > > As for using the CS for an application's data storage: I want understand the role of the CS in the special, but common, case of end hosts/local nodes/whatever-you-want to-call-them where we have vastly more storage capabilities, both memory and disk, than a router. Is the CS still just the store-and-forwarder buffer or is it something else? > > Yes, I also hope we can clarify the role of CS. > > > I'm trying to view the end host CS as master storage controller (probably a bad term) that conceptually holds everything on the system and presents a API for interacting with named data. > > In this case, why not just run a repo on the system. It can provide more persistent storage than CS. > > > In other words, something that evolves from just our "store-and-forward buffer with named data" mentioned in the original paper towards NDNFS' idea(s) (unfortunately not familiar with the details). > > I am not sure about NDNFS, maybe repo is a better one to describe your idea? > > > Then, all data storage is using shared resources between the forwarder and applications. > > This is the thing I did not get through the whole discussion: what is the benefit of sharing resources? > > > That said, is the InMemoryStorage proposal just a temporary solution or are we also making a statement about what we anticipate non-duplicating data storage on end hosts to look like? > > I would not take InMemoryStorage as a temporary solution. It represents one of the data management models, that is, apps should take care of their data, rather than relying on forwarder. I am not saying this is only model, therefore apps can make their own choice whether to adopt this model or not. This is also why I always take the in-app storage as a helper class in the library. > > > I took Lixia's initial prompt to be a discussion of the latter, but it's very possible that I'm dragging the discussion to somewhere it was never intended to go. > > I was hoping to discuss the design a new/alternative CS residing within NFD that communicates with applications through a library API (hopefully similar to InMemoryStorage). The idea being to provide unified named data storage across the NDN app ecosystem that emulates some of the OS' storage functionality. This would obviously be *a lot* of work, but it makes a statement of "how things should be". > > One thing I do not agree with Lixia's argument is that this is not about resources, this is also about management. If there is enough resources, it does not matter where the data is stored. If there is not enough resources, neither putting data in CS nor in app cannot help. > > However, no matter where the data is stored, the holder should manage the data. The cost of managing apps' data in CS is much higher than asking apps to do it by themselves. > > Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingdi at CS.UCLA.EDU Mon Sep 15 22:11:14 2014 From: yingdi at CS.UCLA.EDU (Yingdi Yu) Date: Mon, 15 Sep 2014 22:11:14 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> <62875D2B-821C-4225-927B-66382C686397@ucla.edu> <4104A1A1-F9AD-4A1D-8507-A237302ECA75@cs.ucla.edu> Message-ID: <307AB15A-84F3-4177-B7E7-563F9EA81D1C@cs.ucla.edu> On Sep 15, 2014, at 9:54 PM, Wentao Shang wrote: > On Mon, Sep 15, 2014 at 9:19 PM, Yingdi Yu wrote: > > On Sep 15, 2014, at 5:20 PM, Wentao Shang wrote: > >> On Mon, Sep 15, 2014 at 4:02 PM, Alex Afanasyev wrote: >> >> On Sep 15, 2014, at 3:50 PM, Lixia Zhang wrote: >> >> > 1/ I agree with Wentao on the following: >> > >> >> a link object "A -> B" involves two parties, i.e., A and B. Unless A and B are actually the same party, you need to have two signatures, one from A and one from B, to indicate that both parties have agreed on this link relationship. >> >> This is very general statement. On a surface, this could be a desired property. The whole question which goal is this property achieving? In my opinion, giving provider a tool to allow/deny hosting things inside the provider is much bigger harm then allowing anybody to express their desire for their client to try to request data from the specific provider. >> >> I don't quite understand this argument: if the provider doesn't want to host the client's content, the link relationship wouldn't have existed in the first place. > > No, it is not about hosting the client's content, the client still hosts its own data from its own machine, but the machine is connected to the internet through att's network. Basically, the client only bought the internet access from att. If a link object signed by att is required to publish the data, it implies that att can decide the type of data that the client can publish. I think this violates the network neutrality, and nobody would like that. > > In this case, yes, double signature is not necessary because the link is pointing to the client's own data. As I said, double signature is only meaningful when the link tries to connect two different parties. then what is the gain of the second signature in the second case? For those who just want to send the interests, they do not care whether the second signature exists or not. For the provider, they can simply discard the interests for the data that the provider does not host. I just do no think the second signature can mitigate the DDoS attack, but I do agree with Lixia that the second signature might be useful if some data consumers or intermediate routers want to know whether the data is indeed from the requested provider. Yingdi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From wentaoshang at gmail.com Mon Sep 15 22:33:06 2014 From: wentaoshang at gmail.com (Wentao Shang) Date: Mon, 15 Sep 2014 22:33:06 -0700 Subject: [Nfd-dev] LINK spec discussion In-Reply-To: <307AB15A-84F3-4177-B7E7-563F9EA81D1C@cs.ucla.edu> References: <27A77B58-9406-445E-8765-4EE022CF3ECF@ucla.edu> <9970891D-27D1-471D-9644-BC14DD42D0A9@cs.ucla.edu> <01735EE7-1C07-426F-B39A-521599A15B02@ucla.edu> <9E583A0D-A35E-42A9-9E47-8DCD7ED21B85@cs.ucla.edu> <62875D2B-821C-4225-927B-66382C686397@ucla.edu> <4104A1A1-F9AD-4A1D-8507-A237302ECA75@cs.ucla.edu> <307AB15A-84F3-4177-B7E7-563F9EA81D1C@cs.ucla.edu> Message-ID: On Mon, Sep 15, 2014 at 10:11 PM, Yingdi Yu wrote: > On Sep 15, 2014, at 9:54 PM, Wentao Shang wrote: > > On Mon, Sep 15, 2014 at 9:19 PM, Yingdi Yu wrote: > >> >> On Sep 15, 2014, at 5:20 PM, Wentao Shang wrote: >> >> On Mon, Sep 15, 2014 at 4:02 PM, Alex Afanasyev < >> alexander.afanasyev at ucla.edu> wrote: >> >>> >>> On Sep 15, 2014, at 3:50 PM, Lixia Zhang wrote: >>> >>> > 1/ I agree with Wentao on the following: >>> > >>> >> a link object "A -> B" involves two parties, i.e., A and B. Unless A >>> and B are actually the same party, you need to have two signatures, one >>> from A and one from B, to indicate that both parties have agreed on this >>> link relationship. >>> >>> This is very general statement. On a surface, this could be a desired >>> property. The whole question which goal is this property achieving? In my >>> opinion, giving provider a tool to allow/deny hosting things inside the >>> provider is much bigger harm then allowing anybody to express their desire >>> for their client to try to request data from the specific provider. >> >> >> I don't quite understand this argument: if the provider doesn't want to >> host the client's content, the link relationship wouldn't have existed in >> the first place. >> >> >> No, it is not about hosting the client's content, the client still hosts >> its own data from its own machine, but the machine is connected to the >> internet through att's network. Basically, the client only bought the >> internet access from att. If a link object signed by att is required to >> publish the data, it implies that att can decide the type of data that the >> client can publish. I think this violates the network neutrality, and >> nobody would like that. >> > > In this case, yes, double signature is not necessary because the link is > pointing to the client's own data. As I said, double signature is only > meaningful when the link tries to connect two different parties. > > > then what is the gain of the second signature in the second case? For > those who just want to send the interests, they do not care whether the > second signature exists or not. For the provider, they can simply discard > the interests for the data that the provider does not host. > At least this avoids providing *new* methods for DDoS attack. But, again, I don't disagree that such kind of attacks may not be practical in real life (as we see now) and it's not worth it to add such complexity at network packet level. BTW: given that I didn't participate in the early discussions, I'm curious how this "delegation object" was proposed in the first place and what the original purpose is. I'd like to check whether my understanding is consistent with the original author's. Wentao > > I just do no think the second signature can mitigate the DDoS attack, but > I do agree with Lixia that the second signature might be useful if some > data consumers or intermediate routers want to know whether the data is > indeed from the requested provider. > > Yingdi > > -- PhD @ IRL, CSD, UCLA -------------- next part -------------- An HTML attachment was scrubbed... URL: From lanwang at memphis.edu Mon Sep 22 07:05:38 2014 From: lanwang at memphis.edu (Lan Wang (lanwang)) Date: Mon, 22 Sep 2014 14:05:38 +0000 Subject: [Nfd-dev] Fwd: question about how to handle dangling state References: Message-ID: <2FB0F5C5-7F98-4291-A8BB-8BF0320035AD@memphis.edu> Junxiao, I saw you created Bug #1966: Interest unsatisfied due to similar Interest suppression combined with loop detection. This is basically the dangling state blocking similar Interest problem. There was some discussion more than two years ago exploring possible solutions (e.g., return data if the duplicate Interest is blocking other Interests and how to differentiate a blocking dangling state from a non-blocking one). I'm forwarding one of those emails. Lan Begin forwarded message: > From: Beichuan Zhang > Subject: Re: question about how to handle dangling state > Date: March 23, 2011 5:31:45 PM CDT > To: "Lan Wang (lanwang)" > Cc: Lixia Zhang > > > On Mar 23, 2011, at 2:01 PM, Lan Wang wrote: > >> >> On Mar 22, 2011, at 3:05 PM, Beichuan Zhang wrote: >> >>> Assuming all the problems in section 2 warrant interest return as a >>> necessary feedback mechanism, I'd re-phrase the question regarding >>> dangling state as: do we need a better solution (than interest >>> return) >>> to solve the dangling state problem? >>> >>> We can also ask the same question to every other problem listed in >>> section 2. What I'm afraid of is that we may end up with many >>> specific >>> mechanisms, each dealing with one specific problem, and altogether >>> making the system more complicated than using a single interest >>> return >>> mechanism. >>> >>> I have some comments inline below. >>> >>> On Mar 22, 2011, at 9:58 AM, Lan Wang wrote: >>> >>>> Hi Beichuan and Lixia, >>>> >>>> I have been thinking about the dangling state: >>>> >>>> First, if a particular piece of dangling state doesn't block other >>>> users from retrieving data (i.e. the state doesn't lead to the >>>> actual >>>> data or there's no other users sharing the path), it only adds to >>>> the >>>> memory consumption in routers. Is it a problem? I remember Van >>>> doesn't think the memory overhead *alone* is a problem. My answer >>>> would be it depends on whether there is enough dangling state to >>>> cause a memory problem in routers. For this we need more >>>> simulations. >>> >>> The memory overhead depends on how aggressive each node explores the >>> network and how long the interest lifetime is. >> >> Now I realize that in the case of dangling state not blocking other >> users, the memory overhead of dangling state is not the only problem >> under the current CCN's router-based retry scheme. When these >> dangling states timeout, they lead to unnecessary path explorations, >> even if data has been returned to the consumer over another path. >> However, this wouldn't be a problem in our consumer-based retry >> scheme, since the consumer will stop retrying when it gets the data. >> This is true without using Interest returns to clear up the dangling >> state (in other words, we don't have to use Interest return to solve >> the unnecessary path exploration problem in this case assuming memory >> overhead is not a problem). >> >> In the case of dangling state blocking other users, there may be >> solutions other than sending data back to duplicate Interests or >> returned Interests. See below. >>>> >>>> Second, what we care more about is when a dangling state does block >>>> other users to get their data. But the question is can we solve it >>>> without using Interest returns? One straight-forward solution to >>>> the >>>> example we gave in the writeup is to let the producer send back >>>> another copy of the data even if the Interest is a duplicate of a >>>> previously satisfied Interest. If we are dealing with dangling >>>> state that does block other users, sending the data is not a waste >>>> of bandwidth as it actually helps the other users get their data >>>> (actually faster than using Interest returns) and removes the >>>> dangling state. >>>> >>>> For the above solution, you may say that not only producers need to >>>> do this, routers need to do the same (return data to duplicate >>>> Interests). Yes, the question is whether that would lead to a >>>> problem. One potential problem I can think of is that the duplicate >>>> may indicate a looping Interest and, in that case, we shouldn't send >>>> data back into the loop. But not all the duplicate Interests are >>>> caused by loops -- it could be caused by branching out the same >>>> Interest along different paths and the Interests eventually meet >>>> at a >>>> single point (as in our example). I think what we need is to >>>> distinguish the duplicate Interests caused by looping and that >>>> caused >>>> by branching. For the former case, we shouldn't return any data. >>>> For the second case, we could return data. How to distinguish the >>>> two cases? One possible solution is for each router to append a >>>> hash >>>> value to an Interest using its own unique number and the nonce in >>>> the >>>> Interest. This way it can quickly recognize Interests that have >>>> looped back to itself and discard the Interests. There may be other >>>> solutions. >>> >>> Here's another case: Node A sends out an interest, and the interest >>> loops back to A. However, along the path that the interest has >>> traveled, it has suppressed some interests from other nodes. When A >>> receives this interest, it has no idea whether other interests are >>> being blocked or not. So if you're willing to live with the overhead >>> (see below), you may just return data anyway and not to differentiate >>> whether it's due to looping or branching. >>> >> Agree. >> >> Fundamentally, the problem of dangling state blocking other users has >> several contributing factors: path exploration, state aggregation, >> etc. We can look at the state aggregation factor. Basically, in >> order to avoid sending data unnecessarily (as in the above approach), >> we need to differentiate dangling state that is blocking other users >> from dangling state that's not blocking other users. So suppose in >> the example in our writeup, D has already forwarded A's Interest >> toward its upstream and then it receives the second Interest from J >> for the same data, D can immediately send another Interest upstream >> with a flag that says this Interest actually represents multiple >> consumers' requests. Therefore, whoever receives the Interest will >> not simply drop it as a duplicate based on the nonce (which is >> meaningless now). Note that this only needs to happen once. Even >> if D receives other consumers' requests for the data, it doesn't need >> to send another Interest upstream. This scheme will not respond >> with data if A's interest doesn't block J's interest. >> >> Does sending the second Interest incur additional overhead compared >> to the Interest return scheme? No. Because in the Interest return >> scheme, we still need to forward J's interest after clearing up A's >> Interest, so there's no more message overhead. Moreover, the data >> will come back faster than the Interest return scheme. > > Instead of using a special flag, D can make up a new nonce and send > the interest. D only does this when the number of pending interests > (for the same name) increases from 1 to 2. This is compatible with > other parts of the current system and no other changes are needed. > > However, this doesn't completely remove the overhead, because A will > receive the second copy of the data from D. > > This doesn't fix the excessive path exploration (when there's no > blocked interests) as you pointed out above. > >>> >>>> The only downside as I can see is that perhaps some data >>>> will be sent along paths that have dangling state that doesn't >>>> actually block other users. The data may still be useful for future >>>> users though. >>> >>> This overhead is my main concern about returning data vs. returning >>> interests. When there's no blocked interests, the overhead of >>> replying >>> data is extra bandwidth and cache space. So exploring paths becomes >>> more expensive. A node needs to be frugal in sending interests out >>> via >>> different faces. >>> >>> Ideally whether a data stays in the cache should reflect the data's >>> popularity, i.e., the number of consumers that want the data. Now if >>> we return data to duplicate interests, then it will reflect how many >>> interests a consumer sends out. So the more-aggressively sought >>> data may >>> take the cache space of the more popular data, which reduces the >>> effectiveness of cache. >> >> The new scheme described above addresses the data overhead problem. >> The data is only returned when it is likely to reach consumers that >> still want the data. >> >>> >>>> >>>> So I think we need to reach a common understanding on whether >>>> returns >>>> are absolutely necessary in removing the dangling state. >>>> >>>> Another dimension is to prevent dangling state in the first place. >>>> What are the possible mechanisms there? If the prevention is good >>>> enough, do we still need to worry about removing the dangling state? >>> >>> by prevention, do you mean something like forwarding all interests >>> without suppression? I cannot think of anything else right now. >> >> By prevention, I mean limiting "unnecessary path exploration" (the >> consumer already has the data, but there is state in the network that >> causes unnecessary path exploration", and "aggressive path >> exploration" (the consumer sends duplicate interests along too many >> paths). The first one can be limited by using consumer-based retry >> instead of router-based retry. The second one I think needs to be >> limited somehow by the network. This is like dealing with a denial- >> of-service problem. >> >> Lan >> > From shijunxiao at email.arizona.edu Mon Sep 29 15:46:50 2014 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Mon, 29 Sep 2014 15:46:50 -0700 Subject: [Nfd-dev] Strategy for testbed routers Message-ID: Dear folks This message talks about the necessity of new/improved forwarding strategies on testbed routers, argues that characteristics for local site prefix and other prefixes are different and need two strategies, and presents the requirements for each strategy. This message's *purpose is to agree on the requirements. *This is not a design. Requirements must be defined before a design can be made. Please give me comments. Necessity Recent experience about application performance on testbed suggests that existing strategies are insufficient to support all application scenarios. The most notable example is NDN-RTC: NCC strategy suppresses all consumer retransmissions so that any packet loss prevents playout of a video frame; best-route strategy cannot use multiple paths so that multiple connected laptops on the same HUB prevents video producer from being found efficiently. Therefore, we need new/improved forwarding strategies to use on testbed routers. Local site prefix vs Remote prefixes Two types of prefixes are reachable on a given testbed router: A local site prefix (eg. /ndn/edu/arizona on ARIZONA router) is served by local applications on the router, and unmanaged nodes or laptops connected to the router. Several remote prefixes (eg. /ndn/fr/lip6 on ARIZONA router) can be reached by forwarding the Interest to other routers, where the path is learnt from routing protocols. The characteristics of these two types of prefixes are different. A sub-namespace under the *local site prefix* (eg. /ndn/edu/arizona/ndnrtc/session1) is only reachable via a small subset of nexthops installed in the FIB entry, due to the usage of nfd-autoreg-server. Even after we stop using nfd-autoreg-server, a sub-namespace under the local site prefix may become unreachable from a previously working nexthop due to mobility. A sub-namespace under a *remote prefix* is usually reachable via all nexthops installed in the FIB entry, but the performance may vary between those nexthops. A sub-namespace under a remote prefix may become unreachable from a previously working nexthop only if there is a link failure. Backbone router strategy and Access router strategy As a consequence of different characteristics of local site prefix and remote prefixes, I propose that two strategies should be designed. On each testbed router, the access router strategy should be assigned to the local site prefix, while the backbone router strategy should be assigned to all remote prefixes. On ARIZONA router, this yields a setup like: the root namespace uses backbone router strategy, /ndn/edu/arizona uses access router strategy. Please remember that strategy choice is inherited to sub namespaces. Setting backbone router strategy at the root namespace ensures all remote prefixes are using the backbone router strategy. The only exception is when a remote prefix is contained within the local site prefix, which happens on UCLA router, that needs: the root namespace and /ndn/edu/ucla/remap uses backbone router strategy, /ndn/edu/ucla uses access router strategy. I identify the requirements of each strategy as: *Backbone router strategy should have the following features*: - Make use of multiple paths in FIB entry, and learn the best path. - In the normal case, it's expected that all paths can reach the destination. - Recovery from packet loss in wide-area tunnels. - Consumer retransmission is a hint for strategy to retransmit. *Access router strategy should have the following features*: - Make use of multiple paths in FIB entry, and remember which path can lead to contents. - The paths could be installed by nfd-autoreg-server, so that some paths cannot reach the producer. - Recovery from packet loss in last-hop link. - Consumer retransmission is a hint for strategy to retransmit. - Expect producer mobility. Yours, Junxiao -------------- next part -------------- An HTML attachment was scrubbed... URL: From lanwang at memphis.edu Tue Sep 30 10:00:53 2014 From: lanwang at memphis.edu (Lan Wang (lanwang)) Date: Tue, 30 Sep 2014 17:00:53 +0000 Subject: [Nfd-dev] Strategy for testbed routers In-Reply-To: References: Message-ID: Junxiao, Just a few questions and comments: 1. I remember the problem NDN-RTC had is the following: they have two producers connected to the same HUB, but the autoreg server gives them the same prefix even though they serve different content (each producer serves the data from a different camera). Therefore, the forwarding strategy routes all the Interests to one of those producers (best route) or oscillates between the two producers (ncc). They have to use broadcast strategy to get this work. If this is the problem you are referring to, then the problem is not the forwarding strategy, but the lack of a way for the laptops to register their own prefixes with the hub (under the hub's prefix). Is this the problem you are referring to? 2. Can you explain a bit the relationship between "NCC strategy suppresses all consumer retransmissions" and "any packet loss prevents playout of a video frame"? 3. Even on the backbone, you cannot be sure that all the available next hops in the FIB should lead you to the destination. This really depends on the routing protocol. If hyperbolic routing is used, it's possible that some of the next hops do not lead to the destination (due to local minima) and we have observed this. 4. Given the above, I do not feel that there is any major difference between the requirements for remote strategy and local strategy. The following two seem to be enough: > ? Make use of multiple paths in FIB entry, and learn the best path. > ? Recovery from packet loss Lan On Sep 29, 2014, at 5:46 PM, Junxiao Shi wrote: > Dear folks > > This message talks about the necessity of new/improved forwarding strategies on testbed routers, argues that characteristics for local site prefix and other prefixes are different and need two strategies, and presents the requirements for each strategy. > This message's purpose is to agree on the requirements. This is not a design. Requirements must be defined before a design can be made. > Please give me comments. > > Necessity > Recent experience about application performance on testbed suggests that existing strategies are insufficient to support all application scenarios. > The most notable example is NDN-RTC: NCC strategy suppresses all consumer retransmissions so that any packet loss prevents playout of a video frame; best-route strategy cannot use multiple paths so that multiple connected laptops on the same HUB prevents video producer from being found efficiently. > Therefore, we need new/improved forwarding strategies to use on testbed routers. > > Local site prefix vs Remote prefixes > Two types of prefixes are reachable on a given testbed router: > A local site prefix (eg. /ndn/edu/arizona on ARIZONA router) is served by local applications on the router, and unmanaged nodes or laptops connected to the router. > Several remote prefixes (eg. /ndn/fr/lip6 on ARIZONA router) can be reached by forwarding the Interest to other routers, where the path is learnt from routing protocols. > > The characteristics of these two types of prefixes are different. > > A sub-namespace under the local site prefix (eg. /ndn/edu/arizona/ndnrtc/session1) is only reachable via a small subset of nexthops installed in the FIB entry, due to the usage of nfd-autoreg-server. Even after we stop using nfd-autoreg-server, a sub-namespace under the local site prefix may become unreachable from a previously working nexthop due to mobility. > > A sub-namespace under a remote prefix is usually reachable via all nexthops installed in the FIB entry, but the performance may vary between those nexthops. > A sub-namespace under a remote prefix may become unreachable from a previously working nexthop only if there is a link failure. > > Backbone router strategy and Access router strategy > As a consequence of different characteristics of local site prefix and remote prefixes, I propose that two strategies should be designed. > On each testbed router, the access router strategy should be assigned to the local site prefix, while the backbone router strategy should be assigned to all remote prefixes. > On ARIZONA router, this yields a setup like: the root namespace uses backbone router strategy, /ndn/edu/arizona uses access router strategy. > Please remember that strategy choice is inherited to sub namespaces. Setting backbone router strategy at the root namespace ensures all remote prefixes are using the backbone router strategy. > The only exception is when a remote prefix is contained within the local site prefix, which happens on UCLA router, that needs: the root namespace and /ndn/edu/ucla/remap uses backbone router strategy, /ndn/edu/ucla uses access router strategy. > > I identify the requirements of each strategy as: > > Backbone router strategy should have the following features: > ? Make use of multiple paths in FIB entry, and learn the best path. > ? In the normal case, it's expected that all paths can reach the destination. > ? Recovery from packet loss in wide-area tunnels. > ? Consumer retransmission is a hint for strategy to retransmit. > > Access router strategy should have the following features: > ? Make use of multiple paths in FIB entry, and remember which path can lead to contents. > ? The paths could be installed by nfd-autoreg-server, so that some paths cannot reach the producer. > ? Recovery from packet loss in last-hop link. > ? Consumer retransmission is a hint for strategy to retransmit. > ? Expect producer mobility. > > Yours, Junxiao > _______________________________________________ > Nfd-dev mailing list > Nfd-dev at lists.cs.ucla.edu > http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev