From nfd-call-notification at mail1.yoursunny.com Thu Sep 1 07:00:03 2016 From: nfd-call-notification at mail1.yoursunny.com (NFD call notification) Date: Thu, 1 Sep 2016 07:00:03 -0700 Subject: [Nfd-dev] NFD call 20160901 Message-ID: <201609011400.u81E035L007542@lectura.cs.arizona.edu> An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Mon Sep 5 20:49:04 2016 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Mon, 5 Sep 2016 20:49:04 -0700 Subject: [Nfd-dev] ndn-cxx deprecated feature removal: LocalControlHeader Message-ID: Dear folks The following deprecated feature will be removed from ndn-cxx soon: ndn::nfd::LocalControlHeader Interest::getLocalControlHeader Interest::getIncomingFaceId Interest::setIncomingFaceId Interest::getNextHopFaceId Interest::setNextHopFaceId Data::getLocalControlHeader Data::getIncomingFaceId Data::setIncomingFaceId Data::getCachingPolicy Data::setCachingPolicy interest.hpp includes data.hpp includes If you have code that depend on this feature and cannot be fixed now, please reply no later than Sep 09. Otherwise, we'll proceed with the removal after that time. You can compile your projects with ndn-cxx Change https://gerrit.named-data.net/3133 and verify it works. I have already prepared patches for these projects: NFD https://gerrit.named-data.net/3164 traffic-generator https://gerrit.named-data.net/3163 If you are a developer in one of these projects, please review the patch on Gerrit. Thanks. I have also confirmed these projects are unaffected: NLSR, ChronoChat, ChronoSync, ndns, repo-ng, ndn-tools. Yours, Junxiao -------------- next part -------------- An HTML attachment was scrubbed... URL: From nfd-call-notification at mail1.yoursunny.com Tue Sep 6 07:00:02 2016 From: nfd-call-notification at mail1.yoursunny.com (NFD call notification) Date: Tue, 6 Sep 2016 07:00:02 -0700 Subject: [Nfd-dev] NFD call 20160906 Message-ID: <201609061400.u86E02km030970@lectura.cs.arizona.edu> An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Wed Sep 7 22:23:51 2016 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Thu, 8 Sep 2016 05:23:51 +0000 Subject: [Nfd-dev] Troubleshooting interest path Message-ID: Hi, I have a simple topology like this: [cid:image001.png at 01D20956.828F9110] Where NFD1 is the forwarder for /ndn/edu/ucla and NFD2 is the forwarder for the subnamespace /ndn/edu/ucla/jburke/golem. I am using ndnputfile to store information in the repo and then trying to retrieve it from the consumer. Retrieval times out for the consumer node but not for things directly connected to NFD2, as one would hope. I doubt my problem is complicated. But in trying to figure out why this doesn?t work, I am curious what the state-of-the-art is for tracing Interest/Data exchange? If I want to see whether an Interest expressed by Consumer hits the Repo, is my only solution to watch the logs in NFD2 (or for anything potentially in the path, in a larger example), keep track of which face Repo corresponds to and see if things are forwarded? Repo-ng doesn?t seem to say/log much itself.. but more generally the question here is about whether there are techniques to do Interest/Data packet tracing without manually- or custom- instrumenting each potential node in the path(s). Thanks, Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 11659 bytes Desc: image001.png URL: From davide.pesavento at lip6.fr Thu Sep 8 02:33:44 2016 From: davide.pesavento at lip6.fr (Davide Pesavento) Date: Thu, 8 Sep 2016 11:33:44 +0200 Subject: [Nfd-dev] Troubleshooting interest path In-Reply-To: References: Message-ID: On Thu, Sep 8, 2016 at 7:23 AM, Burke, Jeff wrote: > I doubt my problem is complicated. But in trying to figure out why this > doesn?t work, I am curious what the state-of-the-art is for tracing > Interest/Data exchange? If I want to see whether an Interest expressed by > Consumer hits the Repo, is my only solution to watch the logs in NFD2 (or > for anything potentially in the path, in a larger example), keep track of > which face Repo corresponds to and see if things are forwarded? Repo-ng > doesn?t seem to say/log much itself.. but more generally the question here > is about whether there are techniques to do Interest/Data packet tracing > without manually- or custom- instrumenting each potential node in the > path(s). > > > Hi Jeff, You can use ndndump, or you can capture a trace in pcap format and then read it with Wireshark + the NDN dissector. Both are available in the ndn-tools repository. Thanks, Davide -------------- next part -------------- An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Thu Sep 8 06:54:20 2016 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Thu, 8 Sep 2016 13:54:20 +0000 Subject: [Nfd-dev] Troubleshooting interest path In-Reply-To: References: Message-ID: Hi Davide, Yes, but doesn?t this approach require: 1) pcap access on each (potential) node in the path (I have this on some but not all) 2) capturing potentially a lot of traffic (if on, say, a testbed node and I did have access) 3) then having to gather all capture records from the various hosts and filter on names/nonces That?s what I meant by manually instrumenting each node, which I can do for some nodes, but not the testbed nodes. It also seems to require a lot of instrumentation and post-gathering/-filtering (and access) effort that grows quickly as the number of nodes increase. I guess what I?m looking for is whether anyone has returned to work on something that looks a little more like traceroute in its usage? Thanks, Jeff From: Davide Pesavento Date: Thursday, September 8, 2016 at 2:33 AM To: Jeff Burke Cc: "nfd-dev at lists.cs.ucla.edu" Subject: Re: [Nfd-dev] Troubleshooting interest path On Thu, Sep 8, 2016 at 7:23 AM, Burke, Jeff > wrote: I doubt my problem is complicated. But in trying to figure out why this doesn?t work, I am curious what the state-of-the-art is for tracing Interest/Data exchange? If I want to see whether an Interest expressed by Consumer hits the Repo, is my only solution to watch the logs in NFD2 (or for anything potentially in the path, in a larger example), keep track of which face Repo corresponds to and see if things are forwarded? Repo-ng doesn?t seem to say/log much itself.. but more generally the question here is about whether there are techniques to do Interest/Data packet tracing without manually- or custom- instrumenting each potential node in the path(s). Hi Jeff, You can use ndndump, or you can capture a trace in pcap format and then read it with Wireshark + the NDN dissector. Both are available in the ndn-tools repository. Thanks, Davide -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: topo.png Type: image/png Size: 4490 bytes Desc: topo.png URL: From nfd-call-notification at mail1.yoursunny.com Thu Sep 8 07:00:02 2016 From: nfd-call-notification at mail1.yoursunny.com (NFD call notification) Date: Thu, 8 Sep 2016 07:00:02 -0700 Subject: [Nfd-dev] NFD call 20160908 Message-ID: <201609081400.u88E02C5022570@lectura.cs.arizona.edu> An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Thu Sep 8 07:18:01 2016 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Thu, 8 Sep 2016 14:18:01 +0000 Subject: [Nfd-dev] Clarification of 'repo' definition Message-ID: Hi, (Sorry for these messages all at once, and let me know if this should go to ndn-interest.) Through an earlier conversation with Lixia this week, I realized that the definition / expectations of a ?repo? may be more specific than I previously understood. In particular, she mentioned that repos storing and registering the same namespace should (eventually) have the same data for that namespace, presumably via sync. A few related questions: 1) Is this attempt at eventual consistency of what?s stored for a given namespace a requirement to be called a ?repo?? 2) Practically, if such sync is ultimately required behavior for a repo, will/should repo-ng itself incorporate some basic synchronization features? 3) Does opportunistic caching behavior (without sync) mean that the storage is not a ?repo?? 4) Would, for example, network-attached storage that stores everything for a prefix but only up to a given depth in the tree not qualify as a repo? 5) Or, for example, in the BMS case, if I use a repo to store all of the electrical current samples for the UCLA campus, but not chilled water, it will have only have some of the tree for the campus bms prefix. Is the storage not a repo? Should it not be registering the root bms prefix? Should I have / what do I call storage that is filling in part of the tree but don?t need to or can?t store all of it? 6) What are other requirements to be a ?repo?? (Alternatively, is there a canonical reference in the literature for the type of storage that constitutes a repo?) Thanks for any clarification... Maybe I am reading too much into this? Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: From davide.pesavento at lip6.fr Thu Sep 8 07:25:17 2016 From: davide.pesavento at lip6.fr (Davide Pesavento) Date: Thu, 8 Sep 2016 16:25:17 +0200 Subject: [Nfd-dev] Troubleshooting interest path In-Reply-To: References: Message-ID: Oh I see... I misunderstood your question then. I am not aware of any traceroute-like tools for NDN. Thanks, Davide On Thu, Sep 8, 2016 at 3:54 PM, Burke, Jeff wrote: > Hi Davide, > > > > Yes, but doesn?t this approach require: > > > > 1) pcap access on each (potential) node in the path (I have this on some > but not all) > > 2) capturing potentially a lot of traffic (if on, say, a testbed node and > I did have access) > > 3) then having to gather all capture records from the various hosts and > filter on names/nonces > > > > That?s what I meant by manually instrumenting each node, which I can do > for some nodes, but not the testbed nodes. It also seems to require a lot > of instrumentation and post-gathering/-filtering (and access) effort that > grows quickly as the number of nodes increase. I guess what I?m looking > for is whether anyone has returned to work on something that looks a little > more like traceroute in its usage? > > > > Thanks, > > Jeff > > > > > > *From: *Davide Pesavento > *Date: *Thursday, September 8, 2016 at 2:33 AM > *To: *Jeff Burke > *Cc: *"nfd-dev at lists.cs.ucla.edu" > *Subject: *Re: [Nfd-dev] Troubleshooting interest path > > > > On Thu, Sep 8, 2016 at 7:23 AM, Burke, Jeff wrote: > > I doubt my problem is complicated. But in trying to figure out why this > doesn?t work, I am curious what the state-of-the-art is for tracing > Interest/Data exchange? If I want to see whether an Interest expressed by > Consumer hits the Repo, is my only solution to watch the logs in NFD2 (or > for anything potentially in the path, in a larger example), keep track of > which face Repo corresponds to and see if things are forwarded? Repo-ng > doesn?t seem to say/log much itself.. but more generally the question here > is about whether there are techniques to do Interest/Data packet tracing > without manually- or custom- instrumenting each potential node in the > path(s). > > > > > > Hi Jeff, > > > > You can use ndndump, or you can capture a trace in pcap format and then > read it with Wireshark + the NDN dissector. Both are available in the > ndn-tools repository. > > > > Thanks, > > Davide > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdd at wustl.EDU Thu Sep 8 12:04:33 2016 From: jdd at wustl.EDU (Dehart, John) Date: Thu, 8 Sep 2016 19:04:33 +0000 Subject: [Nfd-dev] nNameTreeEntries revisited Message-ID: <996FC59B-6005-46BF-A03B-800463DBF106@wustl.edu> All: I have finally gotten back to trying to complete some monitoring tasks on the NDN Testbed. In doing so I noticed an interesting occurence yesterday. I am building up a set of Cacti graphs of monitored data from the NDN Testbed. A while back we had noticed that the nNameTreeEntries counter sometimes grew without ever shrinking back down. I am still seeing this. Here are the graphs from Arizona, Memphis and UCLA from the last 24 hours: [cid:AC99EC17-6A6E-4AB6-845C-B94CE8578006 at seas.wustl.edu] [cid:8BA1E283-BAD6-4E43-A0B9-D93059811C31 at seas.wustl.edu] [cid:5E93F324-1302-4E62-986C-FE2B40C750BD at seas.wustl.edu] As you can see the NameTree entries grows on Arizona and Memphis but does not come back down. I believe we expect it to be more like what we see on UCLA where we get bumps that then come back down. Also, we have other nodes like the WU node where we have this: [cid:1B31EE27-7A52-4B90-8046-8CBB999DB4F8 at seas.wustl.edu] Here we see steady grown over a week long period. If you are interested in looking at these and other graphs you can find them here: http://ndndemo.arl.wustl.edu/cacti/index.php login: guest pw: ndnTest Any thoughts or suggestions? John -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-8.png Type: image/png Size: 26008 bytes Desc: PastedGraphic-8.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-6.png Type: image/png Size: 26303 bytes Desc: PastedGraphic-6.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-7.png Type: image/png Size: 26921 bytes Desc: PastedGraphic-7.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-9.png Type: image/png Size: 20650 bytes Desc: PastedGraphic-9.png URL: From shijunxiao at email.arizona.edu Thu Sep 8 12:28:08 2016 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Thu, 8 Sep 2016 12:28:08 -0700 Subject: [Nfd-dev] Troubleshooting interest path In-Reply-To: References: Message-ID: Hi Jeff As Davide has already answered, ndndump and NDN packet dissector for Wireshark are the best available tools. If an application lacks logging capability (repo-ng in your example), there are two techniques to observe packets to and from this application: - Force the application to connect to NFD over TCP. In ndn-cxx, you can set environ NDN_CLIENT_TRANSPORT=tcp4://127.0.0.1:6363 to change the default to TCP. Then, you can use ndndump or Wireshark to capture packets on the loopback interface. - Rename the Unix socket of NFD in nfd.conf. Have a "proxy" program listen on /var/run/nfd.sock, log all traffic to a file and also forward them to NFD's socket. socat can be used as the proxy. The traffic log can then be analyzed with ndn-dissect . CCNx-Trace is a traceroute tool for NDN platform 0.2 and earlier. When we design NFD, the authors of CCNx-Trace confirm that CCNx-Trace can be ported to NFD without special support from NFD. As per my understanding of CCNx-Trace, this claim is true. However, even if CCNx-Trace is ported to NFD, it's ineffective in revealing Interest/Data paths. How CCNx-Trace works when you run CCNx-Trace client on Arizona router to find path to UCLA: (I'm explaining with NFD terminology although the tool is implemented for CCNx) 1. Client expresses an Interest /trace/ndn/edu/ucla/app/. 2. Trace daemon on Arizona (listening on /trace) receives this Interest, requests the FIB dataset, and finds that a longest prefix match for /ndn/edu/ucla/app would match FIB entry /ndn/edu/ucla. This FIB entry has two nexthops: REMAP cost=26, CAIDA cost=28. 3. Trace daemon sends an Interest /trace/ndn/edu/ucla/app/ and uses NextHopFaceId to direct this Interest to REMAP. 4. Trace daemon on REMAP does the same as step2-3. 5. Trace daemon on UCLA router receives step4 Interest from REMAP, and finds that /ndn/edu/ucla/app points to a local producer. Thus, it responds with a Data containing "spurs". 6. Trace daemon on REMAP responds to step3 Interest with a Data containing "128.97.98.7,spurs". 7. Trace daemon on Arizona responds to step1 Interest with a Data containing "hobo,128.97.98.7,spurs". 8. Client displays the collected path from step7. CCNx-Trace can only reveal the path indicates in FIB entry as lowest cost. It does not necessarily reflect the path taken by actual application Interests, because it does not take into consideration: - Forwarding strategy may send Interests towards a different nexthop. - ContentStore may satisfy an Interest, so that the Interest doesn't go all the way to producer machine. Current NDN architecture does not support true traceroute. It requires sending a normal application Interest, and collecting router identities along the reverse path taken by Data or Nack. Revealing router identities (including producer machine's identity) is in violation of NDN's privacy benefits. Yours, Junxiao On Wed, Sep 7, 2016 at 10:23 PM, Burke, Jeff wrote: > I am curious what the state-of-the-art is for tracing Interest/Data > exchange? If I want to see whether an Interest expressed by Consumer hits > the Repo, is my only solution to watch the logs in NFD2 (or for anything > potentially in the path, in a larger example), keep track of which face > Repo corresponds to and see if things are forwarded? Repo-ng doesn?t > seem to say/log much itself.. but more generally the question here is about > whether there are techniques to do Interest/Data packet tracing without > manually- or custom- instrumenting each potential node in the path(s). > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shijunxiao at email.arizona.edu Thu Sep 8 12:31:18 2016 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Thu, 8 Sep 2016 12:31:18 -0700 Subject: [Nfd-dev] nNameTreeEntries revisited In-Reply-To: <996FC59B-6005-46BF-A03B-800463DBF106@wustl.edu> References: <996FC59B-6005-46BF-A03B-800463DBF106@wustl.edu> Message-ID: Hi John This issue has been fixed in #3619 , but it's not deployed. The patch will be included in NFD 0.5.0 release scheduled on Sep 30. Yours, Junxiao On Thu, Sep 8, 2016 at 12:04 PM, Dehart, John wrote: > > All: > > I have finally gotten back to trying to complete some monitoring tasks on > the NDN Testbed. > In doing so I noticed an interesting occurence yesterday. > > I am building up a set of Cacti graphs of monitored data from the NDN > Testbed. > A while back we had noticed that the nNameTreeEntries counter sometimes > grew > without ever shrinking back down. I am still seeing this. > > Here are the graphs from Arizona, Memphis and UCLA from the last 24 hours: > > > > > > > As you can see the NameTree entries grows on Arizona and Memphis but does > not come back down. > I believe we expect it to be more like what we see on UCLA where we get > bumps that then come back down. > > Also, we have other nodes like the WU node where we have this: > > > Here we see steady grown over a week long period. > > If you are interested in looking at these and other graphs you can find > them here: > > http://ndndemo.arl.wustl.edu/cacti/index.php > > login: guest > pw: ndnTest > > > Any thoughts or suggestions? > > John > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-6.png Type: image/png Size: 26303 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-9.png Type: image/png Size: 20650 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-7.png Type: image/png Size: 26921 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-8.png Type: image/png Size: 26008 bytes Desc: not available URL: From jdd at wustl.EDU Thu Sep 8 12:34:38 2016 From: jdd at wustl.EDU (Dehart, John) Date: Thu, 8 Sep 2016 19:34:38 +0000 Subject: [Nfd-dev] nNameTreeEntries revisited In-Reply-To: References: <996FC59B-6005-46BF-A03B-800463DBF106@wustl.edu> Message-ID: <38A6BD86-9BB4-4CE9-8D8F-7F5DC589620C@wustl.edu> Junxiao, I thought we had but I searched for that and couldn?t find it. Thanks. John On Sep 8, 2016, at 2:31 PM, Junxiao Shi > wrote: Hi John This issue has been fixed in #3619, but it's not deployed. The patch will be included in NFD 0.5.0 release scheduled on Sep 30. Yours, Junxiao On Thu, Sep 8, 2016 at 12:04 PM, Dehart, John > wrote: All: I have finally gotten back to trying to complete some monitoring tasks on the NDN Testbed. In doing so I noticed an interesting occurence yesterday. I am building up a set of Cacti graphs of monitored data from the NDN Testbed. A while back we had noticed that the nNameTreeEntries counter sometimes grew without ever shrinking back down. I am still seeing this. Here are the graphs from Arizona, Memphis and UCLA from the last 24 hours: As you can see the NameTree entries grows on Arizona and Memphis but does not come back down. I believe we expect it to be more like what we see on UCLA where we get bumps that then come back down. Also, we have other nodes like the WU node where we have this: Here we see steady grown over a week long period. If you are interested in looking at these and other graphs you can find them here: http://ndndemo.arl.wustl.edu/cacti/index.php login: guest pw: ndnTest Any thoughts or suggestions? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From lixia at CS.UCLA.EDU Thu Sep 8 17:08:07 2016 From: lixia at CS.UCLA.EDU (Lixia Zhang) Date: Thu, 8 Sep 2016 17:08:07 -0700 Subject: [Nfd-dev] Clarification of 'repo' definition In-Reply-To: References: Message-ID: <18E0FAE4-89B4-4199-9403-06E8E80F8F2D@cs.ucla.edu> > On Sep 8, 2016, at 7:18 AM, Burke, Jeff wrote: > > Hi, > > (Sorry for these messages all at once, and let me know if this should go to ndn-interest.) > > Through an earlier conversation with Lixia this week, I realized that the definition / expectations of a ?repo? may be more specific than I previously understood. In particular, she mentioned that repos storing and registering the same namespace should (eventually) have the same data for that namespace, presumably via sync. I'll start by saying that this is all researchy: we are walking into a new territory that we are yet to fully explore. So what I said, before or here, is just to share ideas with others. It is easy to see why a desire of all repos for the same name prefix being sync'ed up: an interest for that prefix may be forwarded to any of the repos, if all repos have the same data, anyone can answer it; otherwise routers have to handle NACKs and reroute the interest to try other repos. > A few related questions: > > 1) Is this attempt at eventual consistency of what?s stored for a given namespace a requirement to be called a ?repo?? see the above explanation. I wont call it a requirement, but desirable for performance concerns. > 2) Practically, if such sync is ultimately required behavior for a repo, will/should repo-ng itself incorporate some basic synchronization features? as I mentioned, there was a piece of work done on repo sync 2 years back (but I dont know its implementation status) > 3) Does opportunistic caching behavior (without sync) mean that the storage is not a ?repo?? My own view: no. my explanation to others: - caching: opportunistic - (managed) repo: managed storage > 4) Would, for example, network-attached storage that stores everything for a prefix but only up to a given depth in the tree not qualify as a repo? everything is attached to the network :) so I am not sure what "network-attached storage" really means. if your question is "what if a managed repo for a name prefix only contains incomplete data under that prefix" -- that still works, and *may* even work OK if one understands the traffic patterns so that most of the Interests forwarded to that repo can be satisfied, without having to zigzagging to other places seeking for data. > 5) Or, for example, in the BMS case, if I use a repo to store all of the electrical current samples for the UCLA campus, but not chilled water, it will have only have some of the tree for the campus bms prefix. Is the storage not a repo? Should it not be registering the root bms prefix? Should I have / what do I call storage that is filling in part of the tree but don?t need to or can?t store all of it? 1/ if chilled water has its own prefix announcement, maybe one can find a way to attract all interests for chilled water data to the place for chilled water. 3/ again one needs to think not just what one wants to put where, but also how forwarding can work well. > 6) What are other requirements to be a ?repo?? (Alternatively, is there a canonical reference in the literature for the type of storage that constitutes a repo?) it does not really matter what we want to call something. there is no arbitrary requirement. as I said already, this is research, we have not done much playing with repo up to now. It is all about figuring out how to make the system work in the best way it can. my 2 cents, Lixia -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.tschudin at unibas.ch Thu Sep 8 23:05:03 2016 From: christian.tschudin at unibas.ch (christian.tschudin at unibas.ch) Date: Fri, 9 Sep 2016 08:05:03 +0200 (CEST) Subject: [Nfd-dev] Clarification of 'repo' definition In-Reply-To: <18E0FAE4-89B4-4199-9403-06E8E80F8F2D@cs.ucla.edu> References: <18E0FAE4-89B4-4199-9403-06E8E80F8F2D@cs.ucla.edu> Message-ID: The chilled water example is a nice study case, the two research questions that come to mind are NACK and mount: - asserting the non-existence of an item in distibuted systems is notoriously hard, in this case across repos or caches - mounting a tree into the routing fabric, with the possibility to delegate mount points to one or more (!) repos, is exactly what publishing is about. Having generations of repos with increasing sets of features would be useful to have (and the routing layer being aware of them). The current 2016 repo generation would be full subtree and not replicated, next year's generation adds replication, exclusion for later etc, backwards compatible all the time. Researchy for sure ;-) best, c On Fri, 9 Sep 2016, Lixia Zhang wrote: > > On Sep 8, 2016, at 7:18 AM, Burke, Jeff wrote: > > Hi, > > (Sorry for these messages all at once, and let me know if this should go to ndn-interest.) > > Through an earlier conversation with Lixia this week, I realized that the definition / expectations > of a ?repo? may be more specific than I previously understood.? In particular, she mentioned that > repos storing and registering the same namespace should (eventually) have the same data for that > namespace, presumably via sync. > > > I'll start by saying that this is all researchy: we are walking into a new territory that we are yet to > fully explore. > So what I said, before or here, is just to share ideas with others. > > It is easy to see why a desire of all repos for the same name prefix being sync'ed up: an interest for that > prefix may be forwarded to any of the repos, if all repos have the same data, anyone can answer it; > otherwise routers have to handle NACKs and reroute the interest to try other repos. > > > A few related questions: > > 1) Is this attempt at eventual consistency of what?s stored for a given namespace a requirement to be > called a ?repo?? > > > see the above explanation. > I wont call it a requirement, but desirable for performance concerns. > > 2) Practically, if such sync is ultimately required behavior for a repo, will/should repo-ng > itself incorporate some basic synchronization features? > > > as I mentioned, there was a piece of work done on repo sync 2 years back (but I dont know its > implementation status) > > > 3) Does opportunistic caching behavior (without sync) mean that the storage is not a ?repo?? > > > My own view: no. > my explanation to others: > - caching: opportunistic > - (managed) repo: managed storage > > > 4) Would, for example, network-attached storage that stores everything for a prefix but only up > to a given depth in the tree not qualify as a repo? > > > everything is attached to the network :) > so I am not sure what "network-attached storage" really means. > > if your question is "what if a managed repo for a name prefix only contains incomplete data under that > prefix" -- that still works, and *may* even work OK if one understands the traffic patterns so that most of > the Interests forwarded to that repo can be satisfied, without having to?zigzagging?to other places seeking > for data. > > > 5) Or, for example, in the BMS case, if I use a repo to store all of the electrical current > samples for the UCLA campus, but not chilled water, it will have only have some of the tree for > the campus bms prefix.? Is the storage not a repo?? Should it not be registering the root bms > prefix? ??Should I have / what do I call storage that is filling in part of the tree but don?t > need to or can?t store all of it? > > > 1/ if chilled water has its own prefix announcement, maybe one can find a way to attract all interests for > chilled water data to the place for chilled water. > 3/ again one needs to think not just what one wants to put where, but also how forwarding can work well. > > > 6) What are other requirements to be a ?repo??? (Alternatively, is there a canonical reference > in the literature for the type of storage that constitutes a repo?) > > > it does not really matter what we want to call something. > there is no arbitrary requirement. > as I said already, this is research, we have not done much playing with repo up to now. ?It is all about > figuring out how to make the system work in the best way it can. > > my 2 cents, > Lixia > > > From shijunxiao at email.arizona.edu Sat Sep 10 10:09:02 2016 From: shijunxiao at email.arizona.edu (Junxiao Shi) Date: Sat, 10 Sep 2016 10:09:02 -0700 Subject: [Nfd-dev] ndn-cxx feature deprecation: management/nfd-* moved to mgmt/nfd/* Message-ID: Dear folks In #3760 , I have moved ndn-cxx/management/nfd-* headers to ndn-cxx/mgmt/nfd/*. The old headers are kept as alias, but they are deprecated and will be deleted after 0.5.0 release. I have already prepared patches for these projects: NFD https://gerrit.named-data.net/3191 NLSR https://gerrit.named-data.net/3196 If you are a developer in one of these projects, please review the patch on Gerrit. Thanks. I have also confirmed these projects are unaffected: ndn-tools ndn-traffic-generator repo-ng ndn-group-encrypt ChronoChat ChronoSync ndns. Yours, Junxiao -------------- next part -------------- An HTML attachment was scrubbed... URL: From jburke at remap.ucla.edu Sat Sep 10 14:57:47 2016 From: jburke at remap.ucla.edu (Burke, Jeff) Date: Sat, 10 Sep 2016 21:57:47 +0000 Subject: [Nfd-dev] Clarification of 'repo' definition In-Reply-To: <18E0FAE4-89B4-4199-9403-06E8E80F8F2D@cs.ucla.edu> References: <18E0FAE4-89B4-4199-9403-06E8E80F8F2D@cs.ucla.edu> Message-ID: <052FF36E-9718-42AC-98B3-D8A5FFD988E0@remap.ucla.edu> On Sep 8, 2016, at 7:18 AM, Burke, Jeff > wrote: Hi, (Sorry for these messages all at once, and let me know if this should go to ndn-interest.) Through an earlier conversation with Lixia this week, I realized that the definition / expectations of a ?repo? may be more specific than I previously understood. In particular, she mentioned that repos storing and registering the same namespace should (eventually) have the same data for that namespace, presumably via sync. I'll start by saying that this is all researchy: we are walking into a new territory that we are yet to fully explore. So what I said, before or here, is just to share ideas with others. [jb] Yes, of course. :) It is easy to see why a desire of all repos for the same name prefix being sync'ed up: an interest for that prefix may be forwarded to any of the repos, if all repos have the same data, anyone can answer it; otherwise routers have to handle NACKs and reroute the interest to try other repos. [jb] Right, but this is a forwarding-centric view. If I am an app that has some (sparse) knowledge of a namespace, perhaps I just want to make it available to the network, without having to worry about forwarding? I thought we are not supposed to have to think about forwarding! (At least this is what I get told when I ask about publisher mobility... What about a mobile repo? :) A few related questions: 1) Is this attempt at eventual consistency of what?s stored for a given namespace a requirement to be called a ?repo?? see the above explanation. I wont call it a requirement, but desirable for performance concerns. [jb] As I alluded to above, I am having trouble reconciling this with the ?share what you know? approach of sync-based communication. If I am a repo storing the thousand most popular netflix titles for the local geographic region (persistently, for a month, let?s say), do I register a thousand prefixes, or do I need a namespace for popular content (hope not), or knowledge of other names to NACK the content I don?t have? I?m just not sure of the implications of what it means to know about all content for a prefix you are, as a repo, publishing... So I am wondering still if I?m missing a connotation of exactly what a repo is vs. some other publisher. 2) Practically, if such sync is ultimately required behavior for a repo, will/should repo-ng itself incorporate some basic synchronization features? as I mentioned, there was a piece of work done on repo sync 2 years back (but I dont know its implementation status) [jb] Yes, I remember this somewhat too. But it seems that 1) persistent storage is fundamental to apps; 2) interaction with storage happens near/right above the network layer in the case of repo; and 3) there are some performance and conceptual concerns with providing consistency between nodes providing storage in the same namespace, so this should be important to support? (Or, at least to we need to inform users of the existing repo-ng the probably trajectory of development that is expected but not happening yet, in the repo documentation on the wiki? I think this type of getting our implicit plans out there is important to supporting community effort.) 3) Does opportunistic caching behavior (without sync) mean that the storage is not a ?repo?? My own view: no. my explanation to others: - caching: opportunistic - (managed) repo: managed storage [jb] Ok. (I will go back to the chilled water example, though... what about a publisher that naturally only has knowledge of part of a tree..? Is that not really a repo if it?s knowledge isn?t complete for the branch it stores? What about if it doesn?t store depth all the way down) 4) Would, for example, network-attached storage that stores everything for a prefix but only up to a given depth in the tree not qualify as a repo? everything is attached to the network :) so I am not sure what "network-attached storage" really means. [jb] Sorry, the adjective probably wasn?t that relevant. :) if your question is "what if a managed repo for a name prefix only contains incomplete data under that prefix" -- that still works, and *may* even work OK if one understands the traffic patterns so that most of the Interests forwarded to that repo can be satisfied, without having to zigzagging to other places seeking for data. [jb] Not sure I understand who the ?one? is and how they control Interest forwarding? I?m trying to think in terms of scenarios where app developers don?t really have the ability to manage network forwarding any more than they do today. (Yes, of course, some people deploying large apps control their network infrastructure to that level of detail but many don?t.) 5) Or, for example, in the BMS case, if I use a repo to store all of the electrical current samples for the UCLA campus, but not chilled water, it will have only have some of the tree for the campus bms prefix. Is the storage not a repo? Should it not be registering the root bms prefix? Should I have / what do I call storage that is filling in part of the tree but don?t need to or can?t store all of it? 1/ if chilled water has its own prefix announcement, maybe one can find a way to attract all interests for chilled water data to the place for chilled water. [jb] Yes, I think I understand that approach, but that?s not really the deployment that I?m thinking of... the scenario I have in mind is this. Two buildings, with data described like this: /building-1/electrical/power/