[Nfd-dev] NDN-RTC testing roll-up for this week (update 3/13)

Burke, Jeff jburke at remap.UCLA.edu
Sun Mar 13 17:45:33 PDT 2016

Hi folks,

Here's the roll-up of ongoing activity related to NDN-RTC.  Please take a look and update / correct anything omitted here or in the related redmine issues.

We will do another test during the seminar. Peter and John may be in touch to recruit help. In general, please make sure to update your version of NdnCon before participating in any tests and, as site / user certificates have expired and rollover is not yet in place, please get your new user certificate here: http://ndncert.named-data.net/  and install prior to using NdnCon again.



Please add information within the appropriate redmine issue, if possible, rather than by email.

A. Next tests

1. Seminar tests.

March 2 test. Christos will give the next seminar on March 2.  Anyone at ColoState that might be able to provide support to him in running both NDN-RTC and WebEx at the same time?  Peter, can you follow up with Christos and Steve as a starting point, and try to push a new version of NDN-RTC with any things that we want to test?  My suggestion is also to provide only one bandwidth option for each stream, so we see uniform requests across all nodes.   http://redmine.named-data.net/issues/3485

3/4 update:  March 9 test. Please review the redmine issue above and items below for some results and discussion since last week. As next week's seminar will be rescheduled, Peter will give an ndncon usage Q&A to generate traffic for the test.   Plans and results for this upcoming seminar will be tracked here: http://redmine.named-data.net/issues/3512 Peter requests 6 participants for now; please help... (I guess 5 plus Peter.)

3/13 update: March 9 performance was not good (see #3512) – choppy audio, with two probable causes: CPU load of the producer and impact of multiple consumers of the same stream (perhaps strategy issue).  Subsequent tests with a smaller number of consumers (http://redmine.named-data.net/issues/3526) did not replicate the problem; plan is to continue isolated tests in addition to seminar tests.

3/13 update:  We will test again during the March 16 seminar, with continued bugfixes and updates
(#3538<http://redmine.named-data.net/issues/3538>, #3536<http://redmine.named-data.net/issues/3536>, #3535<http://redmine.named-data.net/issues/3535>, #3532<http://redmine.named-data.net/issues/3532>, #3531<http://redmine.named-data.net/issues/3531>) in the meantime.   Tracked here: http://redmine.named-data.net/issues/3549

2. Test plan. Peter proposes to put together a methodical testing strategy, using the seminars and other regular meetings as tests of NDN-RTC.  John is already helping with this.  We would like a single point of contact from the NFD team to act as a liaison for this, and help get questions answers / design an approach.  Who should it be?    http://redmine.named-data.net/issues/3487

3/4 update:  Junxiao and Alex will be the POCs.  Junxiao has requested access to REMAP test boxes, we are working on this.

3. Enable ARC detectors.  Peter, I know that we are not ready for incorporating ARC support, and it's not likely to solve current problems, but I'm wondering if we could include the congestion detection mechanism and log its behavior, just to see how it does?   http://redmine.named-data.net/issues/3488

3/4 update:  Postponed until March 9.  Peter, can you please confirm that this is target for this? (I'm hoping we can keep moving on ARC support in parallel to other testing, without disturbing it.)

3/13 update:  Still postponed until other issues addressed.

4. Testing ndncon sync behavior.  We are running tests to chase down and fix (prior to March 9) a potential issue with Chronosync use in ndncon that leads to high CPU load. See http://redmine.named-data.net/issues/3508.  "Given the current implementation, a problematic case could be when sync broadcast interest brings back data that does not result in digest root update (though this, by design, should not happen), and the same interest would be sent again, bringing back the same data thus causing the same interest to be sent again with no interval, and quickly flooding (only) the local NFD."

3/13 update: Fixed. http://redmine.named-data.net/issues/3508  Please make sure to update your version of NdnCon before participating in any tests.

5. Always available NDN-RTC test producers.   The following test producers should always be available, should anyone want to test ndn-rtc:  /ndn/edu/ucla/remap/freeculture and /ndn/edu/ucla/remap/test.  See   http://www.lists.cs.ucla.edu/pipermail/nfd-dev/2016-March/001598.html for details.

B. Triage on last tests

1. PIT / name-tree entries.  After the test on Wednesday, the REMAP node contained nNameTreeEntries=96278, nPitEntries=52185. Can someone help us figure out what they are / what caused them so we can try to mitigate?  Or, provide instrumentation to do this after the next test on this coming Wednesday?    http://redmine.named-data.net/issues/3484  Peter, please let us know what the maximum Interest Lifetime is in NDN-RTC packets ASAP; respond in the redmine thread.

3/4 update: Progress on this issue is discussed in #3484<http://redmine.named-data.net/issues/3484>, and is described as a "design problem introduced with Dead Nonce List"

3/6 update: According to Thursday's NFD call, Junxiao plans to fix in next release (v0.5).  Given the ongoing ndn-rtc testing, any chance we can push this and other NFD bug fixes to the testbed sooner than the retreat?

3/13 update. Fix merged per http://redmine.named-data.net/issues/3484.  @John / Peter, any reason to update REMAP node or any other test nodes to check?

2. Capture of NFD parameters during the test.  John, is it possible to do periodic capture of stats like the above – perhaps every few seconds per node? (Or do you already do that?)   This would be useful in evaluating these tests and also looking for relevant behavior during the test.  Assignee: WUSTL?  http://redmine.named-data.net/issues/3486

3/4 update A: Example live stats capture by John: http://ndndemo.arl.wustl.edu/cacti/ (guest:ndnTest)

3/4 update B: Example of live stats capture by Peter here:
For the seminar-
Live for current ndn-rtc test streams-
Next week will include outgoing Interest rate, incoming segments rate, incoming bitrate, consumer’s state (Chasing, Fetching, Adjusting, etc.), chasing time. Please contact Peter to add other stats.

3/13 update: Stats now tracked in related redmine issues, live links available as well.  @John – Peter asks if it might be possible to log NFD statistics into the same DB he is using for NDN stats.  He'll contact you about this.

3. NDN-RTC: Summarize results of internal testing.  Peter, folks need to understand better what we think should work based on the testing you and Jiachen did.  Perhaps summarize in bullet-point format what we've learned / observed so far, and where we think there are issues. Assignee: Peter.  Due: ASAP.  http://redmine.named-data.net/issues/3483

3/4 update: Completed. @Lixia or others from NFD team, let us know if you need more information.

4. Autoregistration / Backroute creation problem. There is a problem that causes backroutes to not be added reliably to the FIB when connecting a new client to the testbed. Please see this issue: http://redmine.named-data.net/issues/1968  and proposed solution: http://redmine.named-data.net/issues/2182   NFD folks, these tasks are unassigned – can they be assigned? Is the release target of v0.5 correct? We need to do regular restarts of testbed hosts in the meantime to achieve proper functionality.

3/6 update: Blocked by http://redmine.named-data.net/issues/3274, which Yingdi is now working on.  Can a due date for this be set?  Can it be issued as a bug fix prior to v0.5 release and used on the testbed?

3/13 update:  Prerequisites completed, #2182<http://redmine.named-data.net/issues/2182> is now in code review with active discussion.

5. Access strategy assumptions fail for NDN-RTC traffic. Access strategy seems to flood NDN-RTC producers with interests for other producers  if they are connected to the same hub.  Please see issue http://redmine.named-data.net/issues/3219    This issue has been identified for five months and likely impacts NDN-RTC performance on the testbed, can it please be assigned and a release target given?

3/6 update:  In #3513, a solution is proposed to revise the application to use "auto prefix propagation", but this depends on 1) each user having a valid cert (which may require new cert requests, see B-7 below), 2) an application namespace change (C-4), and 3) availability of auto prefix propagation, which is described in a currently unpublished memo, uploaded to the issue.  (Can someone confirm deployment status?) http://redmine.named-data.net/issues/3513

3/13 update: Peter still needs to to update app namespace (#3513<http://redmine.named-data.net/issues/3513>). He'll look at it first thing on Monday and see if it can be completed for Wednesday test.  (Headless consumer/producer is top priority.)

6. Best route strategy may still cause retransmissions by NDN-RTC. The behavior similar to that documented in issue http://redmine.named-data.net/issues/3230 was being observed, visually on the testbed map, during the March 9 test, independently by two participants.  (See #3485 notes #9 and #13. http://redmine.named-data.net/issues/3485)   NFD team, please advise on how to test or resolve this in issue #3485, and/or reopen #3230 if necessary.)

3/13 update:  Junxiao indicates in #3230<http://redmine.named-data.net/issues/3230> that if this is indeed an issue and we need to alter suppression timing, it should come after NDNLP2 improvements to link reliability to be completed in ~4 months.  @Peter, for now, perhaps we can try to verify that this is indeed causing problems – tracked here  http://redmine.named-data.net/issues/3551

7. Need everyone to update expired site and user certs.  Though not yet a dependency of NDN-RTC, current user and testbed  site certificates are needed for the proposed solution to B-5 above, and generally part of the plan for security experimentation on the testbed.  Opened http://redmine.named-data.net/issues/3515  to track.   See Operators list threads "site key expiring<http://lists.named-data.net/mailman/private/operators/2016-January/000996.html>" - site signing mechanisms are ready and John will be updating them.  Peter can include instructions for user cert updates in next ndncon application prep.

3/13 update:  Site certs have been reissued, per John DeHart's recent email to the operators list. http://redmine.named-data.net/issues/3515  has been closed. Users should re-request certs and install prior to continued use of NdnCon.  http://ndncert.named-data.net/

C. Application issues

1. NDN-RTC: Bandwidth performance.  Significantly higher bandwidth than expected was observed during the test, and higher than I remember from previous versions. Peter, can you let us know to expect for a given stream (in both directions), and breakdown in percentages with its major contributions are?  Also, you may wish to change the menu options to reflect both the media bitrate and the estimated total network traffic so people know what to expect.  Assignee: Peter  Due: Before next seminar.  http://redmine.named-data.net/issues/3478

3/4 Update:  Updated with estimates by Peter. Please review.  Peter, please also comment on whether similar bandwidth was observed on March 2 and/or measure on March 9.

2. NDN-RTC: Silence suppression. Several people noted that silence suppression should be incorporate so that audio fetching from silent participants is reduced/eliminated, potentially reducing PIT and CPU load.  (We have discussed this before. Though this is an optimization, we should consider fixing it to remove this critique.  This optimization should be something that can be turned on/off in the GUI.)   Assignee: Peter  Due: Perhaps before retreat  http://redmine.named-data.net/issues/3477

3/4 Update:  Issue updated to describe design challenges. We will start design discussion next week.

3/6 Update: Should this be a hackathon project?

3/13 Update: Elected not to submit as hackathon project – focus on simulation/emulation support.  However, Peter will sketch out a design to discuss at the retreat, this week.

3. Sync behavior.  High CPU load / local interest transmission may be caused sometimes by sync usage in ndncon.  See A-4 above.

4. Update application namespace.  To work around access strategy problem (B-5), the NDN-RTC prefix should be changed from /site/ndnrtc/user to /site/user/ndnrtc.  http://redmine.named-data.net/issues/3513   Multicast conference discovery traffic will still happen in /ndn/broadcast or equivalent, I assume.  Peter, can we change this before the test on Wednesday?

3/13 Update: Targeting completion for this week's demo, if possible, though headless producer/consumer is priority.

5. Decrease delay.   Beichuan notes in the 3/1 NFD call that there is "more of a delay" than WebEx. This is a known issue.  Peter, we discussed some of the causes of additional delay in Osaka.  As time allows amidst other testing, we should expose parameters that make it easy to tune the pipeline size, retransmission, FEC, etc. to achieve lower delay.  Tracked in http://redmine.named-data.net/issues/3516

D. NFD performance

The previous section are performance optimizations.  We can do them, but have to know our target:

1. NFD/Testbed – Acceptable target bandwidth. Even with inefficiencies in the application, the nominal bandwidths that were observed (~100-500 kbyte/sec) should be reasonable on the university networks and nodes in questions.  Given that we are being advised to optimize the app, can someone please advise on what are appropriate target bandwidths for the current NFDs running on the current testbed?  http://redmine.named-data.net/issues/3482

3/4 Update:  No response to issue; can NFD team consider it?

E. NDN-RTC evaluation tools

I really hope that we don't have to wait to the hackathon to make progress. Peter (and perhaps Zhehao or Jeff) can help get NDN-RTC running with the simulation/emulation environment desired by the NFD team, just let us know which of these should be targeted first.

1. Simulation: Real app.  Klaus and others would like to be able to simulate NDN-RTC traffic.  Junxiao suggests that ns3 can support external traffic. I'm not sure if this is what Junxiao was referring to, but here's a HOWTO - https://www.nsnam.org/wiki/HOWTO_make_ns–3_interact_with_the_real_world<https://www.nsnam.org/wiki/HOWTO_make_ns-3_interact_with_the_real_world>.     http://redmine.named-data.net/issues/3479

3/4 Update:  Planned as a potential Hackathon project by Klaus (thanks!). (Preferred to #3) Dependent on completion of NDN-RTC headless producer/consumer, targeted for next week, tracked in http://redmine.named-data.net/issues/3509.

2. Emulation.  In addition, it was discussed that NDN-RTC could be run with the Mini-NDN emulation environment. This seems feasible. Does someone from the NFD team want to try, this perhaps in the context of figuring out NFD PIT growth issue?    https://github.com/named-data/mini-ndn   http://redmine.named-data.net/issues/3481

3/4 Update: Ashlesh will take this on.  (Thanks!) Dependent on completion of NDN-RTC headless producer/consumer, targeted for next week, tracked in http://redmine.named-data.net/issues/3509.

3. Simulation: traffic generator.  If the above is not feasible, Lixia suggests a simple traffic generator could also be written.  (See 2/26 17:26 message).   http://redmine.named-data.net/issues/3480

3/4 Update:  Considered as a potential Hackathon project.

3/13 Update:  Apparently this was not submitted as a hackathon project, given that we're pursuing the other two options.

F. Debugging support

1. NFD Instrumentation.  Though it might take time to add, perhaps we can brainstorm about what instrumentation for NFD would help with debugging these types of applications, and perhaps come up with short-term ways to achieve it.  Could the NFD team contribute ideas to this and Peter, can you add notes to this on an ongoing basses about what might be useful for NDN-RTC?)   http://redmine.named-data.net/issues/3476

3/4 Update: Some suggestions in redmine. No assignee yet.

G. Design challenges / Useful mechanisms

1. Prioritization.  For both audio and eventually scalable video traffic, NDN-RTC could make use of the type of one-hop priority mechanism discussed briefly with Van a year or so ago.  Can we pursue design and discussion of this?  http://redmine.named-data.net/issues/3475  [If a design is sketched in the next few weeks, could a trial implementation be a hackathon project?]

3/4 Update: Several people have asked for motivation for this.  I'll try to document/describe the motivation next week.

3/13 Update:  Aim to discuss at retreat.

2. Congestion control work continues in parallel by Klaus.  Lixia suggests that NDN-RTC specific CC mechanisms are not needed, and asked Klaus for a schedule of the work that he plans.  Klaus, could you let us know this in the redmine issue,  http://redmine.named-data.net/issues/1624

3/4 Update: Short-term approach planned as a hackathon project.

3/6 Update: Klaus asks for input on his congestion control proposal. Now attached to issue #1624.

3/13 Update: Klaus asks for input...


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20160314/f33dc9f5/attachment.html>

More information about the Nfd-dev mailing list