[Nfd-dev] NDN-RTC testing roll-up for this week

Fri Mar 4 16:49:25 PST 2016

Hi folks,

Here are updates (in red) on NDN-RTC testing progress, as reported by Peter and John, and summarized to the best of my ability.

Thanks to everyone that has been working on this!!

Please correct inline or directly in the associated redmine issue.

Note that there are three existing issues with NFD/related tools which were recently identified as impacting NDN-RTC performance on the testbed. See B-4,5,6 below.

Thanks,
Jeff

Please add information within the appropriate redmine issue, if possible, rather than by email.

A. Next tests

1. March 2 seminar. Christos will give the next seminar on March 2.  Anyone at ColoState that might be able to provide support to him in running both NDN-RTC and WebEx at the same time?  Peter, can you follow up with Christos and Steve as a starting point, and try to push a new version of NDN-RTC with any things that we want to test?  My suggestion is also to provide only one bandwidth option for each stream, so we see uniform requests across all nodes.   http://redmine.named-data.net/issues/3485

3/4 update:  March 9 test. Please review the redmine issue above and items below for some results and discussion since last week. As next week's seminar will be rescheduled, Peter will give an ndncon usage Q&A to generate traffic for the test.   Plans and results for this upcoming seminar will be tracked here: http://redmine.named-data.net/issues/3512  Peter requests 6 participants for now; please help... (I guess 5 plus Peter.)

2. Test plan. Peter proposes to put together a methodical testing strategy, using the seminars and other regular meetings as tests of NDN-RTC.  John is already helping with this.  We would like a single point of contact from the NFD team to act as a liaison for this, and help get questions answers / design an approach.  Who should it be?    http://redmine.named-data.net/issues/3487

3/4 update:  Junxiao and Alex will be the POCs.  Junxiao has requested access to REMAP test boxes, we are working on this.

3. Enable ARC detectors.  Peter, I know that we are not ready for incorporating ARC support, and it's not likely to solve current problems, but I'm wondering if we could include the congestion detection mechanism and log its behavior, just to see how it does?   http://redmine.named-data.net/issues/3488

3/4 update:  Postponed until March 9.  Peter, can you please confirm that this is target for this? (I'm hoping we can keep moving on ARC support in parallel to other testing, without disturbing it.)

4. Testing ndncon sync behavior (added 3/4).  We are running tests to chase down and fix (prior to March 9) a potential issue with Chronosync use in ndncon that leads to high CPU load. See http://redmine.named-data.net/issues/3508.  "Given the current implementation, a problematic case could be when sync broadcast interest brings back data that does not result in digest root update (though this, by design, should not happen), and the same interest would be sent again, bringing back the same data thus causing the same interest to be sent again with no interval, and quickly flooding (only) the local NFD."

B. Triage on last tests

1. PIT / name-tree entries.  After the test on Wednesday, the REMAP node contained nNameTreeEntries=96278, nPitEntries=52185. Can someone help us figure out what they are / what caused them so we can try to mitigate?  Or, provide instrumentation to do this after the next test on this coming Wednesday?    http://redmine.named-data.net/issues/3484  @Peter, please let us know what the maximum Interest Lifetime is in NDN-RTC packets ASAP; respond in the redmine thread.

3/4 update: Progress on this issue is discussed in #3484<http://redmine.named-data.net/issues/3484>, and is described as a "design problem introduced with Dead Nonce List"  @Alex or @Junxiao, can you confirm any plan to resolve this?

2. Capture of NFD parameters during the test.  John, is it possible to do periodic capture of stats like the above – perhaps every few seconds per node? (Or do you already do that?)   This would be useful in evaluating these tests and also looking for relevant behavior during the test.  Assignee: WUSTL?  http://redmine.named-data.net/issues/3486

3/4 update A: Example live stats capture by John: http://ndndemo.arl.wustl.edu/cacti/ (guest:ndnTest)

3/4 update B: Example of live stats capture by Peter here:
For the seminar-
http://ec2-52-91-190-26.compute-1.amazonaws.com:3000/dashboard/db/seminar-3-2-2016-overview
(guest:ndnguest)
Live for current ndn-rtc test streams-
http://ec2-52-91-190-26.compute-1.amazonaws.com:3000/dashboard/db/resources-monitor
(guest:ndnguest)
Next week will include outgoing Interest rate, incoming segments rate, incoming bitrate, consumer’s state (Chasing, Fetching, Adjusting, etc.), chasing time. Please contact Peter to add other stats.

3. NDN-RTC: Summarize results of internal testing.  Peter, folks need to understand better what we think should work based on the testing you and Jiachen did.  Perhaps summarize in bullet-point format what we've learned / observed so far, and where we think there are issues. Assignee: Peter.  Due: ASAP.  http://redmine.named-data.net/issues/3483

3/4 update: Completed. @Lixia or others from NFD team, let us know if you need more information.

As of 3/4, three new issues to triage:

4. Autoregistration / Backroute creation problem. There is a problem that causes backroutes to not be added reliably to the FIB when connecting a new client to the testbed. Please see this issue: http://redmine.named-data.net/issues/1968  and proposed solution: http://redmine.named-data.net/issues/2182   NFD folks, these tasks are unassigned – can they be assigned? Is the release target of v0.5 correct? We need to do regular restarts of testbed hosts in the meantime to achieve proper functionality.

5. Access strategy assumptions fail for NDN-RTC traffic. Access strategy seems to flood NDN-RTC producers with interests for other producers  if they are connected to the same hub.  Please see issue http://redmine.named-data.net/issues/3219    This issue has been identified for five months and likely impacts NDN-RTC performance on the testbed, can it please be assigned and a release target given?

6. Best route strategy may still cause retransmissions by NDN-RTC.  The behavior similar to that documented in issue http://redmine.named-data.net/issues/3230 was being observed, visually on the testbed map, during the March 9 test, independently by two participants.  (See #3485 notes #9 and #13. http://redmine.named-data.net/issues/3485)   NFD team, please advise on how to test or resolve this in issue #3485, and/or reopen #3230 if necessary.)

C. Application issues

1. NDN-RTC: Bandwidth performance.  Significantly higher bandwidth than expected was observed during the test, and higher than I remember from previous versions. Peter, can you let us know to expect for a given stream (in both directions), and breakdown in percentages with its major contributions are?  Also, you may wish to change the menu options to reflect both the media bitrate and the estimated total network traffic so people know what to expect.  Assignee: Peter  Due: Before next seminar.  http://redmine.named-data.net/issues/3478

3/4 Update:  Updated with estimates by Peter. Please review.  Peter, please also comment on whether similar bandwidth was observed on March 2 and/or measure on March 9.

2. NDN-RTC: Silence suppression. Several people noted that silence suppression should be incorporate so that audio fetching from silent participants is reduced/eliminated, potentially reducing PIT and CPU load.  (We have discussed this before. Though this is an optimization, we should consider fixing it to remove this critique.  This optimization should be something that can be turned on/off in the GUI.)   Assignee: Peter  Due: Perhaps before retreat  http://redmine.named-data.net/issues/3477

3/4 Update:  Issue updated to describe design challenges. We will start design discussion next week.

3. Sync behavior.  High CPU load / local interest transmission may be caused sometimes by sync usage in ndncon.  See A-4 above.

D. NFD performance

The previous section are performance optimizations.  We can do them, but have to know our target:

1. NFD/Testbed – Acceptable target bandwidth. Even with inefficiencies in the application, the nominal bandwidths that were observed (~100-500 kbyte/sec) should be reasonable on the university networks and nodes in questions.  Given that we are being advised to optimize the app, can someone please advise on what are appropriate target bandwidths for the current NFDs running on the current testbed?  http://redmine.named-data.net/issues/3482

3/4 Update:  No response to issue; can NFD team consider it?

E. NDN-RTC evaluation tools

I really hope that we don't have to wait to the hackathon to make progress. Peter (and perhaps Zhehao or Jeff) can help get NDN-RTC running with the simulation/emulation environment desired by the NFD team, just let us know which of these should be targeted first.

1. Simulation: Real app.  Klaus and others would like to be able to simulate NDN-RTC traffic.  Junxiao suggests that ns3 can support external traffic. I'm not sure if this is what Junxiao was referring to, but here's a HOWTO - https://www.nsnam.org/wiki/HOWTO_make_ns–3_interact_with_the_real_world<https://www.nsnam.org/wiki/HOWTO_make_ns-3_interact_with_the_real_world>.     http://redmine.named-data.net/issues/3479

3/4 Update:  Planned as a potential Hackathon project by Klaus (thanks!). (Preferred to #3) Dependent on completion of NDN-RTC headless producer/consumer, targeted for next week, tracked in http://redmine.named-data.net/issues/3509.

2. Emulation.  In addition, it was discussed that NDN-RTC could be run with the Mini-NDN emulation environment. This seems feasible. Does someone from the NFD team want to try, this perhaps in the context of figuring out NFD PIT growth issue?    https://github.com/named-data/mini-ndn   http://redmine.named-data.net/issues/3481

3/4 Update: Ashlesh will take this on.  (Thanks!) Dependent on completion of NDN-RTC headless producer/consumer, targeted for next week, tracked in http://redmine.named-data.net/issues/3509.

3. Simulation: traffic generator.  If the above is not feasible, Lixia suggests a simple traffic generator could also be written.  (See 2/26 17:26 message).   http://redmine.named-data.net/issues/3480

3/4 Update:  Considered as a potential Hackathon project.

F. Debugging support

1. NFD Instrumentation.  Though it might take time to add, perhaps we can brainstorm about what instrumentation for NFD would help with debugging these types of applications, and perhaps come up with short-term ways to achieve it.  Could the NFD team contribute ideas to this and Peter, can you add notes to this on an ongoing basses about what might be useful for NDN-RTC?)   http://redmine.named-data.net/issues/3476

3/4 Update: Some suggestions in redmine. No assignee yet.

G. Design challenges / Useful mechanisms

1. Prioritization.  For both audio and eventually scalable video traffic, NDN-RTC could make use of the type of one-hop priority mechanism discussed briefly with Van a year or so ago.  Can we pursue design and discussion of this?  http://redmine.named-data.net/issues/3475  [If a design is sketched in the next few weeks, could a trial implementation be a hackathon project?]

3/4 Update: Several people have asked for motivation for this.  I'll try to document/describe the motivation next week.

2. Congestion control work continues in parallel by Klaus.  Lixia suggests that NDN-RTC specific CC mechanisms are not needed, and asked Klaus for a schedule of the work that he plans.  Klaus, could you let us know this in the redmine issue,  http://redmine.named-data.net/issues/1624

3/4 Update: Short-term approach planned as a hackathon project.

Thanks!
Jeff

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20160305/24c1371a/attachment.html>