<div dir="ltr"><div>Hi Davide & Klaus</div><div><br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> I have some recent numbers for ndncatchunks from April 2019. These were using NFD 0.6.5 and ndn-tools 0.6.3 on Ubuntu 16.04. Hardware is 2x Xeon E5-2640 and 32GB memory.<br>
<br>
The experiments should be repeated with ndn-tools 0.6.4, ndncatchunks uses a new CUBIC-like congestion control in that version.<br></blockquote><div><br></div><div>Yes I'll do that soon.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> Raw numbers are read from ndncatchunks directly. Reported numbers are average and stdev over 18 flows (6 flows per trial).<br>
<br>
As I already privately told Junxiao some time ago, I believe these numbers are rather inconclusive, but they do raise some eyebrows and should motivate further investigations into some peculiar (if not pathological) behaviors of NFD that should be fixed or improved. On the other hand, some results may need to be dismissed due to incorrect testing methodology.<br></blockquote><div><br></div><div>Can you elaborate on what is "incorrect testing methodology"?</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Btw, what do you mean "3 flows per trial"? If you start 6 prod/cons pairs, won't you get 6 concurrent flows?<br></blockquote><div><br></div><div>It's a typo. It should be "6 flows per trial".</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><font face="courier new, monospace">> Goodput (Mbps)<br>
> SETUP | PROD avg | PROD stdev | CS avg | CS stdev<br>
> ----------|----------|------------|--------|-----------<br>
> NFD-small | 63.88 | 0.27 | 61.27 | 2.52<br>
> NFD-large | 65.68 | 1.55 | 85.17 | 1.91<br></font>
<br>
So the CS is only beneficial if you get a large percentage of hits...?<br>
(The NFD-large/CS case has 100% hit ratio)<br></blockquote><div><br></div><div>Looking at PROD avg column, NFD-small is slower than NFD-large because forwarding thread needs to spend time evicting entries.</div><div>NFD-small CS avg column is possibly 0% hit because packets at front of files are evicted when the second retrievals start.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><font face="courier new, monospace">
> Total number of lost/retransmitted segments<br>
> SETUP | PROD avg | PROD stdev | CS avg | CS stdev<br>
> ----------|----------|------------|--------|-----------<br>
> NFD-small | 361.61 | 75.00 | 164.00 | 114.51<br>
> NFD-large | 226.28 | 164.70 | 0.00 | 0.00<br></font>
<br>
The stddev is huge on these numbers, which means either you need more trials or something weird is going on.<br></blockquote><div><br></div><div>Yeah they are unstable. I don't have automated experiment runner yet so I can only do three manual trials.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><font face="courier new, monospace">
> RTT avg (ms)<br>
> SETUP | PROD avg | PROD stdev | CS avg | CS stdev<br>
> ----------|----------|------------|--------|-----------<br>
> NFD-small | 164.87 | 22.42 | 153.04 | 23.14<br>
> NFD-large | 174.48 | 18.36 | 8.19 | 0.11<br></font>
<br>
With the exception of NFD-large/CS (where IIUC no packets should be forwarded to the producers), the RTT looks quite bad, considering the link delay is effectively zero. My guess is that there is too much buffering/queueing somewhere.<br></blockquote><div><br></div><div>Kernel has finite buffering. Boost has no buffering. StreamTransport has infinite buffering. #<a href="https://redmine.named-data.net/issues/4499">4499</a> strikes again.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> If Data come from CS instead of producers, it's even faster.<br>
<br>
This is not true in general. In fact, it has been proven false more than once in the past.<br></blockquote><div><br></div><div>Because lookup is too slow?</div><div><br></div><div> </div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> <br>> i7-6600U has higher frequency (2.60GHz) than E5-2640 (2.50GHz). </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>But you have 12 CPU cores vs 2 on my laptop.<br></blockquote><div><br></div><div>12 cores won't help because NFD forwarding is single threaded.</div><div>Also, i7-6600U is newer Skylake architecture than E5-2640 Sandy Bridge architecture.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> Having only one flow does not reflect usual workload on a file server or <br>> a web browser client.<br><br>Yes, but 6 simultaneous flows should actually be *faster* in their combined throughput. At least that's the case when link bandwidth is the limiting factor.</blockquote><div><br></div><div>Yes.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Moreover, you can distribute the flows among different CPUs.<br></blockquote><div><br></div><div>No. NFD forwarding is single threaded.</div><div> </div><div>Yours, Junxiao</div></div></div></div>