[Nfd-dev] ndn-cxx Bug 4808

[Junxiao Shi]

Dear folks

There is an urgent developer disagreement during code review related to
issue 4808. It seems that this could not wait until the next NFD call, so
I’ll explain the facts here.

The technical problem: ndn-cxx has a random number generator implemented by
calling into third party libraries. There was a unit test using statistics
method to check that the generated random number conforms to a uniform
distribution. Given it’s a randomized test, the unit test fails “softly”:
it creates a warning when fails, not an error.
Recent changes: developer A made a commit changing the soft failure to hard
failure. As a result, many Jenkins builds are failing. An independent test
indicates the failure rate is 14% or more.

Attempted fix A: delete the unit test outright because it’s “semi-broken”.
Afterwards, there would be no unit test to check the numbers are random.
Opinion B: “testing random number generator is not ndn-cxx’s business”, so
the unit test can be deleted.
Opinion C: every line of code requires at least one failing test.
Therefore, without any unit test on the random number generator, one could
just make “return 0;” the random number generator.
Attempted fix D: rewrite the “goodness of fit” unit test, loosening
numerical requirements. Execute the test five times, and declare a hard
failure only if the test fails three or more times out of five. The failure
rate of this method is 0.01%.
Opinion E: revert to soft failure.

I am one of the parties involved but I tried to summerize the facts. I hope
everyone (including myself) can calm down and make a decision at next NFD
call, and don’t rush with merging one of the changes. Please keep all
discussions on the mailing list by using reply-all only.

Yours, Junxiao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20190118/6b369b01/attachment.html>