[Nfd-dev] How to physically separate FIB and PIT ?

[Ilya Moiseenko]

Thanks Junxiao for clarification. 
This can’t be right though, because it does not follow NDN architecture. Speed of PIT lookup depends on the cumulative rate of Interest/Data transmissions. Speed of FIB lookup depends on the size of the FIB. If these are mixed up, both lookups slow down to the lowest common denominator. The assumption that PIT is always larger than FIB cannot not be valid. In my case, PIT lookup should be almost instantaneous because I push several thousands on Interest/Data per second (all Interests are satisfied), but instead I’m hit with the penalty of searching 1M FIB entries even though I’m not using FIB lookup.

I think NFD might have a shared nameTree for both PIT and FIB, but it is necessary to have two completely separate indexes: one for PIT and one for FIB. This is a serious bug in my opinion.

Ilya

> On Apr 20, 2017, at 11:25 PM, Junxiao Shi <shijunxiao at email.arizona.edu> wrote:
> 
> Hi Ilya
> 
> NFD puts FIB and PIT together for a reason: FIB and PIT lookups are often used together during Interest packet processing. They achieve a speed-up via NameTree::get() functions which take advantage of the shortcuts between table entries.
> 
> "allocate secondary nameTree and feed it to PIT" does not work because NameTree::get() functions are still assuming PIT and other tables are on the same NameTree. You'll need to change that function to always use name-based lookups and not use the shortcuts.
> 
> During 4th NDN hackathon, my project was an initial implementation of Interest digest in NFD. https://github.com/4th-ndn-hackathon/Initial-Implementation-of-Interest-Digest-in-NFD <https://github.com/4th-ndn-hackathon/Initial-Implementation-of-Interest-Digest-in-NFD>
> In this project, the PIT is switched to a digest index and is thus taken off the NameTree. You may get some ideas on how much work is needed to separate PIT and FIB.
> 
> Yours, Junxiao
> 
> On Thu, Apr 20, 2017 at 10:25 PM, Ilya Moiseenko <iliamo at mailbox.org <mailto:iliamo at mailbox.org>> wrote:
> Hi NFD developers,
> I insert 10K to 1M FIB entries in NFD for some performance tests related to FIB lookup speed. As a result PIT lookups become proportionally slower to the FIB size, which prevents me from getting correct results. I noticed that PIT and FIB physically share the same data structure - NameTree, and tried to allocate secondary nameTree and feed it to PIT. Unfortunately, it seems that there is an additional interplay between PIT, FIB, StrategyChoice, Measurements which results in NFD termination once I pass Interest/Data through it.
> 
> Does anyone have an idea how to disintegrate PIT and FIB ? I don’t really need Measurement and StrategyChoice in working condition.
> 
> Thanks
> Ilya
> _______________________________________________
> Nfd-dev mailing list
> Nfd-dev at lists.cs.ucla.edu
> http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20170421/f4c19cbb/attachment.html>