[Nfd-dev] [Operators] "No buffer space available"

[Junxiao Shi]

Hi John


> I am also collecting face information every minute (nfdc face list) and
> storing
> that in a log file so I can see what faces existed around the time of the
> crash.
> My thinking was that when we see the giant interest in the nfd.log I would
> be able
> to tell where it was coming from and track down what was sending it.
>
You can do so without logging `nfdc face list`. Forwarder DEBUG log tells
you the FaceId. FaceTable::add log tells you local and remote FaceUri
associated with this FaceId.


> In this case, during the time that nfd is timing out Fib Updates, I see in
> my
> nfdc log file that nfdc no longer gets any results. I neglected to
> redirect stderr
> to the log file but I suspect that nfdc is failing to connect to get its
> results.
> But just before it starts failing it showed 501 TCP faces.
> I have a small number of nodes in the Testbed that have a problem with
> fragmentation and for them I run their faces as TCP. That seems to be a
> problem.
>

I used to observed similar behavior for unix faces on several nodes' HTTP
status page. Occasionally, there were more than 50 unix faces. I believed
it's some large monitoring script starting up. I cannot tell how often this
occurs.


> For this node, SRRU, it has faces to 6 other nodes. But just before it
> crashed
> it had 70-100 faces to each of those 6 nodes.
>

Are these faces "on-demand" (created because remote node connecting in) or
"persistent" (created by a local node)? Do these faces have the same
local/remote port number?

Why would the TCP faces not go away when a new one was needed?
>

If the faces were "persistent" and have same remote port but different
local port, this suggests a bug in NLSR. Deployed NLSR creates faces if one
does not exist, but it might repeat face creation if face dataset does not
reflect new faces.
However, I'm not concerned by this potential bug because this feature is no
longer present in NLSR source code. It instead relies on "permanent" faces
already created in NFD.

We had another FATAL error overnight.
>
> https://www.arl.wustl.edu/~jdd/nfd.log.SRRU.FATAL.gz
>
> This log does not end with FATAL.

> <https://www.arl.wustl.edu/~jdd/nfd.log.SRRU.FATAL.gz>
>
>
> I’m also very curious about what Eric said in the nfd conference call
> yesterday.
> I believe he said that in integration testing he has seen an instance of a
> long interest
> with empty components. Is that actually something built into the tests or
> is this something
> we should be concerned about?
>
> No, there isn't an integration test scenario that uses a long name.

Yours, Junxiao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20170907/2833aca1/attachment.html>