[Nfd-dev] [Operators] "No buffer space available"

[Dehart, John]

On Sep 5, 2017, at 11:34 AM, Davide Pesavento <davide.pesavento at lip6.fr<mailto:davide.pesavento at lip6.fr>> wrote:

John,

On Tue, Sep 5, 2017 at 4:59 PM, Dehart, John <jdd at wustl.edu<mailto:jdd at wustl.edu>> wrote:
All:

(Moving this to nfd-dev)

Seems like we are having a lot of FATAL errors and other issues with the
current installation.

When did the instabilities start? Was anything updated recently?

I think we have been seeing problems for at least a couple of weeks but haven’t been able
to identify anything until the last few days. We now have seen
  - FATAL errors associated with the super long prefixes
  - FATAL error for std::bad_alloc
  - nfd huge and stuck and nfdc not able to connect because of “No buffer space available”

I’m still checking on others.

I don’t know of any NDN updates in that time but I’ll check on other system updates.

John




I am seeing this error on the CSU node right now.

ndnops at mccoy:~$ nfdc status show
error while connecting to the forwarder (No buffer space available)

"No buffer space available" is ENOBUFS, I believe, which could mean
that the socket buffers are full.

ndnops at mccoy:~$


I have also had the following error on UCLA recently:

1504524817.021009 FATAL: [NFD] std::bad_alloc


I’m also investigating other nodes that have restarted in the last couple of
days.

I am about to install a scheme for running the nodes in the Testbed in DEBUG
and when nfd
restarts, capture the current set of nfd.log files. I have nfd.log being
rotated and am only saving
the last 10 copies of 10MB each. So, they can disappear fairly quickly with
DEBUG turned on.
But with my scheme I should be able to save copies of anything FATAL that
happened recently.
I’ll also be collecting ‘nfdc face list’ info so we can know what faces were
pointing where when
something FATAL happened.

I’m open to other suggestions.

John


Begin forwarded message:

From: Junxiao Shi <shijunxiao at email.arizona.edu<mailto:shijunxiao at email.arizona.edu>>
Subject: [Operators] "No buffer space available"
Date: September 5, 2017 at 12:06:46 AM CDT
To: "<operators at lists.named-data.net<mailto:operators at lists.named-data.net>>" <operators at lists.named-data.net<mailto:operators at lists.named-data.net>>

Dear folks

ARIZONA node is not working at the moment. `nfdc status` gives error:
error while connecting to the forwarder (No buffer space available)

The error went away when I restart NFD. However, I wonder what does this
error message mean? I've never seen it.
Neither system memory nor disk space was full, and there wasn't high CPU
usage.

Yours, Junxiao
_______________________________________________
Operators mailing list
Operators at lists.named-data.net<mailto:Operators at lists.named-data.net>
http://lists.named-data.net/mailman/listinfo/operators



_______________________________________________
Nfd-dev mailing list
Nfd-dev at lists.cs.ucla.edu<mailto:Nfd-dev at lists.cs.ucla.edu>
http://www.lists.cs.ucla.edu/mailman/listinfo/nfd-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/nfd-dev/attachments/20170905/c765907c/attachment-0001.html>