[Mini-NDN] [EXT]Re: Issue with publishing multiple video files

[Ritu Bordoloi]

Dear Alex and Junxiao,

Thank you for your reply. It seems that it is a memory issue, as I can
publish small size videos without any issue.

As for the repo, I can see two repo packages
https://github.com/named-data/repo-ng and
https://github.com/JonnyKong/ndn-python-repo. I tried to install
https://github.com/named-data/repo-ng. However, I was getting the following
error.

Waf: Entering directory `/home/ritu/Downloads/repo-ng-master/build'
[ 2/20] Compiling src/handles/read-handle.cpp
[ 3/20] Compiling src/repo-command-parameter.cpp
In file included from ../src/handles/read-handle.hpp:23:0,
                 from ../src/handles/read-handle.cpp:20:
../src/common.hpp:31:10: fatal error: ndn-cxx/security/validator.hpp: No
such file or directory
 #include <ndn-cxx/security/validator.hpp>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Waf: Leaving directory `/home/ritu/Downloads/repo-ng-master/build'
Build failed
 -> task in 'repo-objects' failed with exit status 1 (run with -v to
display more information)

Can someone give me any idea on this?

Thank you in advance.

Kind regards,
Ritu


On Thu, 10 Sep 2020, 9:55 am Junxiao Shi, <shijunxiao at email.arizona.edu>
wrote:

> Hi Ritu Bordoloi
>
> ndnputchunks is known to consume large amount of memory when publishing a
> large file. The main reason is that it must buffer all Data packets in
> memory.
> For a 300MB file, you can expect 1GB memory usage.
> https://redmine.named-data.net/issues/4866
> If you do not have sufficient memory, publishing will either fail or take
> a very long time due to usage of swapfile.
>
> If you just want a producer that publishes files, but do not require all
> Data packets to be buffered in memory, consider using a repository service.
> For command line use, try ndn-python-repo
> <https://github.com/JonnyKong/ndn-python-repo> (Python).
> For programmatic/API use, try @ndn/repo package
> <https://ndnts-docs.ndn.today/typedoc/modules/repo.html> (Node.js).
> Note that repo-ng is deprecated and should not be used in new applications.
>
> Yours, Junxiao
>
> On Wed, Sep 9, 2020 at 4:25 PM Alexander Wilhelm Lane (awlane) <
> awlane at memphis.edu> wrote:
>
>> *External Email*
>> This seems to be a putchunks issue- did you review the logs and can you
>> post anything notable? I’m unsure if this would be a specifically Mini-NDN
>> issue, as I’ve had no trouble with transfers of that size in the past.
>> -Alex
>>
>> On Sep 9, 2020, at 1:42 AM, Ritu Bordoloi via Mini-NDN <
>> mini-ndn at lists.cs.ucla.edu> wrote:
>>
>> Dear All,
>>
>> I am facing a problem while working on a topology. In my topology, a
>> Producer periodically published video files, and multiple Consumers can
>> request for those published video files. I am using "ndnputchunks" to
>> publish the videos and "nlsrc advertise" to add the routing information. I
>> am using "ndncatchunks" to request for the video files.
>>
>> My problem is when the Producer tries to publish several video files
>> (sizes can vary from 100 Mb to 1GB), some of the video files are not
>> getting published. Can someone give me any idea on this?
>>
>> I am attaching a screenshot of the same for your reference, where the
>> Publisher tried to publish 6 video files, but only 4 of them were published
>> successfully.
>>
>> Thank you in advance.
>>
>> Kind regards,
>> Ritu
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/mini-ndn/attachments/20200911/43829be3/attachment.html>