[Mini-NDN] [EXT]Re: Issue with publishing multiple video files

[Junxiao Shi]

Hi Ritu Bordoloi

ndnputchunks is known to consume large amount of memory when publishing a
large file. The main reason is that it must buffer all Data packets in
memory.
For a 300MB file, you can expect 1GB memory usage.
https://redmine.named-data.net/issues/4866
If you do not have sufficient memory, publishing will either fail or take a
very long time due to usage of swapfile.

If you just want a producer that publishes files, but do not require all
Data packets to be buffered in memory, consider using a repository service.
For command line use, try ndn-python-repo
<https://github.com/JonnyKong/ndn-python-repo> (Python).
For programmatic/API use, try @ndn/repo package
<https://ndnts-docs.ndn.today/typedoc/modules/repo.html> (Node.js).
Note that repo-ng is deprecated and should not be used in new applications.

Yours, Junxiao

On Wed, Sep 9, 2020 at 4:25 PM Alexander Wilhelm Lane (awlane) <
awlane at memphis.edu> wrote:

> *External Email*
> This seems to be a putchunks issue- did you review the logs and can you
> post anything notable? I’m unsure if this would be a specifically Mini-NDN
> issue, as I’ve had no trouble with transfers of that size in the past.
> -Alex
>
> On Sep 9, 2020, at 1:42 AM, Ritu Bordoloi via Mini-NDN <
> mini-ndn at lists.cs.ucla.edu> wrote:
>
> Dear All,
>
> I am facing a problem while working on a topology. In my topology, a
> Producer periodically published video files, and multiple Consumers can
> request for those published video files. I am using "ndnputchunks" to
> publish the videos and "nlsrc advertise" to add the routing information. I
> am using "ndncatchunks" to request for the video files.
>
> My problem is when the Producer tries to publish several video files
> (sizes can vary from 100 Mb to 1GB), some of the video files are not
> getting published. Can someone give me any idea on this?
>
> I am attaching a screenshot of the same for your reference, where the
> Publisher tried to publish 6 video files, but only 4 of them were published
> successfully.
>
> Thank you in advance.
>
> Kind regards,
> Ritu
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/mini-ndn/attachments/20200909/042beac7/attachment-0001.html>