[Ndn-interest] trouble with downloading File size > 1MB

[Junxiao Shi]

Hi Ishita

So the problem is:

   - ndnfs-port creates a zero-length segment if the file size is exactly
   8KB.
   - The "PyNDN client" cannot correctly handle a zero-length segment.

I'm unsure which software is violating the protocol. A zero-length segment
by itself is certainly not a protocol violation because it is the only way
to represent an empty file, but I'm unsure about a zero-length segment at
the end of a non-empty file.
I have CC'ed ndn-app, the support list of ndnfs-port. Let's see what the
maintainer of ndnfs-port says about this problem.

In the meanwhile, can you provide a link to the "PyNDN client" you are
using? I can't find an official PyNDN-based client for NDNFS protocol.

Yours, Junxiao

On Wed, Jul 19, 2017 at 11:04 AM, Ishita Dasgupta <ishita.dasgupta at gmail.com
> wrote:

> Hi Junxiao,
>
> You're right. I tested with a 30MB file and it worked. Somehow for the
> multiples of 8KB files, the last segment is set as 0. Is there any
> workaround that situation or is this a bug with ndnfs-port that has not
> been corrected yet? Can I just change the fixed segment size of 8 KB?
>
> Thanks,
>
> Ishita
>
> On Sat, Jul 15, 2017 at 3:08 PM, Junxiao Shi <shijunxiao at email.arizona.edu
> > wrote:
>
>> Hi Ishita
>>
>> I have successfully served 30MB files in my deployment.
>>
>> This might be a bug on segmentation, instead of a bug related to large
>> file size. 1KB, 10KB, 100KB are not multiply of 8KB.
>> If my hypothesis is true, you should see this problem with exact 8KB,
>> 16KB, 24KB files; this problem should disappear if you add 1 byte after an
>> 1MB or 100MB file.
>> Can you try those sizes and report back?
>>
>> Yours, Junxiao
>>
>> On Sat, Jul 15, 2017 at 09:33 Ishita Dasgupta <ishita.dasgupta at gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> I am trying to study performance of file download over ndn across
>>> different file sizes.
>>> I am using ndnfs-port server to do the same, but I am receiving weird
>>> anomaly for file sizes > 1 MB.
>>> It works fine for file sizes of 1KB,10KB and 100KB, but when it comes to
>>> 1MB or greater, the ndnfs server application gives the following debug
>>> log for the last segment for each of these files
>>>
>>> - 1500131726 DEBUG: sign_segment: path=/1MB.txt, ver=1500131708,
>>>> seg=127, len=8192
>>>> - 1500131726 DEBUG: sign_segment: segment name is
>>>> /ndn/broadcast/ndnfs/1MB.txt/%FDYj1%7C/%00%7F
>>>> - 1500131726 DEBUG: sign_segment: path=/1MB.txt, ver=1500131708,
>>>> seg=128, len=0
>>>> - 1500131726 DEBUG: sign_segment: segment name is
>>>> /ndn/broadcast/ndnfs/1MB.txt/%FDYj1%7C/%00%80
>>>
>>>
>>>
>>>> - 1499891047 DEBUG: sign_segment: path=/10MB.txt, ver=1499890866,
>>>> seg=1279, len=8192
>>>> - 1499891047 DEBUG: sign_segment: segment name is
>>>> /ndn/broadcast/ndnfs/10MB.txt/%FDYf%84%B2/%00%04%FF
>>>> - 1499891047 DEBUG: sign_segment: path=/10MB.txt, ver=1499890866,
>>>> seg=1280, len=0
>>>> - 1499891047 DEBUG: sign_segment: segment name is
>>>> /ndn/broadcast/ndnfs/10MB.txt/%FDYf%84%B2/%00%05%00
>>>
>>>
>>> - 1499893665 DEBUG: sign_segment: path=/100MB.txt, ver=1499892176,
>>>> seg=12799, len=8192
>>>> - 1499893665 DEBUG: sign_segment: segment name is
>>>> /ndn/broadcast/ndnfs/100MB.txt/%FDYf%89%D0/%001%FF
>>>> - 1499893665 DEBUG: sign_segment: path=/100MB.txt, ver=1499892176,
>>>> seg=12800, len=0
>>>> - 1499893665 DEBUG: sign_segment: segment name is
>>>> /ndn/broadcast/ndnfs/100MB.txt/%FDYf%89%D0/%002%00
>>>
>>>
>>>
>>> This further causes *“File empty” for the last data packet when using
>>> PyNDN client**.* The client works fine as long as the last segment
>>> (data packet) size on the server is not 0.
>>>
>>> Does anyone have any clue as to why this might be happening? Could this
>>> be a problem with the files itself? I know ndnfs-port fixes file segment
>>> sizes to 8KB, but I couldn't find any explanation that would suggest that
>>> it has a size limit when it comes to file download.
>>> _______________________________________________
>>> Ndn-interest mailing list
>>> Ndn-interest at lists.cs.ucla.edu
>>> http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-interest/attachments/20170723/fc8862ef/attachment.html>