[Ndn-interest] trouble with downloading File size > 1MB

[Ishita Dasgupta]

Hi Junxiao,

Sure.

My application imports PyNDN libraries. I am utilising this segment fetcher
PyNDN library(https://github.com/named-data/PyNDN2/blob/
a4df2d207c1b76434a400f54fcaf5d45eea8b844/python/pyndn/util/
segment_fetcher.py) to download the file segments that ndnfs offers.

Regards,
Ishita


On Tue, Jul 25, 2017 at 10:03 PM, Junxiao Shi <shijunxiao at email.arizona.edu>
wrote:

> Can you resend the last message to ndn-interest mailing list? Also,
> include a GitHub/gist/Bitbucket link to the code.
>
> On Jul 25, 2017 17:54, "Ishita Dasgupta" <ishita.dasgupta at gmail.com>
> wrote:
>
>> Hi Junxiao,
>>
>> My application imports PyNDN libraries. I am utilising this segment
>> fetcher PyNDN library
>> <https://github.com/named-data/PyNDN2/blob/a4df2d207c1b76434a400f54fcaf5d45eea8b844/python/pyndn/util/segment_fetcher.py>
>> to download the file segments that ndnfs offers.
>>
>> Regards,
>> Ishita
>>
>> On Sun, Jul 23, 2017 at 3:43 PM, Junxiao Shi <
>> shijunxiao at email.arizona.edu> wrote:
>>
>>> Hi Ishita
>>>
>>> So the problem is:
>>>
>>>    - ndnfs-port creates a zero-length segment if the file size is
>>>    exactly 8KB.
>>>    - The "PyNDN client" cannot correctly handle a zero-length segment.
>>>
>>> I'm unsure which software is violating the protocol. A zero-length
>>> segment by itself is certainly not a protocol violation because it is the
>>> only way to represent an empty file, but I'm unsure about a zero-length
>>> segment at the end of a non-empty file.
>>> I have CC'ed ndn-app, the support list of ndnfs-port. Let's see what the
>>> maintainer of ndnfs-port says about this problem.
>>>
>>> In the meanwhile, can you provide a link to the "PyNDN client" you are
>>> using? I can't find an official PyNDN-based client for NDNFS protocol.
>>>
>>> Yours, Junxiao
>>>
>>> On Wed, Jul 19, 2017 at 11:04 AM, Ishita Dasgupta <
>>> ishita.dasgupta at gmail.com> wrote:
>>>
>>>> Hi Junxiao,
>>>>
>>>> You're right. I tested with a 30MB file and it worked. Somehow for the
>>>> multiples of 8KB files, the last segment is set as 0. Is there any
>>>> workaround that situation or is this a bug with ndnfs-port that has not
>>>> been corrected yet? Can I just change the fixed segment size of 8 KB?
>>>>
>>>> Thanks,
>>>>
>>>> Ishita
>>>>
>>>> On Sat, Jul 15, 2017 at 3:08 PM, Junxiao Shi <
>>>> shijunxiao at email.arizona.edu> wrote:
>>>>
>>>>> Hi Ishita
>>>>>
>>>>> I have successfully served 30MB files in my deployment.
>>>>>
>>>>> This might be a bug on segmentation, instead of a bug related to large
>>>>> file size. 1KB, 10KB, 100KB are not multiply of 8KB.
>>>>> If my hypothesis is true, you should see this problem with exact 8KB,
>>>>> 16KB, 24KB files; this problem should disappear if you add 1 byte after an
>>>>> 1MB or 100MB file.
>>>>> Can you try those sizes and report back?
>>>>>
>>>>> Yours, Junxiao
>>>>>
>>>>> On Sat, Jul 15, 2017 at 09:33 Ishita Dasgupta <
>>>>> ishita.dasgupta at gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I am trying to study performance of file download over ndn across
>>>>>> different file sizes.
>>>>>> I am using ndnfs-port server to do the same, but I am receiving weird
>>>>>> anomaly for file sizes > 1 MB.
>>>>>> It works fine for file sizes of 1KB,10KB and 100KB, but when it comes
>>>>>> to 1MB or greater, the ndnfs server application gives the following
>>>>>> debug log for the last segment for each of these files
>>>>>>
>>>>>> - 1500131726 DEBUG: sign_segment: path=/1MB.txt, ver=1500131708,
>>>>>>> seg=127, len=8192
>>>>>>> - 1500131726 DEBUG: sign_segment: segment name is
>>>>>>> /ndn/broadcast/ndnfs/1MB.txt/%FDYj1%7C/%00%7F
>>>>>>> - 1500131726 DEBUG: sign_segment: path=/1MB.txt, ver=1500131708,
>>>>>>> seg=128, len=0
>>>>>>> - 1500131726 DEBUG: sign_segment: segment name is
>>>>>>> /ndn/broadcast/ndnfs/1MB.txt/%FDYj1%7C/%00%80
>>>>>>
>>>>>>
>>>>>>
>>>>>>> - 1499891047 DEBUG: sign_segment: path=/10MB.txt, ver=1499890866,
>>>>>>> seg=1279, len=8192
>>>>>>> - 1499891047 DEBUG: sign_segment: segment name is
>>>>>>> /ndn/broadcast/ndnfs/10MB.txt/%FDYf%84%B2/%00%04%FF
>>>>>>> - 1499891047 DEBUG: sign_segment: path=/10MB.txt, ver=1499890866,
>>>>>>> seg=1280, len=0
>>>>>>> - 1499891047 DEBUG: sign_segment: segment name is
>>>>>>> /ndn/broadcast/ndnfs/10MB.txt/%FDYf%84%B2/%00%05%00
>>>>>>
>>>>>>
>>>>>> - 1499893665 DEBUG: sign_segment: path=/100MB.txt, ver=1499892176,
>>>>>>> seg=12799, len=8192
>>>>>>> - 1499893665 DEBUG: sign_segment: segment name is
>>>>>>> /ndn/broadcast/ndnfs/100MB.txt/%FDYf%89%D0/%001%FF
>>>>>>> - 1499893665 DEBUG: sign_segment: path=/100MB.txt, ver=1499892176,
>>>>>>> seg=12800, len=0
>>>>>>> - 1499893665 DEBUG: sign_segment: segment name is
>>>>>>> /ndn/broadcast/ndnfs/100MB.txt/%FDYf%89%D0/%002%00
>>>>>>
>>>>>>
>>>>>>
>>>>>> This further causes *“File empty” for the last data packet when
>>>>>> using PyNDN client**.* The client works fine as long as the last
>>>>>> segment (data packet) size on the server is not 0.
>>>>>>
>>>>>> Does anyone have any clue as to why this might be happening? Could
>>>>>> this be a problem with the files itself? I know ndnfs-port fixes file
>>>>>> segment sizes to 8KB, but I couldn't find any explanation that would
>>>>>> suggest that it has a size limit when it comes to file download.
>>>>>> _______________________________________________
>>>>>> Ndn-interest mailing list
>>>>>> Ndn-interest at lists.cs.ucla.edu
>>>>>> http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest
>>>>>>
>>>>>
>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-interest/attachments/20170726/809bb626/attachment.html>