[Ndn-interest] trouble with downloading File size > 1MB

Nick Briggs nicholas.h.briggs at gmail.com
Sun Jul 23 13:56:27 PDT 2017


I believe the client is the problem.  The protocol was that it was not at the end of the set of segments until it received a segment with the final segment number attribute matching itself.  That allows for it to provide a final segment estimate early, but it can be adjusted forward if the stream is extended.

-- Nick


> On Jul 23, 2017, at 12:43 PM, Junxiao Shi <shijunxiao at email.arizona.edu> wrote:
> 
> Hi Ishita
> 
> So the problem is:
> ndnfs-port creates a zero-length segment if the file size is exactly 8KB.
> The "PyNDN client" cannot correctly handle a zero-length segment.
> I'm unsure which software is violating the protocol. A zero-length segment by itself is certainly not a protocol violation because it is the only way to represent an empty file, but I'm unsure about a zero-length segment at the end of a non-empty file.
> I have CC'ed ndn-app, the support list of ndnfs-port. Let's see what the maintainer of ndnfs-port says about this problem.
> 
> In the meanwhile, can you provide a link to the "PyNDN client" you are using? I can't find an official PyNDN-based client for NDNFS protocol.
> 
> Yours, Junxiao
> 
> On Wed, Jul 19, 2017 at 11:04 AM, Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>> wrote:
> Hi Junxiao, 
> 
> You're right. I tested with a 30MB file and it worked. Somehow for the multiples of 8KB files, the last segment is set as 0. Is there any workaround that situation or is this a bug with ndnfs-port that has not been corrected yet? Can I just change the fixed segment size of 8 KB?
> 
> Thanks,
> Ishita
> 
> On Sat, Jul 15, 2017 at 3:08 PM, Junxiao Shi <shijunxiao at email.arizona.edu <mailto:shijunxiao at email.arizona.edu>> wrote:
> Hi Ishita
> 
> I have successfully served 30MB files in my deployment.
> 
> This might be a bug on segmentation, instead of a bug related to large file size. 1KB, 10KB, 100KB are not multiply of 8KB.
> If my hypothesis is true, you should see this problem with exact 8KB, 16KB, 24KB files; this problem should disappear if you add 1 byte after an 1MB or 100MB file.
> Can you try those sizes and report back?
> 
> Yours, Junxiao
> 
> On Sat, Jul 15, 2017 at 09:33 Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>> wrote:
> Hi All, 
> 
> I am trying to study performance of file download over ndn across different file sizes. 
> I am using ndnfs-port server to do the same, but I am receiving weird anomaly for file sizes > 1 MB. 
> It works fine for file sizes of 1KB,10KB and 100KB, but when it comes to 1MB or greater, the ndnfs server application gives the following debug log for the last segment for each of these files 
> 
> - 1500131726 DEBUG: sign_segment: path=/1MB.txt, ver=1500131708, seg=127, len=8192
> - 1500131726 DEBUG: sign_segment: segment name is /ndn/broadcast/ndnfs/1MB.txt/%FDYj1%7C/%00%7F
> - 1500131726 DEBUG: sign_segment: path=/1MB.txt, ver=1500131708, seg=128, len=0
> - 1500131726 DEBUG: sign_segment: segment name is /ndn/broadcast/ndnfs/1MB.txt/%FDYj1%7C/%00%80
>  
> - 1499891047 DEBUG: sign_segment: path=/10MB.txt, ver=1499890866, seg=1279, len=8192
> - 1499891047 DEBUG: sign_segment: segment name is /ndn/broadcast/ndnfs/10MB.txt/%FDYf%84%B2/%00%04%FF
> - 1499891047 DEBUG: sign_segment: path=/10MB.txt, ver=1499890866, seg=1280, len=0
> - 1499891047 DEBUG: sign_segment: segment name is /ndn/broadcast/ndnfs/10MB.txt/%FDYf%84%B2/%00%05%00 
> 
> - 1499893665 DEBUG: sign_segment: path=/100MB.txt, ver=1499892176, seg=12799, len=8192
> - 1499893665 DEBUG: sign_segment: segment name is /ndn/broadcast/ndnfs/100MB.txt/%FDYf%89%D0/%001%FF
> - 1499893665 DEBUG: sign_segment: path=/100MB.txt, ver=1499892176, seg=12800, len=0
> - 1499893665 DEBUG: sign_segment: segment name is /ndn/broadcast/ndnfs/100MB.txt/%FDYf%89%D0/%002%00 
> 
> 
> This further causes “File empty” for the last data packet when using PyNDN client. The client works fine as long as the last segment (data packet) size on the server is not 0. 
> 
> Does anyone have any clue as to why this might be happening? Could this be a problem with the files itself? I know ndnfs-port fixes file segment sizes to 8KB, but I couldn't find any explanation that would suggest that it has a size limit when it comes to file download.
> _______________________________________________
> Ndn-interest mailing list
> Ndn-interest at lists.cs.ucla.edu <mailto:Ndn-interest at lists.cs.ucla.edu>
> http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest <http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest>
> 
> 
> _______________________________________________
> Ndn-interest mailing list
> Ndn-interest at lists.cs.ucla.edu
> http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-interest/attachments/20170723/ffc6f54d/attachment-0001.html>


More information about the Ndn-interest mailing list