[Ndn-interest] download a file using PyNDN

Nick Briggs nicholas.h.briggs at gmail.com
Wed May 31 11:35:52 PDT 2017


And if he knows he's looking for something of a file flavor then he can ask explicitly for (the highest version of) the name including the %C1.FS.file and proceed from there.

--- Nick Briggs


> On May 31, 2017, at 11:30 AM, Thompson, Jeff <jefft0 at remap.ucla.edu> wrote:
> 
> Yes, that will work. But it’s better to keep the name components in their native form as binary Blobs. The string representation is really only for display, or creating the initial name from a known string representation. You can simply save version as
> data.getName().get(-1) . Then, instead of 
> 
>    name = Name(url+version)
> 
> use
> 
>     name = Name(url).append(version)
> 
> Something else to watch out for: There are data packets with names with and without the %C1.FS.file component. For example:
> /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY.%DA%3D 
> /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY.%DA%3D/%00%01
> 
> So far, you’ve been lucky to get the data packet with %C1.FS.file where the version component is at data.getName().get(-1). But this is not guaranteed. It is valid for the NDN server to return one of the “segment” data packets with the second type of name, where the version component is at data.getName().get(-2). But you can easily check with data.getName().get(-1).isVersion() .
> http://named-data.net/doc/ndn-ccl-api/name-component.html#name-component-isversion-method <http://named-data.net/doc/ndn-ccl-api/name-component.html#name-component-isversion-method>
> 
> - Jeff T
> 
> From: Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
> Date: Wednesday, May 31, 2017 at 11:13:00
> To: Jeff Thompson <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>>
> Cc: "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
> Subject: Re: [Ndn-interest] download a file using PyNDN
> 
> Hi Jeff, 
> 
> Yes, I had gotten it working by removing the shutdown command. 
> Also, this is how I extract the version:
> From the first face expressInterest, I get the version from the _onData method I defined as follows:
> 
>>  defonData(self, interest, data):
>>         self._callbackCount += 1
>>         dump("Got data packet with name", data.getName().toUri())
>>         global version
>>         version = '/'+str(data.getName().get(-1))
>>         dump("Version extracted",version)
> 
> 
> which gives me the output:
>> Got data packet with name /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY.%DA%3D 
>>  
>> Version extracted /%FDY.%DA%3D 
> 
> 
> Regards,
> Ishita Dasgupta
> Graduate Student
> College of Information and Computer Sciences,
> UMass Amherst
> Email: ishitadg at cs.umass.edu <mailto:ishitadg at cs.umass.edu>
> 
> On Wed, May 31, 2017 at 1:32 PM, Thompson, Jeff <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>> wrote:
>> Yes. This is called “name discovery”. 
>> 
>> A question: You have "Name(url+version)”. I don’t see where you get “version”.
>> 
>> A hint: Maybe you shouldn’t call face.shutdown() the first time. Only call it at the end of your program.
>> 
>> - Jeff T
>> 
>> From: Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
>> Date: Wednesday, May 31, 2017 at 8:14:00
>> 
>> To: Jeff Thompson <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>>
>> Cc: "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
>> Subject: Re: [Ndn-interest] download a file using PyNDN
>> 
>> Hi Jeff, 
>> 
>> I have got it working! Let me know if my procedure to extract the version number by sending a sample interest packet and receiving a sample data packet is correct or is there a simpler method to do it?
>> 
>> Regards,
>> Ishita Dasgupta
>> Graduate Student
>> College of Information and Computer Sciences,
>> UMass Amherst
>> Email: ishitadg at cs.umass.edu <mailto:ishitadg at cs.umass.edu>
>> 
>> On Wed, May 31, 2017 at 10:46 AM, Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>> wrote:
>>> Hi Jeff, 
>>> 
>>> To extract the current version, I express an interest on the face and extract the version from the data packet received. Then use this version to extract our data in segments using segment fetcher. 
>>> The main code lines look something like this
>>> 
>>>> ​       ​ name = Name(url)
>>>>         face.expressInterest(name, counter.onData, counter.onTimeout) ​ ​ 
>>>>         while counter._callbackCount < 1:
>>>>                 face.processEvents()
>>>>         face.shutdown()
>>>>         name = Name(url+version)
>>>>         dump("Express name ", name.toUri())
>>>>         interest = Interest(name)
>>>>         interest.setInterestLifetimeMilliseconds(1000)
>>>>         SegmentFetcher.fetch(face, interest, None, counter.onComplete, counter.onError)
>>>>         while counter._callbackCount < 2:
>>>>                 face.processEvents()
>>>> ​       ​ face.shutdown()
>>>  
>>> But for the first interest, i receive a "broken" version number ( %FDY.%CF- ) ? which is why the segment fetcher doesn't work (?)
>>> 
>>>> 1496241583.724584 From: 192.1.242.150, To: 192.1.242.151, Tunnel Type: TCP, INTEREST: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.MustBeFresh=1&ndn.Nonce=1500003601
>>>> 1496241583.734759 From: 192.1.242.151, To: 192.1.242.150, Tunnel Type: TCP, DATA: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY.%CF-
>>> 
>>> Is this how I should go about to fetch the version number before expressing the interest or is there a simpler method? 
>>> 
>>> Regards,
>>> Ishita Dasgupta
>>> Graduate Student
>>> College of Information and Computer Sciences,
>>> UMass Amherst
>>> Email: ishitadg at cs.umass.edu <mailto:ishitadg at cs.umass.edu>
>>> 
>>> On Fri, May 26, 2017 at 2:39 PM, Thompson, Jeff <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>> wrote:
>>>> That’s right. The names of the actual segment Data packets should be
>>>> 
>>>> /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8/%00%00
>>>> /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8/%00%01
>>>> /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8/%00%02
>>>> 
>>>> etc., up to the maximum segment number. You give SegmentFetcher the name without the final segment number component, and it adds the segment number component as it fetches each one.
>>>> 
>>>> - Jeff T
>>>> 
>>>> From: Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
>>>> Date: Friday, May 26, 2017 at 11:35:00
>>>> 
>>>> To: Jeff Thompson <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>>
>>>> Cc: "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
>>>> Subject: Re: [Ndn-interest] download a file using PyNDN
>>>> 
>>>> So, just so that I understand this correctly: " %FDY%28Q8 " is the version component of the all data packet segments representing the file. And then the Segment Fetcher program sequentially downloads the segments individually (and I don't need to worry about the segment numbers?)
>>>> 
>>>> Regards,
>>>> Ishita Dasgupta
>>>> Graduate Student
>>>> College of Information and Computer Sciences,
>>>> UMass Amherst
>>>> Email: ishitadg at cs.umass.edu <mailto:ishitadg at cs.umass.edu>
>>>> 
>>>> On Fri, May 26, 2017 at 2:31 PM, Thompson, Jeff <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>> wrote:
>>>>> Please try the following: Don’t add the line to set the exclude filter, and change the name to include the version number (without “%C1.FS.file”):
>>>>> 
>>>>> name3 = Name("/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8”)
>>>>> 
>>>>> The reason is that the segment component comes after the version component “/%FDY%28Q8”.
>>>>> 
>>>>> - Jeff T
>>>>> 
>>>>> From: Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
>>>>> Date: Friday, May 26, 2017 at 11:22:00
>>>>> 
>>>>> To: Jeff Thompson <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>>
>>>>> Cc: "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
>>>>> Subject: Re: [Ndn-interest] download a file using PyNDN
>>>>> 
>>>>> Hi Jeff, 
>>>>> 
>>>>> Now it would not return any Data packet at all.
>>>>> At consumer(ndndump):
>>>>>> 1495818158.862071 From: 192.1.242.150, To: 192.1.242.151, Tunnel Type: TCP, INTEREST: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.ChildSelector=1&ndn.MustBeFresh=1&ndn.InterestLifetime=1000&ndn.Nonce=1259570516&ndn.Exclude=%C1.FS.file 
>>>>>> <NO DATA PACKET IN RESPONSE> 
>>>>> 
>>>>> 
>>>>> Regards,
>>>>> Ishita Dasgupta
>>>>> Graduate Student
>>>>> College of Information and Computer Sciences,
>>>>> UMass Amherst
>>>>> Email: ishitadg at cs.umass.edu <mailto:ishitadg at cs.umass.edu>
>>>>> 
>>>>> On Fri, May 26, 2017 at 12:54 PM, Thompson, Jeff <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>> wrote:
>>>>>> After
>>>>>> 
>>>>>> interest = Interest(name3)
>>>>>> 
>>>>>> try adding 
>>>>>> 
>>>>>> interest.getExclude().appendComponent(Name("%C1.FS.file")[0])
>>>>>> 
>>>>>> This is because NDNFS not only serves the Data packet segments of the file, but a “meta info” packet where the name has the component “"%C1.FS.file”. The code above should exclude it.
>>>>>> 
>>>>>> Let us know,
>>>>>> - Jeff T
>>>>>> 
>>>>>> From: Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
>>>>>> Date: Friday, May 26, 2017 at 9:38:00
>>>>>> To: Jeff Thompson <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>>
>>>>>> Cc: "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
>>>>>> 
>>>>>> Subject: Re: [Ndn-interest] download a file using PyNDN
>>>>>> 
>>>>>> Hi Jeff, 
>>>>>> 
>>>>>> I have a file test.txt being served on my ndnfs server. Prefix: /ndn/broadcast/ndnfs
>>>>>> I implement SegmentFetcher in the following way and print content and error message 
>>>>>> 
>>>>>>>  name3 = Name(url)
>>>>>>>  dump("Express name ", name3.toUri())
>>>>>>>  interest = Interest(name3)
>>>>>>>  dump(interest.toUri())
>>>>>>>  interest.setInterestLifetimeMilliseconds(1000)
>>>>>>> SegmentFetcher.fetch(face, interest, None, counter.onComplete, counter.onError) 
>>>>>> 
>>>>>> This is what i get: 
>>>>>>> /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.MustBeFresh=true 
>>>>>>> onError
>>>>>>> 2
>>>>>>> Got an unexpected packet without a segment number: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY%28Q8
>>>>>> 
>>>>>> When I do an ndndump on the consumer, I see: 
>>>>>>> 1495816236.074843 From: 192.1.242.150, To: 192.1.242.151, Tunnel Type: TCP, INTEREST: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.ChildSelector=1&ndn.MustBeFresh=1&ndn.InterestLifetime=1000&ndn.Nonce=3797640908 
>>>>>>>  
>>>>>>> 1495816236.085709 From: 192.1.242.151, To: 192.1.242.150, Tunnel Type: TCP, DATA: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY%28Q8
>>>>>> 
>>>>>> I thought fetching segment number was internally done in your Segment Fetcher util. Do I need to implement it some way on my end as well? 
>>>>>> 
>>>>>> Regards,
>>>>>> Ishita Dasgupta
>>>>>> Graduate Student
>>>>>> College of Information and Computer Sciences,
>>>>>> UMass Amherst
>>>>>> Email: ishitadg at cs.umass.edu <mailto:ishitadg at cs.umass.edu>
>>>>>> 
>>>>>> On Thu, May 25, 2017 at 2:15 PM, Thompson, Jeff <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>> wrote:
>>>>>>> Hi Ishita,
>>>>>>> 
>>>>>>> We have NDNFS producing sample data at memoria.ndn.ucla.edu <http://memoria.ndn.ucla.edu/>. The PyNDN code below uses the SegmentFetcher to fetch the segments of a file (a JavaScript file). NDNFS has split the file into 12 segments, which SegmentFetcher retrieves to supply the content to the callback. SegmentFetcher documentation is here:
>>>>>>> https://github.com/named-data/PyNDN2/blob/master/python/pyndn/util/segment_fetcher.py#L142 <https://github.com/named-data/PyNDN2/blob/master/python/pyndn/util/segment_fetcher.py#L142> 
>>>>>>> 
>>>>>>> - Jeff T
>>>>>>> 
>>>>>>> import time, sys
>>>>>>> from pyndn import Name, Interest, Face
>>>>>>> from pyndn.util import SegmentFetcher
>>>>>>> 
>>>>>>> face = Face("memoria.ndn.ucla.edu <http://memoria.ndn.ucla.edu/>")
>>>>>>> SegmentFetcher.fetch(face, Interest(Name(
>>>>>>>   "/ndn/edu/ucla/remap/demo/ndn-js-test/named-data.net/wp-includes/js/jquery/jquery.js/%FDX%DC5B <http://named-data.net/wp-includes/js/jquery/jquery.js/%FDX%DC5B>")),
>>>>>>>   None, lambda content: sys.stdout.write(content.toRawStr()),
>>>>>>>   lambda errorCode, message: sys.stdout.write("Got error " + message))
>>>>>>> 
>>>>>>> while True:
>>>>>>>     face.processEvents()
>>>>>>>     time.sleep(0.01)
>>>>>>> 
>>>>>>> 
>>>>>>> From: Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
>>>>>>> Date: Thursday, May 25, 2017 at 6:00:00
>>>>>>> To: Jeff Thompson <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>>, "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
>>>>>>> Subject: Re: [Ndn-interest] download a file using PyNDN
>>>>>>> 
>>>>>>> Hi Jeff, 
>>>>>>> 
>>>>>>> Yes ​, the producer is responding to Interests with Data packets representing the file. I run NDNFS on the producer.
>>>>>>>>>>>>>> On Wed, May 24, 2017 at 4:35 PM Thompson, Jeff <jefft0 at remap.ucla.edu <mailto:jefft0 at remap.ucla.edu>> wrote:
>>>>>>>> Hi Ishta,
>>>>>>>> 
>>>>>>>> In NDN, there is a producer and consumer. The consumer sends an Interest and the producer (or a cache in the network) responds with a Data packet. When you say “an entire file from an NDN server” I assume you mean a producer that is responding to Interests with Data packets representing the file. What producer software is running on the server?
>>>>>>>> 
>>>>>>>> - Jeff T
>>>>>>>> 
>>>>>>>> From: Ndn-interest <ndn-interest-bounces at lists.cs.ucla.edu <mailto:ndn-interest-bounces at lists.cs.ucla.edu>> on behalf of Ishita Dasgupta <ishita.dasgupta at gmail.com <mailto:ishita.dasgupta at gmail.com>>
>>>>>>>> Date: Wednesday, May 24, 2017 at 8:29:00
>>>>>>>> To: "ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu <mailto:ndn-interest at lists.cs.ucla.edu>>
>>>>>>>> Subject: [Ndn-interest] download a file using PyNDN
>>>>>>>> 
>>>>>>>> Hi All, 
>>>>>>>> 
>>>>>>>> Has anybody used PyNDN to download an entire file from an ndn server?
>>>>>>>> I see or receive jibberish data in form of one data packet but don't seem to copying the exact content of the file(no matter the size, i.e irrespective of the fact that the file gets divided to multiple segments or just one segment).
>>>>>>>> Any guidance with this would be helpful.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Ishita
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 
> _______________________________________________
> Ndn-interest mailing list
> Ndn-interest at lists.cs.ucla.edu
> http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-interest/attachments/20170531/ef0d710e/attachment-0001.html>


More information about the Ndn-interest mailing list