[Ndn-interest] download a file using PyNDN

Thompson, Jeff jefft0 at remap.ucla.edu
Wed May 31 11:30:56 PDT 2017


Yes, that will work. But it’s better to keep the name components in their native form as binary Blobs. The string representation is really only for display, or creating the initial name from a known string representation. You can simply save version as
data.getName().get(-1) . Then, instead of

   name = Name(url+version)

use

    name = Name(url).append(version)

Something else to watch out for: There are data packets with names with and without the %C1.FS.file component. For example:
/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY.%DA%3D
/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY.%DA%3D/%00%01

So far, you’ve been lucky to get the data packet with %C1.FS.file where the version component is at data.getName().get(-1). But this is not guaranteed. It is valid for the NDN server to return one of the “segment” data packets with the second type of name, where the version component is at data.getName().get(-2). But you can easily check with data.getName().get(-1).isVersion() .
http://named-data.net/doc/ndn-ccl-api/name-component.html#name-component-isversion-method

- Jeff T

From: Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Wednesday, May 31, 2017 at 11:13:00
To: Jeff Thompson <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>>
Cc: "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>
Subject: Re: [Ndn-interest] download a file using PyNDN

Hi Jeff,

Yes, I had gotten it working by removing the shutdown command.
Also, this is how I extract the version:
From the first face expressInterest, I get the version from the _onData method I defined as follows:

 defonData(self, interest, data):
        self._callbackCount += 1
        dump("Got data packet with name", data.getName().toUri())
        global version
        version = '/'+str(data.getName().get(-1))
        dump("Version extracted",version)

which gives me the output:
Got data packet with name /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY.%DA%3D

Version extracted /%FDY.%DA%3D

Regards,
Ishita Dasgupta
Graduate Student
College of Information and Computer Sciences,
UMass Amherst
Email: ishitadg at cs.umass.edu<mailto:ishitadg at cs.umass.edu>

On Wed, May 31, 2017 at 1:32 PM, Thompson, Jeff <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>> wrote:
Yes. This is called “name discovery”.

A question: You have "Name(url+version)”. I don’t see where you get “version”.

A hint: Maybe you shouldn’t call face.shutdown() the first time. Only call it at the end of your program.

- Jeff T

From: Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Wednesday, May 31, 2017 at 8:14:00

To: Jeff Thompson <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>>
Cc: "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>
Subject: Re: [Ndn-interest] download a file using PyNDN

Hi Jeff,

I have got it working! Let me know if my procedure to extract the version number by sending a sample interest packet and receiving a sample data packet is correct or is there a simpler method to do it?

Regards,
Ishita Dasgupta
Graduate Student
College of Information and Computer Sciences,
UMass Amherst
Email: ishitadg at cs.umass.edu<mailto:ishitadg at cs.umass.edu>

On Wed, May 31, 2017 at 10:46 AM, Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>> wrote:
Hi Jeff,

To extract the current version, I express an interest on the face and extract the version from the data packet received. Then use this version to extract our data in segments using segment fetcher.
The main code lines look something like this

​       ​
name = Name(url)
        face.expressInterest(name, counter.onData, counter.onTimeout)
​ ​

        while counter._callbackCount < 1:
                face.processEvents()
        face.shutdown()
        name = Name(url+version)
        dump("Express name ", name.toUri())
        interest = Interest(name)
        interest.setInterestLifetimeMilliseconds(1000)
        SegmentFetcher.fetch(face, interest, None, counter.onComplete, counter.onError)
        while counter._callbackCount < 2:
                face.processEvents()
​       ​
face.shutdown()

But for the first interest, i receive a "broken" version number
(
%FDY.%CF-
)
? which is why the segment fetcher doesn't work (?)

1496241583.724584 From: 192.1.242.150, To: 192.1.242.151, Tunnel Type: TCP, INTEREST: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.MustBeFresh=1&ndn.Nonce=1500003601
1496241583.734759 From: 192.1.242.151, To: 192.1.242.150, Tunnel Type: TCP, DATA: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY.%CF-

Is this how I should go about to fetch the version number before expressing the interest or is there a simpler method?

Regards,
Ishita Dasgupta
Graduate Student
College of Information and Computer Sciences,
UMass Amherst
Email: ishitadg at cs.umass.edu<mailto:ishitadg at cs.umass.edu>

On Fri, May 26, 2017 at 2:39 PM, Thompson, Jeff <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>> wrote:
That’s right. The names of the actual segment Data packets should be

/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8/%00%00
/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8/%00%01
/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8/%00%02

etc., up to the maximum segment number. You give SegmentFetcher the name without the final segment number component, and it adds the segment number component as it fetches each one.

- Jeff T

From: Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Friday, May 26, 2017 at 11:35:00

To: Jeff Thompson <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>>
Cc: "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>
Subject: Re: [Ndn-interest] download a file using PyNDN

So, just so that I understand this correctly: " %FDY%28Q8 " is the version component of the all data packet segments representing the file. And then the Segment Fetcher program sequentially downloads the segments individually (and I don't need to worry about the segment numbers?)

Regards,
Ishita Dasgupta
Graduate Student
College of Information and Computer Sciences,
UMass Amherst
Email: ishitadg at cs.umass.edu<mailto:ishitadg at cs.umass.edu>

On Fri, May 26, 2017 at 2:31 PM, Thompson, Jeff <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>> wrote:
Please try the following: Don’t add the line to set the exclude filter, and change the name to include the version number (without “%C1.FS.file”):

name3 = Name("/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%FDY%28Q8”)

The reason is that the segment component comes after the version component “/%FDY%28Q8”.

- Jeff T

From: Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Friday, May 26, 2017 at 11:22:00

To: Jeff Thompson <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>>
Cc: "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>
Subject: Re: [Ndn-interest] download a file using PyNDN

Hi Jeff,

Now it would not return any Data packet at all.
At consumer(ndndump):
1495818158.862071 From: 192.1.242.150, To: 192.1.242.151, Tunnel Type: TCP, INTEREST: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.ChildSelector=1&ndn.MustBeFresh=1&ndn.InterestLifetime=1000&ndn.Nonce=1259570516&ndn.Exclude=%C1.FS.file
<NO DATA PACKET IN RESPONSE>

Regards,
Ishita Dasgupta
Graduate Student
College of Information and Computer Sciences,
UMass Amherst
Email: ishitadg at cs.umass.edu<mailto:ishitadg at cs.umass.edu>

On Fri, May 26, 2017 at 12:54 PM, Thompson, Jeff <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>> wrote:
After

interest = Interest(name3)

try adding

interest.getExclude().appendComponent(Name("%C1.FS.file")[0])

This is because NDNFS not only serves the Data packet segments of the file, but a “meta info” packet where the name has the component “"%C1.FS.file”. The code above should exclude it.

Let us know,
- Jeff T

From: Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Friday, May 26, 2017 at 9:38:00
To: Jeff Thompson <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>>
Cc: "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>

Subject: Re: [Ndn-interest] download a file using PyNDN

Hi Jeff,

I have a file test.txt being served on my ndnfs server. Prefix: /ndn/broadcast/ndnfs
I implement SegmentFetcher in the following way and print content and error message

 name3 = Name(url)
 dump("Express name ", name3.toUri())
 interest = Interest(name3)
 dump(interest.toUri())
 interest.setInterestLifetimeMilliseconds(1000)
SegmentFetcher.fetch(face, interest, None, counter.onComplete, counter.onError)

This is what i get:
/ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.MustBeFresh=true
onError
2
Got an unexpected packet without a segment number: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY%28Q8

When I do an ndndump on the consumer, I see:
1495816236.074843 From: 192.1.242.150, To: 192.1.242.151, Tunnel Type: TCP, INTEREST: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd?ndn.ChildSelector=1&ndn.MustBeFresh=1&ndn.InterestLifetime=1000&ndn.Nonce=3797640908

1495816236.085709 From: 192.1.242.151, To: 192.1.242.150, Tunnel Type: TCP, DATA: /ndn/broadcast/ndnfs/BigBuckBunny_4s_simple_2014_05_09.mpd/%C1.FS.file/%FDY%28Q8

I thought fetching segment number was internally done in your Segment Fetcher util. Do I need to implement it some way on my end as well?

Regards,
Ishita Dasgupta
Graduate Student
College of Information and Computer Sciences,
UMass Amherst
Email: ishitadg at cs.umass.edu<mailto:ishitadg at cs.umass.edu>

On Thu, May 25, 2017 at 2:15 PM, Thompson, Jeff <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>> wrote:
Hi Ishita,

We have NDNFS producing sample data at memoria.ndn.ucla.edu<http://memoria.ndn.ucla.edu>. The PyNDN code below uses the SegmentFetcher to fetch the segments of a file (a JavaScript file). NDNFS has split the file into 12 segments, which SegmentFetcher retrieves to supply the content to the callback. SegmentFetcher documentation is here:
https://github.com/named-data/PyNDN2/blob/master/python/pyndn/util/segment_fetcher.py#L142

- Jeff T

import time, sys
from pyndn import Name, Interest, Face
from pyndn.util import SegmentFetcher

face = Face("memoria.ndn.ucla.edu<http://memoria.ndn.ucla.edu>")
SegmentFetcher.fetch(face, Interest(Name(
  "/ndn/edu/ucla/remap/demo/ndn-js-test/named-data.net/wp-includes/js/jquery/jquery.js/%FDX%DC5B<http://named-data.net/wp-includes/js/jquery/jquery.js/%FDX%DC5B>")),
  None, lambda content: sys.stdout.write(content.toRawStr()),
  lambda errorCode, message: sys.stdout.write("Got error " + message))

while True:
    face.processEvents()
    time.sleep(0.01)


From: Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Thursday, May 25, 2017 at 6:00:00
To: Jeff Thompson <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>>, "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>
Subject: Re: [Ndn-interest] download a file using PyNDN

Hi Jeff,

Yes
​, the producer is responding to Interests with Data packets representing the file. I run NDNFS on the producer.
​

On Wed, May 24, 2017 at 4:35 PM Thompson, Jeff <jefft0 at remap.ucla.edu<mailto:jefft0 at remap.ucla.edu>> wrote:
Hi Ishta,

In NDN, there is a producer and consumer. The consumer sends an Interest and the producer (or a cache in the network) responds with a Data packet. When you say “an entire file from an NDN server” I assume you mean a producer that is responding to Interests with Data packets representing the file. What producer software is running on the server?

- Jeff T

From: Ndn-interest <ndn-interest-bounces at lists.cs.ucla.edu<mailto:ndn-interest-bounces at lists.cs.ucla.edu>> on behalf of Ishita Dasgupta <ishita.dasgupta at gmail.com<mailto:ishita.dasgupta at gmail.com>>
Date: Wednesday, May 24, 2017 at 8:29:00
To: "ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>" <ndn-interest at lists.cs.ucla.edu<mailto:ndn-interest at lists.cs.ucla.edu>>
Subject: [Ndn-interest] download a file using PyNDN

Hi All,

Has anybody used PyNDN to download an entire file from an ndn server?
I see or receive jibberish data in form of one data packet but don't seem to copying the exact content of the file(no matter the size, i.e irrespective of the fact that the file gets divided to multiple segments or just one segment).
Any guidance with this would be helpful.

Thanks,
Ishita






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-interest/attachments/20170531/45ba043b/attachment-0001.html>


More information about the Ndn-interest mailing list