<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On Apr 8, 2015, at 1:52 PM, Chengyu Fan <<a href="mailto:chengyu@cs.colostate.edu">chengyu@cs.colostate.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">I see, so the validation is done at the consumer side, not on the repo.<div><br></div><div>BTW, could you also tell me what the RDR, SDR, and UDR stand for?</div></div></blockquote><div><br></div><div>RDR - reliable retrieval of multi-segment content (multiple data packets). It persistently retransmits interests for lost or unverified data packets, and performs automatic reassembly of segments (Data packets)</div><div>UDR - unreliable retrieval of multi-segment content (multiple data packets). Doesn’t do retransmissions or packet reassembly, but sends multiple Interest packets until it reaches finalBlockId.</div><div>SDR - unreliable retrieval of a single-segment content — pretty much the same as sending out one Interest packet.</div><div><br></div><div>Ilya</div><br><blockquote type="cite"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 8, 2015 at 2:24 PM, Ilya Moiseenko <span dir="ltr"><<a href="mailto:iliamo@cs.ucla.edu" target="_blank">iliamo@cs.ucla.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><span class=""><div>On Apr 8, 2015, at 1:17 PM, Chengyu Fan <<a href="mailto:chengyu@cs.colostate.edu" target="_blank">chengyu@cs.colostate.edu</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr">Thanks Dave and Ilya.<div><br></div><div>I was just about to ask how the <span style="font-size:12.8000001907349px">FAST_SIGNING works ...</span><br><div><br></div><div>It's sure that just signing manifests would improve the performance, but not sure if we still can utilize the repo-ng's validator to verify the content?<br></div></div></div></blockquote><div><br></div></span><div>We have repo-ng with disabled validation. Packets are correctly validated at the time of their retrieval if you use RDR protocol in the consumer API.</div><span class="HOEnZb"><font color="#888888"><div><br></div>Ilya</font></span></div><div><div class="h5"><div><br><blockquote type="cite"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 8, 2015 at 2:07 PM, Ilya Moiseenko <span dir="ltr"><<a href="mailto:iliamo@cs.ucla.edu" target="_blank">iliamo@cs.ucla.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>On Apr 8, 2015, at 1:03 PM, Dave Oran (oran) <<a href="mailto:oran@cisco.com" target="_blank">oran@cisco.com</a>> wrote:<br>
<br>
> Change the architecture - only sign manifests, not individual packets.<br>
</span>Hello Dave, this is how the API provides the speed increase.<br>
<span><font color="#888888"><br>
Ilya<br>
</font></span><div><br>
><br>
>> On Apr 8, 2015, at 12:55 PM, Chengyu Fan <<a href="mailto:chengyu@cs.colostate.edu" target="_blank">chengyu@cs.colostate.edu</a>> wrote:<br>
>><br>
>> I think it is due to the packet signing.<br>
>> ndnputfile signs each segment, not the whole data. Therefore, if the segment size is small, ndnputfile has to sign more segments.<br>
>><br>
>> Wonder if there are efficient signing methods for big data provided in the ndn-cxx library?<br>
>><br>
>> On Wed, Apr 8, 2015 at 1:24 PM, Tai-Lin Chu <<a href="mailto:tailinchu@gmail.com" target="_blank">tailinchu@gmail.com</a>> wrote:<br>
>> Is this due to packet signing? or simply segmentation itself?<br>
>><br>
>> Thanks.<br>
>><br>
>> On Wed, Apr 8, 2015 at 10:26 AM, Chengyu Fan <<a href="mailto:chengyu@cs.colostate.edu" target="_blank">chengyu@cs.colostate.edu</a>> wrote:<br>
>>><br>
>>><br>
>>> On Wed, Apr 8, 2015 at 5:08 AM, Urs Schnurrenberger<br>
>>> <<a href="mailto:urs.schnurrenberger@unibas.ch" target="_blank">urs.schnurrenberger@unibas.ch</a>> wrote:<br>
>>>><br>
>>>> Hi,<br>
>>>><br>
>>>><br>
>>>><br>
>>>> why not using ndnputfile? Encoding is done automatically.<br>
>>>><br>
>>>><br>
>>>><br>
>>>> ndnputfile reponame ndnname filename:<br>
>>>><br>
>>>> e.g.<br>
>>>><br>
>>>> ndnputfile /localhost/repo-ng /ndn/ch/unibas/hello hello.txt<br>
>>><br>
>>><br>
>>> Yes, we tried ndnputfile. It is a little bit slow.<br>
>>> However, the segment size is 1000 by default, and takes about 30 minutes to<br>
>>> insert a 1.2GB file.<br>
>>> I changed the segment size to 8000 (the max NDN packet size is 8800), then<br>
>>> the time reduced to 4 minutes.<br>
>>><br>
>>> I wonder if the Tcp Bulk Insert protocol can improve this more ...<br>
>>><br>
>>> BTW, such large file will generate a lots of segments, I have to set the<br>
>>> max-packets to a larger number in repo-ng.conf.<br>
>>> However if the number is too large, repo-ng would complain "ERROR:<br>
>>> conversion of data to type "i" failed". I think it is a bug.<br>
>>><br>
>>>><br>
>>>><br>
>>>> Interesting as well would be how to delete files from the repo? I still do<br>
>>>> it by directly manipulating the database. Seems that there is no tool<br>
>>>> around…?<br>
>>><br>
>>><br>
>>> I didn't see any tool for data deletion either, but the protocol<br>
>>> specification is given, so it is possible to take some time to implement<br>
>>> it.<br>
>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> Chengyu, don't you have problems to retrieve so large files from the repo?<br>
>>>> I have already problems with files around 50MB. The developers told me that<br>
>>>> it might be a memory issue.<br>
>>><br>
>>><br>
>>> I don't have this problem.<br>
>>><br>
>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> Best,<br>
>>>><br>
>>>> Urs<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> Von: Ndn-interest [mailto:<a href="mailto:ndn-interest-bounces@lists.cs.ucla.edu" target="_blank">ndn-interest-bounces@lists.cs.ucla.edu</a>] Im<br>
>>>> Auftrag von Alex Afanasyev<br>
>>>> Gesendet: Mittwoch, 8. April 2015 02:03<br>
>>>> An: Chengyu Fan<br>
>>>> Cc: <a href="mailto:ndn-interest@lists.cs.ucla.edu" target="_blank">ndn-interest@lists.cs.ucla.edu</a><br>
>>>> Betreff: Re: [Ndn-interest] Tcp Bulk Insert Repo Insertion Protocol in<br>
>>>> NDN-repo-ng<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Apr 7, 2015, at 5:01 PM, Chengyu Fan <<a href="mailto:chengyu@cs.colostate.edu" target="_blank">chengyu@cs.colostate.edu</a>> wrote:<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Okay, so the TcpBulkInsert is to push the NDN packet format encoding data<br>
>>>> packets into the repo.<br>
>>>><br>
>>>> If need to insert the raw data, I must convert them into NDN packet format<br>
>>>> encoding in advance. Correct?<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Yes.<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Tue, Apr 7, 2015 at 5:38 PM, Alex Afanasyev<br>
>>>> <<a href="mailto:alexander.afanasyev@ucla.edu" target="_blank">alexander.afanasyev@ucla.edu</a>> wrote:<br>
>>>><br>
>>>> TcpBulkInsert is not really a protocol, rather a socket that accepts a<br>
>>>> stream of data packets.<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Assuming file bulk-data.tlv contains a collection of data packets in NDN<br>
>>>> packet format encoding (just piled up, one after another one), you can use<br>
>>>> ‘nc’ command to push them into the repo<br>
>>>><br>
>>>><br>
>>>><br>
>>>> nc localhost 7376 < bulk-data.tlv<br>
>>>><br>
>>>><br>
>>>><br>
>>>> —<br>
>>>><br>
>>>> Alex<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Apr 7, 2015, at 3:28 PM, Chengyu Fan <<a href="mailto:chengyu@cs.colostate.edu" target="_blank">chengyu@cs.colostate.edu</a>> wrote:<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Hi,<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Do anyone know the "Tcp Bulk Insert Repo Insertion Protocol" in<br>
>>>> NDN-repo-ng?<br>
>>>><br>
>>>><br>
>>>><br>
>>>> We have lots of data need to insert into the repository, and each one<br>
>>>> typically contains about 1.2GB.<br>
>>>><br>
>>>><br>
>>>><br>
>>>> According to the wiki<br>
>>>> (<a href="http://redmine.named-data.net/projects/repo-ng/wiki" target="_blank">http://redmine.named-data.net/projects/repo-ng/wiki</a>), the "Tcp Bulk Insert<br>
>>>> Repo Insertion Protocol" might be the right option.<br>
>>>><br>
>>>><br>
>>>><br>
>>>> However, the wiki doesn't contain the protocol specification, and there is<br>
>>>> no tool for this protocol either ...<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Thanks,<br>
>>>><br>
>>>> Chengyu<br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Ndn-interest mailing list<br>
>>>> <a href="mailto:Ndn-interest@lists.cs.ucla.edu" target="_blank">Ndn-interest@lists.cs.ucla.edu</a><br>
>>>> <a href="http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest" target="_blank">http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> Ndn-interest mailing list<br>
>>> <a href="mailto:Ndn-interest@lists.cs.ucla.edu" target="_blank">Ndn-interest@lists.cs.ucla.edu</a><br>
>>> <a href="http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest" target="_blank">http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest</a><br>
>>><br>
>><br>
>> _______________________________________________<br>
>> Ndn-interest mailing list<br>
>> <a href="mailto:Ndn-interest@lists.cs.ucla.edu" target="_blank">Ndn-interest@lists.cs.ucla.edu</a><br>
>> <a href="http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest" target="_blank">http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest</a><br>
><br>
><br>
> _______________________________________________<br>
> Ndn-interest mailing list<br>
> <a href="mailto:Ndn-interest@lists.cs.ucla.edu" target="_blank">Ndn-interest@lists.cs.ucla.edu</a><br>
> <a href="http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest" target="_blank">http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest</a><br>
<br>
</div></blockquote></div><br></div>
</blockquote></div><br></div></div></div></blockquote></div><br></div>
</blockquote></div><br></body></html>