From wm at ars.is  Mon Dec  2 07:03:38 2013
From: wm at ars.is (Michael Weiss)
Date: Mon, 02 Dec 2013 16:03:38 +0100
Subject: [Ndn-interest] NDN Packet Format
Message-ID: <529CA14A.9010203@ars.is>

Greetings everyone,

I welcome the decision to revisit the packet format [Spec0.1]. I'm 
probably missing some aspects, that are not discussed in that document, 
but here are my thoughts so far:

VAR-NUMBER: 64 bits for a length field matches the size used in many 
systems today but, given that the length is field is variable length 
anyway, I'd rather have he possibilty to use 128 bit length fields, too.

Interest packet: I have not thought too hard about the computational 
implications on this one but in general, I'd prefer to have all required 
fields before the optional fields and thus move to the nonce right after 
the name and before the selectors.

PublisherPublicKeyLocator: It has taken me some time to figure out how 
to handle colliding key names and the only solution I can come up with 
is to include the digest component in the key name. If that is the 
intended way, I suggest to be more explicit about it in the specification.
On a slightly related note, high on my wishlist for format/protocol 
redesigns is the ability to directly use public keys in place of digests 
or locators, especially since implementations with fast operations and 
256 bit public keys [NaCl] are available. Right next to that wish would 
be to use the packet format within signatures or any other complex data 
needed for the core protocols for that matter.

Now, to a more philosophical discussion: TLV
I'm not convinced that we need types at the structure level. Explicitly 
typed values can always be represented with one more level of nesting 
and a type-value pair inside. My recommendation is to use something 
semantically equivalent to S-expressions and work out an interchange 
format for use on the wire. The reasoning behind a typeless (beyond 
length and value) structure is that types in the structure itself 
require a high level of agreement and may hide computational complexity. 
Protocols would still be able to reuse type dictionaries from other 
protocols but that would not require coordination among all other 
protocols. Further, types can be omitted if protocols are precise about 
the set of fields on each level and their order. I argue that the 
inabilty to erase types is a sign of hidden complexity.
On a more practical note, my suggestion for such a format is to use 
Variable Length Values as used by MIDI for length fields, meaning one 
continuation bit and 7 bits of data in each octet in whatever bit and 
byte order the signaling folks among you favour. The minimum number of 
octets MUST be used to ensure uniqueness of representation. The null 
byte could then be used to represent omitted optional fields.
For more insight on the desire for simple exchange formats, I highly 
recommend watching "The Science of Insecurity" held on 28C3 and related 
material on http://langsec.org

Best wishes, m

[Spec0.1] http://named-data.net/wp-content/uploads/2013/11/packetformat.pdf
[NaCl] http://nacl.cr.yp.to/sign.html


From lixia at cs.ucla.edu  Sun Dec  8 20:10:13 2013
From: lixia at cs.ucla.edu (Lixia Zhang)
Date: Sun, 8 Dec 2013 20:10:13 -0800
Subject: [Ndn-interest] NDN Packet Format
In-Reply-To: <529CA14A.9010203@ars.is>
References: <529CA14A.9010203@ars.is>
Message-ID: <E86F9D85-ADD4-4996-98FB-79EC74753DFA@cs.ucla.edu>

catching up email on a long flight: Michael, thanks for this long and interesting msg!
below is my own 2 cents on the questions.

On Dec 2, 2013, at 7:03 AM, Michael Weiss <wm at ars.is> wrote:

> Greetings everyone,
> 
> I welcome the decision to revisit the packet format [Spec0.1]. I'm probably missing some aspects, that are not discussed in that document, but here are my thoughts so far:
> 
> VAR-NUMBER: 64 bits for a length field matches the size used in many systems today but, given that the length is field is variable length anyway, I'd rather have he possibilty to use 128 bit length fields, too.

hmm, not sure I fully understand the implication of this comment.
the current encoding allows the length field to go up to 8 bytes (for a value field up to 2^64 byte = 1.8X10^19), are you thinking this is not adequate for reasonable future?

> Interest packet: I have not thought too hard about the computational implications on this one but in general, I'd prefer to have all required fields before the optional fields and thus move to the nonce right after the name and before the selectors.

we discussed that option earlier, however people more familiar with implementation argued that the selector fields should go first to ease the lookup (e.g. do a hash of name+selector as a lookup key)

> PublisherPublicKeyLocator: It has taken me some time to figure out how to handle colliding key names and the only solution I can come up with is to include the digest component in the key name. If that is the intended way, I suggest to be more explicit about it in the specification.

digest can certainly prevent collisions, but given that NDN uses hierarchically structured names including key names, I would like to better understand your scenarios of possible key name collisions first, if you dont mind elaborate a bit.

> On a slightly related note, high on my wishlist for format/protocol redesigns is the ability to directly use public keys in place of digests or locators, especially since implementations with fast operations and 256 bit public keys [NaCl] are available.

this is an interesting suggestion. I think it assumes that some a prior steps already occurred that provided one with the keybits of a trusted key.

> Right next to that wish would be to use the packet format within signatures or any other complex data needed for the core protocols for that matter.
> 
> Now, to a more philosophical discussion: TLV
> I'm not convinced that we need types at the structure level. Explicitly typed values can always be represented with one more level of nesting and a type-value pair inside. My recommendation is to use something semantically equivalent to S-expressions and work out an interchange format for use on the wire. The reasoning behind a typeless (beyond length and value) structure is that types in the structure itself require a high level of agreement and may hide computational complexity. Protocols would still be able to reuse type dictionaries from other protocols but that would not require coordination among all other protocols. Further, types can be omitted if protocols are precise about the set of fields on each level and their order. I argue that the inabilty to erase types is a sign of hidden complexity.

another interesting idea. My thought:
1/ yes the type definition requires agreement, but so does protocols with precise fields on each level and their order, right?

2/ as the 0.1 spec stated, the choice of TLV format is for its flexibility in adding new types and phasing out old types as the protocol evolves over time. a precisely defined protocol would not support incremental changes, right?


> On a more practical note, my suggestion for such a format is to use Variable Length Values as used by MIDI for length fields, meaning one continuation bit and 7 bits of data in each octet in whatever bit and byte order the signaling folks among you favour. The minimum number of octets MUST be used to ensure uniqueness of representation. The null byte could then be used to represent omitted optional fields.

we also considered this encoding format earlier, but rejected it due to the processing (which might not best fit hardwire implementations)

> For more insight on the desire for simple exchange formats, I highly recommend watching "The Science of Insecurity" held on 28C3 and related material on http://langsec.org

thanks for the pointer!  
My flight does not have Internet connection but will watch this when I get a chance.

> Best wishes, m
> 
> [Spec0.1] http://named-data.net/wp-content/uploads/2013/11/packetformat.pdf
> [NaCl] http://nacl.cr.yp.to/sign.html
> _______________________________________________
> Ndn-interest mailing list
> Ndn-interest at lists.cs.ucla.edu
> http://www.lists.cs.ucla.edu/mailman/listinfo/ndn-interest




From Ersin.Uzun at parc.com  Thu Dec 12 11:03:07 2013
From: Ersin.Uzun at parc.com (Ersin.Uzun at parc.com)
Date: Thu, 12 Dec 2013 19:03:07 +0000
Subject: [Ndn-interest] NDN Packet Format
In-Reply-To: <E86F9D85-ADD4-4996-98FB-79EC74753DFA@cs.ucla.edu>
References: <529CA14A.9010203@ars.is>
	<E86F9D85-ADD4-4996-98FB-79EC74753DFA@cs.ucla.edu>
Message-ID: <D705D60666D05C479D59A124922B98B73882E16F@e2010dag5.corp.ad.parc.com>

Hi Michael,
I saw your comments regarding the public keys while catching up with my
email, and wanted to quickly say that we believe we have a way to solve
those issues of naming, getting and establishing trust to the keys in an
efficient way. We are finalizing the writeup now and will make it publicly
available soon. 

Typoed from my phone,
Ersin



>
>> PublisherPublicKeyLocator: It has taken me some time to figure out how
>>to handle colliding key names and the only solution I can come up with
>>is to include the digest component in the key name. If that is the
>>intended way, I suggest to be more explicit about it in the
>>specification.
>
>digest can certainly prevent collisions, but given that NDN uses
>hierarchically structured names including key names, I would like to
>better understand your scenarios of possible key name collisions first,
>if you dont mind elaborate a bit.
>
>> On a slightly related note, high on my wishlist for format/protocol
>>redesigns is the ability to directly use public keys in place of digests
>>or locators, especially since implementations with fast operations and
>>256 bit public keys [NaCl] are available.
>
>this is an interesting suggestion. I think it assumes that some a prior
>steps already occurred that provided one with the keybits of a trusted
>key.




From wm at ars.is  Sun Dec 15 05:25:48 2013
From: wm at ars.is (Michael Weiss)
Date: Sun, 15 Dec 2013 14:25:48 +0100
Subject: [Ndn-interest] NDN Packet Format
In-Reply-To: <E86F9D85-ADD4-4996-98FB-79EC74753DFA@cs.ucla.edu>
References: <529CA14A.9010203@ars.is>
	<E86F9D85-ADD4-4996-98FB-79EC74753DFA@cs.ucla.edu>
Message-ID: <BB129279-061D-43C4-86BA-60B0041DB5DB@ars.is>

Hello Lixia,

On 09.12.2013, at 05:10, Lixia Zhang <lixia at cs.ucla.edu> wrote:

> hmm, not sure I fully understand the implication of this comment.
> the current encoding allows the length field to go up to 8 bytes (for a value field up to 2^64 byte = 1.8X10^19), are you thinking this is not adequate for reasonable future?

I don?t see objects of that size passed over a wire but the Petabyte article on wikipedia already lists collections exceeding 10^17 bytes and somebody might want to define an object of that size to represent a snapshot. At least for software based implementations 128 bit length fields would be cheap within the proposed encoding, but I agree that objects beyond 2^64 octets is deep inside the land of ?somebody else?s problem? and are VERY unlikely to ever hit a router.

>> Interest packet: I have not thought too hard about the computational implications on this one but in general, I'd prefer to have all required fields before the optional fields and thus move to the nonce right after the name and before the selectors.
> 
> we discussed that option earlier, however people more familiar with implementation argued that the selector fields should go first to ease the lookup (e.g. do a hash of name+selector as a lookup key)

With hashing in mind, that makes sense.

> digest can certainly prevent collisions, but given that NDN uses hierarchically structured names including key names, I would like to better understand your scenarios of possible key name collisions first, if you dont mind elaborate a bit.

I now realize that I did not distinguish between PublisherPublicKeyLocator in Interests and KeyLocator in data packets. Should those two have the same type?
I was thinking about generic namespaces like /User or key names without properly versioned names, which would require a series of of interests with exclude filters but I came to the conclusion that the keyspace must be reachable along the path that data travels anyway and that issue is external to the packet format. However, I still think it makes sense to include the digest and require to generate interests for keys with MaxSuffixComponents set to 0 for data validation.
Somehow, I find the issue difficult to discuss on a packet level. Intuitively, the signature and the signing keys are valid for all eternity (while trust in signatures decays). Names on the other hand are rather more fluid and runtime-dependent. It is still possible to represent immutable data in a mutable namespace. For public data one could pick a prefix that specifies an algorithm to derive the signing key from the data bits, for example.
Another simple solution to retrieve keys would be to distinguish keys from data as is done in the ContentType field and use that in interests.

> this is an interesting suggestion. I think it assumes that some a prior steps already occurred that provided one with the keybits of a trusted key.

The idea is to bundle the public key with the signature to have everything for verification without additional lookups. For large objets with a long lifetime that should be a reasonable thing to do. The CCN documentation also mentions this possibility to use short keys based on elliptic curve cryptography but discarded that solution because of longer verification times. The benchmarks (http://bench.cr.yp.to/results-sign.html) show a slowdown of roughly a factor of 2 compared to RSA (ronald3072 vs ed25519) on a comparable security level.
As far as trust is concerned one should try to pass keys as early as possible, which is an interesting topic for URI design and maybe the ability to encode Selectors in URIs could be useful.

> another interesting idea. My thought:
> 1/ yes the type definition requires agreement, but so does protocols with precise fields on each level and their order, right?

Right. No matter which way, one needs to agree on types and their representation and primitive types like unsigned integers or words of various lengths certainly warrant global agreement. The 0.1 spec also defines field order for reasons of performance, i guess.

> 2/ as the 0.1 spec stated, the choice of TLV format is for its flexibility in adding new types and phasing out old types as the protocol evolves over time. a precisely defined protocol would not support incremental changes, right?

Right. Changes to a protocol would (and in my opinion must) constitute a new protocol. This sounds rather rigid but can support evolution if there are mechanisms to negotiate endpoints with different protocols. In NDN it is straightforward to talk about different versions of a protocol simply by naming.
Pseudo examples:
(/ndn/prot/transport/tcp/ver1 ((/ietf/ipv4 1.2.3.4) (/ietf/tcp 9393)))
(/ndn/prot/fragmentation/DelayedAuthentication ((ietf/ipv4 1.2.3.4) (/ietf/udp 22222)))
Components could then ask a trusted software provider for an implementation and start using it. Similarly one could define families of protocols that are parameterized by field lengths or cryptographic algorithms.

>> On a more practical note, my suggestion for such a format is to use Variable Length Values as used by MIDI for length fields, meaning one continuation bit and 7 bits of data in each octet in whatever bit and byte order the signaling folks among you favour. The minimum number of octets MUST be used to ensure uniqueness of representation. The null byte could then be used to represent omitted optional fields.
> 
> we also considered this encoding format earlier, but rejected it due to the processing (which might not best fit hardwire implementations)

That was my guess but, I also guessed that the continuation bits for expected sizes could be hardwired and then use the next continuation bit to signal hand-off to software, but my knowledge of electrical engineering is weak.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.cs.ucla.edu/pipermail/ndn-interest/attachments/20131215/d60af6a5/attachment.html>