[RFC] Suggestion for the improvement of the buffering layer

[Jonathan Bastien-Filiatrault]

Nikos Mavrogiannopoulos wrote:
> Jonathan Bastien-Filiatrault wrote:
> 
>> Advantages:
>>
>> - The separation of "message framing" and message processing. Example: A
>> buffer_process function would form as much records as it can from the
>> available transport buffers then a record_read would simply dequeue the
> 
> Hi,
>  If I understand well you propose using something similar to skb's in
> linux?  Can you give me an example of your proposal? Say in the TCP case
> we received a handshake message in 3 different packets and one is not
> yet received. How the processing will go before the 4th is received and
> how after that?

The handshake framing layer would determine how many more bytes it needs
(header present in the first few packets) and compare that with the
number of bytes available in the record FIFO. It would return a code
such as EAGAIN until it has enough data available. When the last packet
is received, it would dequeue the record packets (possibly splitting a
partial record packet in two) and memcpy them into a handshake buffer
and queue the handshake packet in the handshake FIFO. Just to be clear,
there would be a record FIFO for each upper-layer protocol (demultiplex
sub-layer).

Skb's tend to reduce copying to a minimum. In the implementation I
suggest I don't see how I can get away with less that one memcpy per
layer transition (unless I implement some zero-copy record-slicing
code-fu; something I want to avoid).

> 
>> - Better separation of concerns, easier to understand code, better
>> modularity. It can be argued that better understandability, readability
>> and modularity lead to better security (from programming errors).
>> Disadvantages:
>>
>> - Much previously tested/debugged code would be discarded.
> 
> Replacing ugly code with simpler one (or more efficient one) does not
> count as disadvantage to me.

I was hoping someone would say that :P.

> 
>> - Is it worth it ? Would it really be better ?
> 
> I'm curious whether something like that would be more efficient. In
> theory, if I get your idea correct, it would have less memcopy etc but
> in practice we are spending more time in decrypting/encrypting and the
> current code has quite reduced memcpy. However there is still space for
> improvement.

Right, encryption is a few orders of magnitude more costly than memcpy,
 in theory I could therefore get away with more memcpy without a
significant performance degradation (I won't push this too far, hopefully).

> 
> If you are asking a go or not go, I'd say that it's up to you. In any
> case I am curious of any results and more design information.

Well, I have basically brain-dumped the design I had in the email you
replied to. I want to be sure I have the basics right before building a
concrete implementation on top of that.

PS: I have just received the FSF papers, I have signed them and I will
return them soon.

> regards,
> Nikos

Thanks,
Jonathan