[gnutls-devel] GnuTLS | Multiple issues with handling record_size_limit extension (#676)

[Development of GNU's TLS library]

New Issue was created.

Issue 676: https://gitlab.com/gnutls/gnutls/issues/676
Author:    Hubert Kario
Assignee:  

## Description of problem:
There are multiple issues related to handling of the record_size_limit extension, but since there's just one reproducer script, I'm filing a single bug.

### Individual issues
1. record_size_limit extension does not influence renegotiation handshake records or is applied right-away after receiving the new ClientHello, while RFC states "Records are subject to the limits that were set in the handshake that produces the keys that are used to protect those records." this causes `renegotiation with dropped extension` and `renegotiation with changed limit` to fail with record_overflow
1. gnutls sends smaller than possible value in TLS 1.3 - the limit in TLS 1.3 is 2**14 + 1. This causes `HRR sanity`, `change size in TLS 1.3 session resumption`, `check if server accepts maximum size in TLS 1.3`, `too large record payload in TLS 1.3`, `check server sent size in TLS 1.3`, `drop extension in TLS 1.3 session resumption` to fail
1. a modified extension in 2nd CH in HRR handshake is not detected and handshake is not aborted. Test case `modified extension in 2nd CH in HRR handshake`.
1. malformed extension, with bigger payload than specified is not rejected. This causes `padded extension in TLS 1.3` and `padded extension in TLS 1.2` to fail
1. when extension is negotiated, the 1/n-1 record splitting in TLS 1.0 is disabled, this causes `check if server accepts maximum size in TLS 1.0` and `check server sent size in TLS 1.0` to fail
1. Too large application_data records in TLS 1.2 are not detected when the extension is negotiated. This causes `too large record in TLS 1.2` to fail
1. GnuTLS does not advertise the extension during session resumption. This causes `change size in TLS 1.2 resumption` to fail
1. the server sends too large records when minimal size is advertised by client (ignores the extension?!). This causes `check if server accepts minimal size in TLS 1.2`, `check if server accepts minimal size in TLS 1.1` and `check if server accepts minimal size in TLS 1.0` to fail

## Version of gnutls used:
b527be10f829e 

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)
local compile on Fedora 28 

## How reproducible:

chekout https://github.com/tomato42/tlsfuzzer/pull/494
run `test-record-size-limit.py --reply-AD-size 821`

## Actual results:
```
Test end
successful: 12
failed: 21
  'change size in TLS 1.2 resumption'
  'change size in TLS 1.3 session resumption'
  'check if server accepts maximum size in TLS 1.0'
  'check if server accepts maximum size in TLS 1.3'
  'check if server accepts minimal size in TLS 1.0'
  'check if server accepts minimal size in TLS 1.1'
  'check if server accepts minimal size in TLS 1.2'
  'check if server accepts minimal size in TLS 1.3'
  'check interaction with sha256 prf'
  'check interaction with sha384 prf'
  'check server sent size in TLS 1.0'
  'check server sent size in TLS 1.3'
  'drop extension in TLS 1.3 session resumption'
  'HRR sanity'
  'modified extension in 2nd CH in HRR handshake'
  'padded extension in TLS 1.2'
  'padded extension in TLS 1.3'
  'renegotiation with changed limit'
  'renegotiation with dropped extension'
  'too large record in TLS 1.2'
  'too large record payload in TLS 1.3'
```

## Expected results:

```
successful: 33
failed: 0
```

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/issues/676
You're receiving this email because of your account on gitlab.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20190116/c6c2cd6e/attachment.html>