gpgsm: ksba_cms_parse failed: Broken pipe

[Jakob Bohm]

Hi,

On 2023-12-14 19:43, Werner Koch wrote:
> On Thu, 14 Dec 2023 16:19, Jakob Bohm said:
> 
>>    zcat ${infl} |
>>      faketime "${DSTAMP}" gpgsm --verify --validation-model shell
>> --assume-binary --status-fd 3 --output - - 3>${wrkdir}/sigdec.status
>> || :
> 
> 
>>      gpgsm: ksba_cms_parse failed: Broken pipe
> 
> gpgsm expects more data but zcat already sent its EOF.  This might be
> due to a broken ASN.1 structure which announced N bytes but gpgsm was
> not able to read thes N bytes.  CMS chunks up the data so that you can
> have a very long stream of data.  (I have sometimes seen chunks of just
> one byte).
> 
> That is the error message you meant, or did you see more?  Adding
> --verbose is often very helpful.
> 
>> Message on status-fd (copied to stderr by cat command)
>>
>>      [GNUPG:] ERROR verify.leave 32877
> 
> This is the machine interface; which you rightfully parse.
> 
>    $ gpg-error 32877
>    32877 = (0, 32877) = (GPG_ERR_SOURCE_UNKNOWN, GPG_ERR_EPIPE) = (Unspecified source, Broken pipe)
> 
> verify.leave helps us to locate the place where the error as emitted:
> Here is is at the end of gpgsm_verify function but unfortunately we
> can't get the offset of the input data where it stopped parsing.  You
> may use "--debug hashing" to write a file with the data hashed
> (i.e. what is verified).  That gives an indication where inthe stream
> the error occured.
> 
> Anothyer useful tool is the venerable dumpasn1 tool which you could use
> instead of gpgsm to find error in the input data structure.
> 
>> Which of the inputs was being truncated on the way to the that
>> parsing function?
> 
> The decompressed $infl .
> 

Thanks for this decoding, based on this I did further testing and found 
the following:

1. The input file was DER encoded, not BER, so no BER chunking, just a
single 1.3 GiB OCTET STRING as inner data. (Checked with dumpasn1 and a
lot of patience), and was actually OK.

2. The same error messages are produced when the /output/ hits a broken
pipe such as piping into head(1) or having a script typo (my case).

So based on 2, it would have been much more helpful if the error message
had stated if the "parser" hit an I/O error on output or input, such as
"output: -: Broken pipe" while the status-fd stream could have said

[GNUPG:] ERROR verify.leave 32877 output - E_PIPE

Preferably also change GPG_ERR_SOURCE_UNKNOWN to some other constant
indicating output failure.

>> It would be much easier if the gpgsm code could be used standalone
>> like the old gpg1 could, with no entanglement to pinentry, dirmgr
> 
> Yeah But for security reasons we have separated tasks in a way similar
> to what postfix does.
> 
>> status-fd protocol etc. etc.  scripted signing and verification of
>> computer generated data is not suited to a human-centric user
> 
> Right, it is a machine interface.  For gpg we have the extra tool gpgv
> to make verification against a known set of keys easy.  And recently we
> added --assert-signer which basically does the same.  There has never
> been a demand to have a dedicated verification tool for CMS but I can
> imagine to add a  --assert-signer also to gpgsm.

My point was that status-fd isn't an easy interface to script against,
but your bug tracker already contains a rejected task to simplify
things.

> 
> Your use of faketime is interesting ;-)
> 

I use faketime to reflect the age of the data being processed, as the
certificates are rotated as necessary and the overall scripts contain a
table mapping origin dates to certificates and cert validity ranges.

Similar tricks may be needed to verify/decrypt stored e-mail based on
the delivery times found in the newest (topmost) "Received:" header.

P.S.

Your mail server ellsberg.gnupg.com appears to be a bit aggressive in
its greylisting (mails rejected first 3 to 4 times), causing NDR
reports to be mailed back to me after 30 minutes.


Enjoy

Jakob
-- 
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded