wallace breaks dkim signature
Skale, Franz
i.bin at dah.am
Sat Jun 2 09:08:21 CEST 2018
Hi Jan,
respamd must take care of the data, regardless of which encoding stream
injected.
As i develop webservices, you've to guess the encoding prior before
mangling the data stream.
Whit utf-8 you have the ability to call is_utf8 which tells you, if the
the utf8 (2 bytes) representation) is either native or encoded.
Input has to be encoded and output decoded.
Regarding your problem, i think that respamd doesn't check, if the input
stream is alreasy quoted-printable encoded.
I think you've to file a bug report.
Pykolab only set's encoding for string handling, obviously the way to
go.
/usr/lib/python2.7/dist-packages/wallace/module_resources.py:
charset.add_charset('utf-8', charset.SHORTEST, charset.QP)
/usr/lib/python2.7/dist-packages/wallace/module_resources.py: msg
= MIMEText(utils.stripped_message(message_text), _charset='utf-8')
So, it's utf-8 encoded.
Ergo, respamd has to decode the input stream (utf8_decode) when it's not
in native encoded.
The client sets the msg_content_type (Content-Type).
So, if the client uses wrong localesm than , of course the string
representation is wrong.
Respamd has to take care about decoding the string according to the
Content-Type.
I think you've to file a bug report for respamd.
Rgds.
Franz
Am 2018-06-02 03:10, schrieb Jan Kowalsky:
> Hi Franz,
>
> Am 01.06.2018 um 11:17 schrieb Skale, Franz:
>> Hi,
>> i see no mangling of the message body other than setting the default
>> locale to UTF-8 then encode it quoted printable. The header will be
>> parsed and changed (invitation etc.).
>
> yes, that's it. It's changing the Content-Transfer-Encoding to
> quoted-printable. If this is done after dkim signing it breaks
> signature.
>
>> It the message contains html, it will be parsed too.
>> Since you didn't supply a debug example i can only urge you to enable
>> wallace debugging.
>
> I can't see anything in debug log - except if smtplib is called the
> data
> is already qouted printable:
>
>
> 2018-06-02 01:25:54,004 pykolab.wallace INFO Accepted connection
> 2018-06-02 01:25:54,018 pykolab.wallace DEBUG [8771]: Resource
> Management called for ('/var/spool/pykolab/wallace/tmpk9DNom',), {}
> 2018-06-02 01:25:54,019 pykolab.wallace DEBUG [8771]: Renaming
> '/var/spool/pykolab/wallace/tmpk9DNom' to
> '/var/spool/pykolab/wallace/resources/incoming/tmpk9DNom'
> 2018-06-02 01:25:54,020 pykolab.wallace DEBUG [8771]: Nachricht ist
> keine iTip Nachricht (keine Multipart Nachricht)
> 2018-06-02 01:25:54,020 pykolab.wallace INFO Message is not an iTip
> message or does not contain any (valid) iTip.
> 2018-06-02 01:25:54,020 pykolab.wallace DEBUG [8771]: No itips, no
> resources, pass along
> '/var/spool/pykolab/wallace/resources/incoming/tmpk9DNom'
> 2018-06-02 01:25:54,020 pykolab.wallace DEBUG [8771]: Invitation policy
> called for
> ('/var/spool/pykolab/wallace/resources/incoming/tmpk9DNom',), {}
> 2018-06-02 01:25:54,021 pykolab.wallace DEBUG [8771]: Invitation policy
> executing for
> '/var/spool/pykolab/wallace/resources/incoming/tmpk9DNom',
> False
> 2018-06-02 01:25:54,021 pykolab.wallace DEBUG [8771]: Renaming
> '/var/spool/pykolab/wallace/resources/incoming/tmpk9DNom' to
> '/var/spool/pykolab/wallace/invitationpolicy/incoming/tmpk9DNom'
> 2018-06-02 01:25:54,021 pykolab.wallace DEBUG [8771]: Nachricht ist
> keine iTip Nachricht (keine Multipart Nachricht)
> 2018-06-02 01:25:54,022 pykolab.wallace INFO Message is not an iTip
> message or does not contain any (valid) iTip objects.
> 2018-06-02 01:25:54,022 pykolab.wallace DEBUG [8771]: No itips, no
> users, pass along
> '/var/spool/pykolab/wallace/invitationpolicy/incoming/tmpk9DNom'
> 2018-06-02 01:25:54,022 pykolab.wallace INFO Akzeptiere Nachricht in
> /var/spool/pykolab/wallace/invitationpolicy/incoming/tmpk9DNom (durch
> Modul wallace)
> 2018-06-02 01:25:54,022 pykolab.wallace DEBUG [8771]: Akzeptiere
> Nachricht in:
> '/var/spool/pykolab/wallace/invitationpolicy/incoming/tmpk9DNom'
> 2018-06-02 01:25:54,023 pykolab.wallace DEBUG [8771]: recipients:
> ['test.user at example.net']
> send: 'ehlo mx0.example.net\r\n'
> reply: '250-mx0.example.net\r\n'
> reply: '250-PIPELINING\r\n'
> reply: '250-SIZE 20480000\r\n'
> reply: '250-VRFY\r\n'
> reply: '250-ETRN\r\n'
> reply: '250-STARTTLS\r\n'
> reply: '250-XFORWARD NAME ADDR PROTO HELO SOURCE PORT IDENT\r\n'
> reply: '250-ENHANCEDSTATUSCODES\r\n'
> reply: '250-8BITMIME\r\n'
> reply: '250 DSN\r\n'
> reply: retcode (250); Msg: mx0.example.net
> PIPELINING
> SIZE 20480000
> VRFY
> ETRN
> STARTTLS
> XFORWARD NAME ADDR PROTO HELO SOURCE PORT IDENT
> ENHANCEDSTATUSCODES
> 8BITMIME
> DSN
> send: 'mail FROM:<test.user at example.net> size=1123\r\n'
> reply: '250 2.1.0 Ok\r\n'
> reply: retcode (250); Msg: 2.1.0 Ok
> send: 'rcpt TO:<test.user at example.net>\r\n'
> reply: '250 2.1.5 Ok\r\n'
> reply: retcode (250); Msg: 2.1.5 Ok
> send: 'data\r\n'
> reply: '354 End data with <CR><LF>.<CR><LF>\r\n'
> reply: retcode (354); Msg: End data with <CR><LF>.<CR><LF>
> data: (354, 'End data with <CR><LF>.<CR><LF>')
> send: 'Sender: test.user at example.net\r\nDKIM-Signature: v=1;
> a=rsa-sha256; c=relaxed/relaxed; d=example.net;\r\n s=dkim201805;
> t=1527895386;\r\n
> h=from:from:sender:sender:reply-to:subject:subject:date:date:\r\n
> message-id:message-id:to:to:cc:mime-version:mime-version:\r\n
> content-type:content-type:\r\n
> content-transfer-encoding:content-transfer-encoding:in-reply-to:\r\n
> references; bh=IYC/RDnaNtcM6arkuRe/LIW86LUe+V8zvrkFPp/dOoY=;\r\n
> b=XjBEXPP/CfSc9RqxX6G+zVW0gorAevrouaSNdQXIx2GhJVvUvheJszeils1SKtRYV7h3oK\r\n
> ricUH0upeecCDgQJPyGc90aY/JwsoLs2ZpANomt53fQxOJiSyIiuqGbRAyZgsddK0BoW77\r\n
> +TpL2Xatf9c5u017mxvzAWJngXzD52hV7txlM/gKcGy3SZR48F74JNyGdIJX3qmMBe0dSo\r\n
> oGj6g0YHN4nTdtvJ995J7eYYgofUJlUglOezF58rQV7n4Vh44pncZZ+vMDKNaQ2h9eKPw6\r\n
> AcypUALlZvnssOWyuHRBzHqy7Aet2F9dH8sBvOAZCxcLnuOlC8kruqNAF34/WA==\r\nTo:
> Test User <test.user at example.net>\r\nFrom: cu-test
> <test.user at example.net>\r\nSubject: test encoding 2\r\nMessage-ID:
> <7b2f68db-4e2a-1762-a223-08baaef3a380 at example.net>\r\nDate: Sat, 2 Jun
> 2018 01:25:50 +0200\r\nMIME-Version: 1.0\r\nContent-Type: text/plain;
> charset=utf-8\r\nContent-Language: de-LU\r\nContent-Transfer-Encoding:
> quoted-printable\r\n\r\ntest encoding2\r\n=C3=A4=C3=B6=C3=BC\r\n.\r\n'
> reply: '250 2.0.0 Ok: queued as 09238B14\r\n'
> reply: retcode (250); Msg: 2.0.0 Ok: queued as 09238B14
> data: (250, '2.0.0 Ok: queued as 09238B14')
> send: 'quit\r\n'
> reply: '221 2.0.0 Bye\r\n'
> reply: retcode (221); Msg: 2.0.0 Bye
> 2018-06-02 01:25:54,153 pykolab.wallace DEBUG [9175]: Worker process
> PoolWorker-7 initializing
>
>
>> Be sure that your default locale on the server is utf-8 !
>
> yes. it is.
>
> What I actually don't understand: Where exactly wallace is changing the
> encoding? I send email with 8bit encoding and utf-8.
>
> This is my original mail:
>
>
> Date: Sat, 2 Jun 2018 01:25:50 +0200
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
> Thunderbird/52.8.0
> MIME-Version: 1.0
> Content-Type: text/plain; charset=utf-8
> Content-Language: de-LU
> Content-Transfer-Encoding: 8bit
>
> test encoding2
> äöü
>
> (encoded in utf-8 - not quoted printable)
>
> I found the function for converting in the footer and invitation
> modules. But even if I disable this modules the encoding still is
> changed. It's not if I disable wallace in postfix master.cf. I
> wondering
> if this is done by python smtplib.
>
> My submission part of master.cf:
>
> submission inet n - n - -
> smtpd
> -o cleanup_service_name=cleanup_submission
> -o syslog_name=postfix/submission
> -o smtpd_tls_security_level=encrypt
> -o smtpd_sasl_auth_enable=yes
> -o smtpd_sasl_authenticated_header=yes
> -o smtpd_client_restrictions=permit_sasl_authenticated,reject
> -o smtpd_helo_restrictions=$mua_helo_restrictions
> -o smtpd_data_restrictions=$submission_data_restrictions
> -o smtpd_recipient_restrictions=$submission_recipient_restrictions
> -o smtpd_sender_restrictions=$submission_sender_restrictions
> -o content_filter=smtp-wallace:[127.0.0.1]:10026
>
> Because I don't have amavis I call wallace directly in submission.
>
> Milter is called in main.cf:
>
> smtpd_milters = inet:mailpd.example.net:11332
> non_smtpd_milters = inet:mailpd.example.net:11332
> milter_protocol = 6
> milter_mail_macros = i {mail_addr} {client_addr} {client_name}
> {auth_authen}
> milter_default_action = accept
>
>
> So the mailflow now is: going to prequeue-miltering and signs dkim.
> After that there is the content filter set to wallaces - which alteres
> the message and breaks signature.
>
>
>> Also, reordering of the content_filter directive might help.
>
> It's not easy, because wallace always is content filter - and milter is
> prequeue (opendkim or rspamd doesn't matter).
>
> What I tried: using rspamd as a content_filter which is possible. So I
> could pass mails from submission to wallace:
>
> submission inet n - n - -
> smtpd
> -o cleanup_service_name=cleanup_submission
> -o syslog_name=postfix/submission
> -o smtpd_tls_security_level=encrypt
> -o smtpd_sasl_auth_enable=yes
> -o smtpd_sasl_authenticated_header=yes
> -o smtpd_client_restrictions=permit_sasl_authenticated,reject
> -o smtpd_helo_restrictions=$mua_helo_restrictions
> -o smtpd_data_restrictions=$submission_data_restrictions
> -o smtpd_recipient_restrictions=$submission_recipient_restrictions
> -o smtpd_sender_restrictions=$submission_sender_restrictions
> # overwrite the default miter - we can't do that on submission,
> because we have first go to wallace
> -o smtpd_milters=
> -o content_filter=smtp-wallace:[127.0.0.1]:10026
>
> And then in wallace:
>
> # Listener to re-inject email from Wallace into Postfix
> 127.0.0.1:10027 inet n - n - 100
> smtpd
> -o cleanup_service_name=cleanup_internal
> -o content_filter=
> -o local_recipient_maps=
> -o relay_recipient_maps=
> -o smtpd_milters=
> -o smtpd_restriction_classes=
> -o smtpd_client_restrictions=
> -o smtpd_helo_restrictions=
> -o smtpd_sender_restrictions=
> -o smtpd_recipient_restrictions=permit_mynetworks,reject
> -o mynetworks=127.0.0.0/8
> -o smtpd_authorized_xforward_hosts=127.0.0.0/8
> -o content_filter=smtp:[127.0.0.1]:2525
>
> in the last line I call the rspamd content filter which I define hiere:
>
> # rspamd as content filter
> 127.0.0.1:2525 inet n - n - - smtpd
> -o syslog_name=postfix/content-filter
> -o mynetworks=127.0.0.0/8
> -o content_filter=
> -o smtpd_milters=${rspamd}
> -o smtpd_tls_security_level=none
> -o smtpd_client_restrictions=
> -o smtpd_helo_restrictions=
> -o smtpd_sender_restrictions=
> -o smtpd_recipient_restrictions=permit_mynetworks,reject
> -o smtpd_relay_restrictions=permit_mynetworks,reject
> -o smtpd_authorized_xforward_hosts=${mynetworks}
>
>
> Now everything works - but on smtp we have the mailflow:
>
> -> rspamd as milter -> wallace -> rspamd as content_filter
>
> I haven't got any Idea how to either to call rspamd content filter in
> case of submission but not of smtpd. Or have one directive where both
> content filters are called. As far as I understand there is no
> possility
> in postfix to add more then one content_filter without reinjection. But
> even in this case - still the problem to distinguish between smtpd and
> submission.
>
> Regard
> Jan
>
>> Am 2018-06-01 10:08, schrieb Jan Kowalsky:
>>> Hi Franz,
>>>
>>> thanks for answer,
>>>
>>> Am 01.06.2018 um 08:44 schrieb Skale, Franz:
>>>> Hi,
>>>> DKIM uses non standard quoted printable encoding !
>>>> See:
>>>> http://dkim.org/specs/rfc4871-dkimbase.html#dkim-qp
>>>
>>> But as far as I understand, dkim itselfs doesn't change the mail body
>>> at
>>> all. The qp encoding in my understanding is only for calculating
>>> hash.
>>>
>>>> Wallace, of course, uses standard QP encoding and doesn't take care
>>>> for
>>>> special cases, though it's a feature request.
>>>
>>> But wallaces alters the mailbody itself. It changes all to quoted
>>> printable and, if configured, adds footer/header.
>>>
>>> So dkim signing in my experiences is valid if the email already is
>>> quoted printable from the mua. Roundcube is fine, thunderbird in
>>> default
>>> not. But we do not have control about how users are configure there
>>> email programs and 8bit email transfer isn't very uncommon any more.
>>>
>>> The problem is, that wallace alteres mails after dkim milter is
>>> applied.
>>> Even if the encoding is no problem we run in trubles as far as
>>> wallace
>>> adds e.g. footers.
>>>
>>> Regards
>>> Jan
> _______________________________________________
> users mailing list
> users at lists.kolab.org
> https://lists.kolab.org/mailman/listinfo/users
More information about the users
mailing list