gh-78319: add UTF8 marker per RFC

[gordonmessmer]

This change implements RFC 6855 UTF8 APPEND per guidance from Sam Varshavchik:
https://bugs.python.org/issue34138

https://bugs.python.org/issue34138

• Issue: imaplib RFC 6855 issue #78319

[the-knights-who-say-ni]

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately we couldn't find an account corresponding to your GitHub username on bugs.python.org (b.p.o) to verify you have signed the CLA (this might be simply due to a missing "GitHub Name" entry in your b.p.o account settings). This is necessary for legal reasons before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for your contribution, we look forward to reviewing it!

[gordonmessmer]

I thought I'd signed the CLA previously... I've signed it today. Please let me know what other steps I can take to help merge this patch.

[bedevere-bot]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[arhadthedev]

@gordonmessmer tests are failing (see Details links under Some checks were not successful section). Would you mind to continue working on the PR, please?

[bedevere-bot]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[gordonmessmer]

I'll look at it again, yes. I'm getting odd results from updated tests, so some additional work is still needed...

[gordonmessmer]

Tests look like they're passing, but trying this on a live IMAP server fails, because the server actually receives:

LLFC3 APPEND INBOX UTF8 (~{63}\r\n

... when Sam suggested that it should receive:

LLFC3 APPEND INBOX NIL NIL UTF8 (~{63}\r\n

But that's probably a larger bug in the imaplib module, and not directly related to this change.

[arhadthedev]

@warsaw, @maxking (as active e-mail experts)

[gordonmessmer]

I've asked Sam to chime in. The RFC calls those arguments optional. Based on its behavior, it seems that Courier expects them to be specified as NIL when they are not provided, rather than left out entirely.

[gordonmessmer]

I added a commit for consideration. Transforming None to "NIL" is RFC-compliant (which is consistent with the intent of this PR). Tests pass, and this works with Courier IMAP.

[gordonmessmer]

The NIL requirement may be a bug in Courier (ironically also caused by UTF8 support), so the "imaplib: transform None into "NIL"" patch might not be needed.

Everything else looks good, I think.

[gordonmessmer]

Sam released a new version of Courier, and I've verified that the None->NIL transformation is no longer necessary, so I've backed it out.

[gordonmessmer]

Please let me know if there's anything I can do to help move this forward.