It must sometimes seem that I only write a new blog post when I'm frustrated by some feature of Windows, and I'm sorry to say that's the case once again! It turns out that this is a longstanding issue, but it's one that has only been brought to my attention recently by a report of a surprising behaviour in LB Booster: pressing the Escape key causes the window to close unexpectedly, but only if it incorporates a TEXTBOX or TEXTEDITOR control which has the input focus. Superficially this is very peculiar.

I am grateful to Anatoly Sheyanov for having discovered the true cause, which is - to quote Raymond Chen of 'The Old New Thing' fame - "when you hit ESC in a multiline edit, it posts a WM_CLOSE to its parent in the mistaken belief that it is part of a dialog box.  This is for compatibility with Windows 2.0, which behaved this way.  It's annoying now, but compatibility means having to be bug-for-bug compatible with the previous version". This was written back in 1999 but appears to be still true today.

My frustration stems from the obvious fallacy that this bug needed to be preserved - and still does - "for compatibility with Windows 2.0". It would have been straightforward for Microsoft to introduce a new window style bit (or extended style bit) which, when set, disabled this rogue behaviour and instead caused the Edit Control to send the usual WM_COMMAND message containing the IDCANCEL code. Since applications written for Windows 2.0 shouldn't be setting this undefined bit they would carry in working as before, but new applications could set the bit and get the expected behaviour.

Another approach would have been to fix the bug in the Edit Control but to arrange that Windows applies a compatibility shim unless the program contains a manifest declaring that it isn't needed. This is the standard way that Windows addresses compatibility issues that might arise from the introduction of a new feature or changed behaviour of an existing feature. After all, back in the Windows XP days 'Version 6 Common Controls' were introduced, which were significantly different from their predecessors. To take advantage of them a program needed to declare compatibility in its manifest, so the 'Escape sending WM_CLOSE' bug could have been fixed at that stage.

Unfortunately there aren't any very straightforward workarounds for this bug. The only foolproof solution is to subclass the Edit Control and prevent the Escape keypress even reaching it, which is messy and may be impractical in some languages. The bug is an annoyance and an inconvenience which could have been entirely avoided as long ago as 1999, or before, had Microsoft only chosen to do so.