msvc compiled programs output differently under cygwin tty - c

Question

Under Cygwin: How can I stop output from msvc compiled programs from being transcoded in the tty.
Under Cygwin: gcc vs msvc compiled programs appear to run differently to each other under a tty. Specifically, I am seeing some strange character set translations from only msvc generated binaries output under a tty when the character's 8th bit is set. I'd really like to know how to turn off this annoying behaviour please. Consider:
screen-cap of terminal output (duplicated in a code quote below)
! pwd
/tmp/demo_dir
! echo $LC_ALL "," $LANG "," $LC_CTYPE
, ,
! ./compiled_with_gcc.exe | hexdump
0000000 cece cece
0000004
! ./compiled_with_msvc.exe | hexdump
0000000 cece cece
0000004
! ./compiled_with_gcc.exe
▒▒▒▒!
! ./compiled_with_msvc.exe
╬╬╬╬!
The problem is the last line. The output from the msvc compiled version is not as expected. The two programs are demonstrated above to be outputting the same data: so the last two outputs should be the same. But the tty version (without the pipe) gets changed in only the msvc case. gcc compiled program outputs are passed through the tty unharmed. The output presented here is from the cygwin terminal, but I see exactly the same output difference in xterm.
I am confident it is happenning in the tty not the terminal: because I written a standalone cygwin program in C that runs either the gcc and msvc compiled programs, either under a pipe or under a tty that is not connected to a terminal. The program logs the actual bytes received from the tty.
When running the gcc compiled one, the tty gives the '0xce's bytes as expected.
But a sequence of '0x8ec3' patterns is instead received from the msvc compiled program when listening to it via an identical tty.
When using a pipe instead of a tty, they both output '0xce's.
This notes that the msvc compiled program's output via a tty has an increased width. Given cygwin's preference for UTF-8: it is easy to suspect something is going wrong here and cygwin is causing an extra transcoding that does not happen with gcc compiled programs. I wish to turn that off... How do I successfully disable UTF-8 translations in todays cygwin.
I note that LC_ALL does not appear to be respected to stop this happenning for msvc compiled binaries accessed via a tty. Even when the C program begins with setlocale(0,"");
The output-generating program (to be alternately compiled with the two compilers for the test) is exactly as you'd expect it to be. The same C source in both cases. It simply calls printf or write with some bytes. The msvc version is compiled with Visual Studio 2019 cl.exe (all running on Windows10).
reproduce with:
#ifndef __CYGWIN__
#include <windows.h>
#else
#include <unistd.h>
#endif
#include <io.h>
#include <fcntl.h>
#include <locale.h>
int main()
{
if(!setlocale(LC_ALL, "")) {
return 77; //historically: non-filesystem permission-denied exit-code
}
#ifndef __CYGWIN__
//Irrelevent: But avoids stackexchange users asking for it.
_setmode(1,_O_BINARY);
_set_fmode(_O_BINARY);
#endif
char *dat="\316\316\316\316";
write(1,dat,4); // printf/fflush here gives same results.
return 0;
}
#echo off
:: ugly msvc build script. ms_cl.bat
:: full of M$ hardcoded paths. Likely includes some unused libraries.
:: Load compilation environment
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvars64.bat"
:: Invoke compiler with any options passed to this batch file
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30037\bin\Hostx64\x64\cl.exe" /std:c17 %* kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib
Build msvc version with:
! ms_cl.bat code.c
Run in terminal with:
! ./code.exe | hexdump
! ./code.exe
Build gcc version with:
! gcc code.c
Run in terminal with:
! ./a.exe | hexdump
! ./a.exe
Note identical programs, with identical output in a hex, have output transcoded differently. The msvc one being 'wrong' in my usage case.
I obviously, suspected M$ was doing some translation: so I have tried every combination of _fmode setmode() and more to set binary mode. I suspected some failed cygwin UTF-8 detection situation, so tried setting LC_ALL etc. to plain "C" mode with export in the shell. I similarly tried setting the locale within the msvc source.
Cygwin does a lot of work to make a unix-like environment under windows. Given the hexdumps above I can only guess Cygwin (or some hidden msvc console layer) are doing something quite specialised here and getting in my way. It maybe related to cygwin's migration to ConPty. Either way. I'd like help turning it off.

Answer 1

OP: It's been a little while. And the problem presented in the question presently remains unsolved. However, I have discovered a hacky hack hack from the planet hack that allows for avoiding the problem without solving it. I am posting this non-answer problem-avoidance-hack as an answer and will (eventually) mark it as the solution.. But only if no actual solutions to the problem can be found..
To avoid the output of msvc-compiled program from being transcoded in the tty: first pipe the output of the msvc-compiled program to a gcc compiled program that simply repeats it (such as 'cat' or 'tail -f') and connect that gcc-compiled program to the tty instead.
This hides whatever is going on in the msvc case by seperating it from the tty. The environment is then respected. The tty only knows it is connected to a gcc-compiled program -and works right.
! ./compiled_with_gcc.exe # gcc_compiled->tty = good
▒▒▒▒!
! ./compiled_with_msvc.exe # msvc_compiled->tty = bad
╬╬╬╬!
! ./compiled_with_msvc.exe|cat # msvc_compiled->gcc_compiled->tty = hacky but good
▒▒▒▒!