In my C program I've had to swap my unicode box-drawing characters into escaped characters for DOS code page 437 to get it to work in the Windows command prompt. Is it possible to change the code page of gnome-terminal to display these characters correctly when natively compiling the program for linux?
Thanks.
From https://nethackwiki.com/wiki/IBMgraphics
The current gnome-terminal does not
have a setting for code page 437, but
it does support other code pages that
are equivalent for NetHack's purposes,
such as 862 (Hebrew).
To set code page 862 on
gnome-terminal:
Select Terminal->Set Character Encoding->Add or Remove.
In the pane on the left, select the line with description Hebrew and
encoding IBM862.
Click the right-pointing arrow between the two panes.
Click Close.
The above steps only need to be done
once for the lifetime of the Gnome
installation. Once done, it is
sufficient to:
Select Terminal, Set Character Encoding, and then Hebrew (IBM862).
It should be noted that the current
default gnome-terminal font in Ubuntu
Jaunty fully supports DECgraphics as
long as eight_bit_tty is set to false.
If you need these characters, you should use their correct Unicode codepoint values and output them as UTF-8. Or, if you prefer, you can output them as wide characters and let the standard library's locale system take care of converting them to UTF-8 or another "native" encoding the user has selected (which might even be CP437, although I've never seen a system setup that poorly...).
Related
Once i write a c program and try to output special characters (like ä ö ü ß) with printf() on the cmd window on windows 10 it only shows sth like ▒▒▒▒▒▒▒▒▒▒▒▒
But if i just type them in the cmd window without a c programm being executed it displays these characters properly.
When i change the console type to standard output in netbeans the output is correct as well.
I tried to change the codepage of cmd but it didnt fix the problem.
I use the gcc c compiler.
The reason is the usage of different code pages for character encoding.
In GUI text editor on writing program code stored in a file on which each character is encoded with just a single byte the code page Windows-1252 is used in Western European and North American countries.
In console window opened on running a console application an OEM code page is used which is in Western European countries OEM 850 and in North American countries OEM 437.
So you need for ÄÖÜäöüß different byte values written in code to get those characters displayed as expected in the console window at least on execution in Western European and North American countries.
Character Windows-1252 OEM 850
Ä \xC4 \x8E
Ö \xD6 \x99
Ü \xDC \x9A
ä \xE4 \x84
ö \xF6 \x94
ü \xF1 \x8C
ß \xDF \xE1
The code page used by default in a console window can be seen by opening a command prompt window and run either chcp (change code page) or mode which both display the active code page.
The default code page for GUI applications and console applications on a computer for a user account depends on the Windows region and language settings for this user account.
Some web pages you should read to better understand character encoding:
Character encoding (English Wikipedia article)
On the Goodness of Unicode by Tim Bray
The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolksy
What's the best default new file format? (UltraEdit forum topic)
Programmers should not write non ASCII characters into strings output by a compiled executable because it depends on which code page is used by the compiler on creating the binary representation (bytes) of the characters in executable. It is better to use the hexadecimal notation when active code page on execution of the application is known or defined by the application before the string is output.
It is also possible to store strings in the executable in Unicode, determine the encoding of the output handle before output any string and convert each Unicode string to the encoding of the output handle before the string is written to the output handle.
And of course it depends on used output font how the bytes in the strings in the executable are finally really displayed on screen.
While using the default console font Raster Fonts 8x12 I am unable to read extra characters using
ReadConsoleOutputCharacter(). These characters will be printed out as ?.
If I change the console font to "Consolas" or "Lucida Console", these extra characters, read by
ReadConsoleOutputCharacter() are printed out without a problem.
Is there anything I can do about that?
Anyway, I fixed it changing the locale after a conversion for a console I/O mapping:
SetConsoleOutputCP(GetACP());
SetConsoleCP(GetACP());
setlocale(LC_ALL, "");
#David Heffernan
I suggest you to read this
According to the docs https://msdn.microsoft.com/en-us/library/windows/desktop/ms684969%28v=vs.85%29.aspx
This function uses either Unicode characters or 8-bit characters from
the console's current code page. The console's code page defaults
initially to the system's OEM code page. To change the console's code
page, use the SetConsoleCP or SetConsoleOutputCP functions, or use the
chcp or mode con cp select= commands.
I believe you're getting back a unicode string that needs to be encoded to a charset before trying to be displayed.
I'm trying to make a (somewhat) stylish DOS menu as a present for my father.
I was able to get the whole menu system to work, but I wanted to gussy it up with some box drawing characters and, possibly, colored text.
In this YouTube video, the user shows an example of what I'm trying to do (example at the 5:00 mark), but doesn't explain how those characters are being rendered. In the Notepad document, it is displayed as goofy characters.
Do I need to save the file with a special type of encoding? Can it only be done in Notepad (I'm using TextEdit on Mac)? Can someone provide an example menu that can be added to DOSBox's [autoexec] config?
Also, I'm not sure if it is possible, but how can the text color/background color be changed? When running DOSBox initially, it shows their welcome screen with a blue background and box drawing characters, so I would think all of that is possible.
I tried using escaped unicode characters and I tried using a capital-E acute (as shown in the linked video), but they just render funky stuff when run in DOSBox.
The discrepancy in characters is a result of different code pages being used in character rendering. English-speaking Windows uses ANSI code page 1252 (otherwise known as Latin-1), while DOS uses OEM code page 437, or IBM-PC.
The codepage that Windows uses will vary based on your system language, so you many need to experiment to find the correct characters, but basically, find the character you want to print in 437 (say ╔, which is 200) and then in your code use the 1252 version (where 200 is È). Then save the file in ANSI encoding.
So, I'm trying to get some input from the user in a C program, doing fscanf(stdin, "%s", buffer)
When I input the character å i get a value of 134 which corresponds to the codepage 437.
But when i use the windows function GetACP() i get 1252 as the active codepage and 134 doesn't match å in that codepage. I tried setting the codepage to UTF-8 but that didn't give me any input at all.
Is there a way of getting the corresponding codepage for user input and convert that to unicode format? Or if there's a better way of getting the input.
I've been looking around a lot and I can't find much info on this.
The code page used by the console window is called the OEM code page for historical reasons. You can get the default code page with GetOEMCP and the currently selected code page with GetConsoleCP.
You can set the console to use UTF-8 with the command chcp 65001, but Microsoft does not guarantee it to work in all cases.
If you don't need normal C++ I/O to the console, you can use the Console Functions instead e.g. WriteConsoleW to output a Unicode string.
How can I print foreign characters on the screen using C?
Here's my code, which doesn't work:
#include <stdio.h>
#include <locale.h>
int main(){
setlocale(LC_ALL,"Turkish");
printf("İ ş ğ ü ö ı");
system("pause");
return 0;
}
On my Windows there is no such characters in the 'Terminal' font. I think you can't print them.
But I suggest you to check this font yourself. Maybe you have a different version of it.
If you're using a narrow charset then you need to make sure that the terminal/console is using the same charset and the source code file is encoded in the correct encoding, otherwise of course the system will misinterpret the character codes
To set the charset in the console run chcp. For example to use code page Windows-1254 run chcp 1254. You can use SetConsoleOutputCP to set the code page programmatically, like SetConsoleOutputCP(1254)
However you should avoid the legacy ANSI code pages and use Unicode instead. The current preferred way on Windows is to output Unicode characters as wide char with wprintf. You may need to set the mode to wide first with
int result = _setmode(_fileno(stdout), _O_U16TEXT);
then
wprintf(L"İ ş ğ ü ö ı");
See also wprintf manual in Windows, Linux or Mac. However on POSIX systems UTF-8 is preferred
On older Windows UTF-8 support on console is not very good, but it's increasingly getting better, and Windows 10 even supports UTF-8 as a locale so you can just call SetConsoleOutputCP(CP_UTF8); or SetConsoleOutputCP(65001); (or run chcp 65001 in the console) and it'll work immediately, provided that you saved the source code as UTF-8. Remember to also set the font to the one that supports those characters like Lucida Console or Consolas. The default raster font contains very a limited number of characters and appears with a lot of aliasing. It also doesn't work well on modern hidpi displays
There are already lots of questions about outputting Unicode on this site like Output unicode strings in Windows console app or UTF-8 character in .NET Console Application. Please have a look and try to see which one fits you.
Edit
When you use
wchar_t c=L'ğ';
fputwc(c,ptr);
you're printing to a file and not the console. In that case just the stream of bytes is saved into the file. When you open the file again, it's the job of the editor to treat the bytes in the correct charset and print it correctly. For example the character "ğ" is stored as c4 9f in UTF-8 and when open the file as UTF-8, the editor knows that it represents the char "ğ" to display
Unfortunately there's no character encoding information embedded in a text file so the editor must choose one. Remember There Ain't No Such Thing As Plain Text (must read). A simple editor may just choose to open the file as ANSI in the current Windows codepage and the characters won't be displayed correctly if the original encoding is not that one and you'll just see garbage
Some more advanced editors like Notepad++ or MS Word will try to guess the encoding of the file. But as with any guessing, it can be wrong and the result is again a file with garbage
The simplest solution is to add a BOM to the beginning of the file so the editor can recognize the encoding easily. If your files doesn't contain a BOM you need to tell the editor to read the file in the correct encoding if the encoding is wrong (for wchar_t on Windows like that it's UTF-16LE). For example in Notepad++ it's this menu
Unfortunately the OP didn't edit the question to show what was tried, there's nothing more I can explain
Your code works: http://ideone.com/K9hrv5
setlocale(LC_ALL,"Turkish");
printf("İ ş ğ ü ö ı");
The only issue is that you have to set your terminal locale as well before executing your c program's output binary.
Setting your terminal locale works:
setlocale only affects the runtime locale. It doesn't make your compiler support extra source file characters.
You may need to specify the non-ASCII characters in your source file by using character constants (e.g. \xF1 for the character with code 241).