Store multi-byte into char array - c

Question

Following code works:
char *text = "中文";
printf("%s", text);
Then I'm trying to print this text via it's unicode code point which is 0x4e2d for "中" and 0x6587 for "文":
And sure, nothing prints out.
I'm trying to understand what's happening here when I store multi-byte string into char* and how to print multi-byte string with it's unicode code point, and further more, what does it mean by "Format specifier '%ls' requires 'wchar_t *' argument instead of 'wchar_t *'"?
Thanks for any help.
Edit:
I'm on Mac osx (high sierra 10.13.6), with clion
$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix

Answer 1

wchar_t *arr = malloc(2 * sizeof(wchar_t));
arr[0] = 0x4e2d;
arr[1] = 0x6587;
First, the above string is not null-terminated. The printf function knows the beginning of the array, but it has no idea where the array ends, or what size it has. You have to add a zero at the end to make null-terminated C string.
To print this null-terminated wide string, use "printf("%ls", arr);" for Unix based machines (including Mac), use "wprintf("%s", arr);" in Windows (that's a completely different thing, it actually treats the string as UTF16)
Make sure to add setlocale(LC_ALL, "C.UTF-8"); or setlocale(LC_ALL, ""); for Unix based machines.
#include <stdio.h>
#include <stdlib.h>
#include <locale.h>
int main()
{
setlocale(LC_ALL, "C.UTF-8");
//print single character:
printf("%lc\n", 0x00004e2d);
printf("%lc\n", 0x00006587);
printf("%lc\n", 0x0001F310);
wchar_t *arr = malloc((2 + 1)* sizeof(wchar_t));
arr[0] = 0x00004e2d;
arr[1] = 0x00006587;
arr[2] = 0;
printf("%ls\n", arr);
return 0;
}
Aside,
In UTF32, code points always need 4 bytes (example 0x00004e2d) This can be represented with a 4 byte data type char32_t (or wchar_t in POSIX).
In UTF8, code points need 1, 2, 3, or 4 bytes. UTF8 encoding for ASCII characters needs one byte. While 中 needs 3 bytes (or 3 char values). You can confirm this by running this code:
printf("A:%d 中:%d 🙂:%d\n", strlen("A"), strlen("中"), strlen("🙂"));
Se we can't use a single char in UTF8. We can use strings instead:
const char* x = u8"中";
We can use normal string functions in C, like strcpy etc. But some standard C functions don't work. For example strchr just doesn't work for finding 中. This is usually not a problem because characters such as "print format specifiers" are all ASCII and are one byte.