How to read and print a unicode file - c

Question

I have a test input file input.txt with one line with the following contents:
кёльнский
I am using this code to attempt to read it in and print it out.
#include <locale.h>
#include <stdio.h>
#include <wchar.h>
int main()
{
FILE *input;
wchar_t buf[1000];
setlocale(LC_CTYPE,"");
if ((input = fopen("input.txt","r")) == NULL)
return 1;
printf("Read and print\n");
while (fgetws(buf,1000,input)!=NULL)
wprintf(L"%s",buf);
fclose(input);
}
However when I run it I see "Read and print" and then nothing else.
I am compiling with gcc on Ubuntu.
What am I doing wrong?
It turns out that substituting the wprintf line with
printf("%ls",buf);
fixes the problem.
Why is this?

Answer 1

You're doing two things wrong:
Mixing normal (byte-oriented) and wide output functions to standard output. You need to stick to one or the other. From the C11 draft, section 7.21.2:
Each stream has an orientation. After a stream is associated with an external file, but before any operations are performed on it, the stream is without orientation. Once a wide character input/output function has been applied to a stream without orientation, the stream becomes a wide-oriented stream. Similarly, once a byte input/output function has been applied to a stream without orientation, the stream becomes a byte-oriented stream. ...
Byte input/output functions shall not be applied to a wide-oriented stream and wide character input/output functions shall not be applied to a byte-oriented stream.
Using the wrong printf format to print a wide string. %s is for a normal char string. %ls is for a wchar_t string. But for just printing a wide string to a wide stream, prefer fputws(). No point in using a printf function if you're not actually using its formatting capabilities or mixing literal text with variables or printing wide characters to a byte-oriented stream or something else fancy.
One way (Of many alternatives) to fix the above problems, that treats standard output as a wide-oriented stream:
#include <locale.h>
#include <stdio.h>
#include <wchar.h>
int main(void)
{
FILE *input;
wchar_t buf[1000];
setlocale(LC_CTYPE,"");
if ((input = fopen("input.txt","r")) == NULL)
return 1;
fputws(L"Read and print\n", stdout);
while (fgetws(buf,1000,input)!=NULL)
fputws(buf, stdout);
fclose(input);
}
Another, using a byte-oriented standard output:
#include <locale.h>
#include <stdio.h>
#include <wchar.h>
int main()
{
FILE *input;
wchar_t buf[1000];
setlocale(LC_CTYPE,"");
if ((input = fopen("input.txt","r")) == NULL)
return 1;
puts("Read and print");
while (fgetws(buf,1000,input)!=NULL)
printf("%ls", buf);
fclose(input);
}

Answer 2

fgetws reads UTF-16. We normally handle UTF-8 with normal fgets into char [] and expect it to work and print it with normal printf. That's the point of UTF-8. If it doesn't work on display, probably your terminal isn't UTF-8; this is easily checked by running cat input.txt.