Simple encryption/decryption algorithm causing EOF - c

I was playing with very simple encryption/decryption algorithm like this;
#include <stdio.h>
#include <stdlib.h>
#define BUFFESIZE 1024
int main(int argc, char *argv[]) {
int keylen = 0;
char *key = argv[1];
char *buffer = NULL;
size_t buffersize = 0;
size_t nbytes = 0;
size_t nread;
int i = 0;
while(*key++ != 0) keylen++;
key = argv[1];
do {
buffersize+=BUFFESIZE;
buffer = realloc(buffer, buffersize);
nread = fread(buffer+nbytes, 1, BUFFESIZE, stdin);
nbytes+=nread;
} while (nread > 0);
for(i=0; i<nbytes; i++) {
putchar(buffer[i] ^ key[i % keylen]);
}
return 0;
}
Encyption key is the first command-line argument to the program. I expect that this should get me originial file when encrypted/decrypted with same key. However, I sometimes get only small amount of the file back if I encrypt/decrypt it. My guess is that algorithm adds EOF control character in the middle of file.
How can I get around this problem?
I compiled this using MinGW gcc 4.8.1 on windows XP. If you're interested, you can find a sample input file demonstrating the problem in the edit history of this question.

Well, your code works for me on Linux (compiled with GCC 4.8.2), even using your sample input and key. This suggests that the issue is specific to Windows — most likely, that it's caused by stdin and stdout being in text mode by default. (On Linux and other Unix-ish systems, there's usually no difference between text mode and binary mode, so such issues do not show up there.)
To fix it, you will need to set stdin and stdout to binary mode. The standard way of doing this, as of C99, would be:
freopen(NULL, "rb", stdin);
freopen(NULL, "wb", stdout);
but alas, according to the answers in the thread I linked to above, the Windows C library does not support this C99 feature, so you'll need to fall back on the non-standard _setmode() instead:
_setmode(_fileno(stdin), _O_BINARY);
_setmode(_fileno(stdout), _O_BINARY);
If you want to remain portable, you could always use some conditional code, e.g. like this (warning: not actually tested on Windows!):
#if __STDC_VERSION__ >= 199901L
#define binmode(fh, w) freopen(NULL, ((w) ? "wb" : "rb"), (fh)) /* C99 */
#elif _MSC_VER >= 1200
#include <io.h>
#include <fcntl.h>
#define binmode(fh, w) _setmode(_fileno(fh), _O_BINARY) /* MSVC 6.0+ */
#elif defined(__unix__) || (defined(__APPLE__) && defined(__MACH__))
#define binmode(fh, w) /* Unix-ish, just do nothing */
#else
#error Not sure how to define binmode() on this platform
#endif
binmode(stdin, 0);
binmode(stdout, 1);
Or, of course, you could just sidestep the whole issue by opening your own input and output files (in binary mode) instead of using stdin and stdout.

Related

Why can't my editor understand characters like é, è, à [duplicate]

My setup: gcc-4.9.2, UTF-8 environment.
The following C-program works in ASCII, but does not in UTF-8.
Create input file:
echo -n 'привет мир' > /tmp/вход
This is test.c:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define SIZE 10
int main(void)
{
char buf[SIZE+1];
char *pat = "привет мир";
char str[SIZE+2];
FILE *f1;
FILE *f2;
f1 = fopen("/tmp/вход","r");
f2 = fopen("/tmp/выход","w");
if (fread(buf, 1, SIZE, f1) > 0) {
buf[SIZE] = 0;
if (strncmp(buf, pat, SIZE) == 0) {
sprintf(str, "% 11s\n", buf);
fwrite(str, 1, SIZE+2, f2);
}
}
fclose(f1);
fclose(f2);
exit(0);
}
Check the result:
./test; grep -q ' привет мир' /tmp/выход && echo OK
What should be done to make UTF-8 code work as if it was ASCII code - not to bother how many bytes a symbol takes, etc. In other words: what to change in the example to treat any UTF-8 symbol as a single unit (that includes argv, STDIN, STDOUT, STDERR, file input, output and the program code)?
#define SIZE 10
The buffer size of 10 is insufficient to store the UTF-8 string привет мир. Try changing it to a larger value. On my system (Ubuntu 12.04, gcc 4.8.1), changing it to 20, worked perfectly.
UTF-8 is a multibyte encoding which uses between 1 and 4 bytes per character. So, it is safer to use 40 as the buffer size above.
There is a big discussion at How many bytes does one Unicode character take? which might be interesting.
Siddhartha Ghosh's answer gives you the basic problem. Fixing your code requires more work, though.
I used the following script (chk-utf8-test.sh):
echo -n 'привет мир' > вход
make utf8-test
./utf8-test
grep -q 'привет мир' выход && echo OK
I called your program utf8-test.c and amended the source like this, removing the references to /tmp, and being more careful with lengths:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define SIZE 40
int main(void)
{
char buf[SIZE + 1];
char *pat = "привет мир";
char str[SIZE + 2];
FILE *f1 = fopen("вход", "r");
FILE *f2 = fopen("выход", "w");
if (f1 == 0 || f2 == 0)
{
fprintf(stderr, "Failed to open one or both files\n");
return(1);
}
size_t nbytes;
if ((nbytes = fread(buf, 1, SIZE, f1)) > 0)
{
buf[nbytes] = 0;
if (strncmp(buf, pat, nbytes) == 0)
{
sprintf(str, "%.*s\n", (int)nbytes, buf);
fwrite(str, 1, nbytes, f2);
}
}
fclose(f1);
fclose(f2);
return(0);
}
And when I ran the script, I got:
$ bash -x chk-utf8-test.sh
+ '[' -f /etc/bashrc ']'
+ . /etc/bashrc
++ '[' -z '' ']'
++ return
+ alias 'r=fc -e -'
+ echo -n 'привет мир'
+ make utf8-test
gcc -O3 -g -std=c11 -Wall -Wextra -Werror utf8-test.c -o utf8-test
+ ./utf8-test
+ grep -q 'привет мир' $'в?\213?\205од'
+ echo OK
OK
$
For the record, I was using GCC 5.1.0 on Mac OS X 10.10.3.
This is more of a corollary to the other answers, but I'll try to explain this from a slightly different angle.
Here is Jonathan Leffler's version of your code, with three slight changes: (1) I made explicit the actual individual bytes in the UTF-8 strings; and (2) I modified the sprintf formatting string width specifier to hopefully do what you are actually attempting to do. Also tangentially (3) I used perror to get a slightly more useful error message when something fails.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define SIZE 40
int main(void)
{
char buf[SIZE + 1];
char *pat = "\320\277\321\200\320\270\320\262\320\265\321\202"
" \320\274\320\270\321\200"; /* "привет мир" */
char str[SIZE + 2];
FILE *f1 = fopen("\320\262\321\205\320\276\320\264", "r"); /* "вход" */
FILE *f2 = fopen("\320\262\321\213\321\205\320\276\320\264", "w"); /* "выход" */
if (f1 == 0 || f2 == 0)
{
perror("Failed to open one or both files"); /* use perror() */
return(1);
}
size_t nbytes;
if ((nbytes = fread(buf, 1, SIZE, f1)) > 0)
{
buf[nbytes] = 0;
if (strncmp(buf, pat, nbytes) == 0)
{
sprintf(str, "%*s\n", 1+(int)nbytes, buf); /* nbytes+1 length specifier */
fwrite(str, 1, 1+nbytes, f2); /* +1 here too */
}
}
fclose(f1);
fclose(f2);
return(0);
}
The behavior of sprintf with a positive numeric width specifier is to pad with spaces from the left, so the space you tried to use is superfluous. But you have to make sure the target field is wider than the string you are printing in order for any padding to actually take place.
Just to make this answer self-contained, I will repeat what others have already said. A traditional char is always exactly one byte, but one character in UTF-8 is usually not exactly one byte, except when all your characters are actually ASCII. One of the attractions of UTF-8 is that legacy C code doesn't need to know anything about UTF-8 in order to continue to work, but of course, the assumption that one char is one glyph cannot hold. (As you can see, for example, the glyph п in "привет мир" maps to the two bytes -- and hence, two chars -- "\320\277".)
This is clearly less than ideal, but demonstrates that you can treat UTF-8 as "just bytes" if your code doesn't particularly care about glyph semantics. If yours does, you are better off switching to wchar_t as outlined e.g. here: http://www.gnu.org/software/libc/manual/html_node/Extended-Char-Intro.html
However, the standard wchar_t is less than ideal when the standard expectation is UTF-8. See e.g. the GNU libunistring documentation for a less intrusive alternative, and a bit of background. With that, you should be able to replace char with uint8_t and the various str* functions with u8_str* replacements and be done. The assumption that one glyph equals one byte will still need to be addressed, but that becomes a minor technicality in your example program. An adaptation is available at http://ideone.com/p0VfXq (though unfortunately the library is not available on http://ideone.com/ so it cannot be demonstrated there).
The following code works as required:
#include <stdio.h>
#include <locale.h>
#include <stdlib.h>
#include <wchar.h>
#define SIZE 10
int main(void)
{
setlocale(LC_ALL, "");
wchar_t buf[SIZE+1];
wchar_t *pat = L"привет мир";
wchar_t str[SIZE+2];
FILE *f1;
FILE *f2;
f1 = fopen("/tmp/вход","r");
f2 = fopen("/tmp/выход","w");
fgetws(buf, SIZE+1, f1);
if (wcsncmp(buf, pat, SIZE) == 0) {
swprintf(str, SIZE+2, L"% 11ls", buf);
fputws(str, f2);
}
fclose(f1);
fclose(f2);
exit(0);
}
Probably your test.c file is not stored in UTF-8 format and for that reason "привет мир" string is ASCII - and the comparison failed. Change text encoding of source file and try again.

Why program in C won't work on Windows?

I have a program written in Linux in C, and it works well on Linux platform, but it doesn't work in Windows. It compiles successfully in Windows, using Code Blocks(mingw32-gcc), but it doesn't work as expected, it simply throws an error and kills the program. How can I make it to work on Windows, Please help. This is the code:
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
#include <assert.h>
#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[])
{
int i, size, k = 2, ftab, sect = 3;
char buf[512];
char vbuf;
int dev, fil_descr, off=0;
fil_descr = open(argv[2], O_RDONLY);
assert(fil_descr > 0);
read(fil_descr, buf, 512);
close(fil_descr);
printf("Bootsector file: %s\n" ,argv[2]);
dev=open(argv[1], O_RDWR);
assert(dev > 0);
write(dev, buf, 512);
ftab = open("filetable", O_CREAT|O_RDWR);
sprintf(buf, "{");
write(ftab, buf, 1);
for(i = 3; i < argc; i++)
{
off = off + (k * 512);
lseek(dev, off, SEEK_SET);
fil_descr=open(argv[i], O_RDONLY);
assert(fil_descr > 0);
size = 0;
while((read(fil_descr, &vbuf, 1))!=0)
{
size++;
write(dev, &vbuf, 1);
}
k = (size > 512)?2:1;
sprintf(buf, "%s-%d," ,argv[i], sect);
write(ftab, buf, strlen(buf));
printf("Input file \'%s\' written at offset %d\n", argv[i], off);
close(fil_descr);
sect = sect + k;
}
sprintf(buf,"}");
write(ftab, buf, 1);
lseek(ftab, 0, SEEK_SET);
read(ftab, buf, 512);
lseek(dev, 512, SEEK_SET);
write(dev, buf, 512);
close (dev);
close(ftab);
}
This is not a C program not working under Windows (although C is the programming language used).
What isn't woking is trying to compile a program using POSIX API functions (such as open, read, write) using a compiler targetting Win32. This includes the use of several headers that don't exist in this environment (which, if they existed, would declare functions that don't exist on the system).
You will either have to use a Unix compatibility layer (Microsoft used to sell that, not sure if they still do, also Cygwin might work), or use proper C, or use native Win32 API functions.
That said, it will also probably not work in the intended way if you switch to using the C standard library I/O funcitons or the native Win32 functions. The output "Bootsector:" suggests that you are trying something that will not work on the system drive at all, and only work with administrative privilegues on another disk.
In order to debug a problem like this, you should add some lines of code helping you to do so.
I changed the respective part of the program to
printf("Bootsector file: %s\n" ,argv[2]);
fil_descr = open(argv[2], O_RDONLY);
if (fil_descr < 0) {
perror("open");
}
printf("fil_descr: %d\n", fil_descr);
assert(fil_descr > 0);
and became aware of the fact that the 2nd command line argument must be a file name of a file to be read.
If I specify it correctly, it works fine.
The 1st file is being written to, and the remaining arguments are file names to be read as well.
Both the headers are linux specific,
#include <sys/types.h>
#include <unistd.h>

C - Running an executable and retrieving output

What I am trying to do is create a program that will, while running, open examplecliprogram.exe with "--exampleparameter --exampleparameter2" as cli input, wait for examplecliprogram.exe to terminate, and then take the output and do something useful with it. I would like examplecliprogram.exe to run in the background (instead of being opened in another window) while the output from examplecliprogram.exe is displayed in the window running the overhead program.
So far I've explored options such as popen(), ShellExecute(), and CreateProcess() but I can't seem to get any of them working properly.
Primarily, I want this program to be able to run independently in a Windows environment, and compatibility with Linux would be a bonus.
edit: I have found one solution by calling system("arguments"). I don't know if this is a good solution that will transfer well to a gui, but at the very least it solves the fundamental problem.
This code runs on Windows and Unix (I tested in Visual Studio, GCC on Cygwin, and GCC on Mac OS X).
I had to use a macro to define popen depending on the platform, because on Windows, the function is _popen, while on other platforms the function name is popen (note the underscore in the former).
#include <stdlib.h>
#include <stdio.h>
/* Change to whichever program you want */
//#define PROGRAM "program.exe --param1 --param2"
#define PROGRAM "dir"
#define CHUNK_LEN 1024
#ifdef _WIN32
#define popen _popen
#define pclose _pclose
#endif
int main(int argc, char **argv) {
/* Ensure that output of command does interfere with stdout */
fflush(stdin);
FILE *cmd_file = (FILE *) popen(PROGRAM, "r");
if (!cmd_file) {
printf("Error calling popen\n");
}
char *buf = (char *) malloc(CHUNK_LEN);
long cmd_output_len = 0;
int bytes_read = 0;
do {
bytes_read = fread(buf + cmd_output_len, sizeof(char), CHUNK_LEN, cmd_file);
cmd_output_len += bytes_read;
buf = (char *) realloc(buf, cmd_output_len + CHUNK_LEN);
} while (bytes_read == CHUNK_LEN);
/* Nul terminate string */
*((char *) buf + cmd_output_len) = '\0';
/* Close file pointer */
pclose(cmd_file);
/* Do stuff with buffer */
printf("%s\n", buf);
/* Free buffer */
free(buf);
return 0;
}
You may want to have a look to this Microsoft example code. It was useful to me.
http://msdn.microsoft.com/en-us/library/ms682499%28VS.85%29.aspx
I used CreateProcess, unfortunately I can't recommend you anything other than 'carefull reading of msdn' and 'starting from simple and progress to complex'.
As for the portability - if you havent need to use some cross-platform toolkit until now, i wouldnt recommend you to start to use one just because of this. I would recommend you to write some 'start process' wrapper and implement it on each platform by its native way.
The cleanest and most portable way of doing this is to use GLib's g_spawn_sync().
You can find the docs online.
gchar * std_out = NULL;
gchar * std_err = NULL;
gint exit_stat = 0;
const char *argv[] = {"--foo", "123", "--bar", "22323", NULL};
if(!g_spawn_sync (NULL, argv, NULL, NULL, NULL, NULL, &std_out, &std_err, &exit_stat, NULL)){
fprintf(stderr, "Failed to spawn!\n");
};
/* std_out and std_err should now point to the respective output.*/

Windows C SHA256 Invalid Generation Issue

Hey there. I'm having a very strange problem with creating sha256 hashes. I made a simple C console program that takes a file path as an argument and uses the standalone sha256 code that can be found here. I compiled the program using MinGW 5.1.6 on Windows 7 x64.
When testing the program on a file, the resultant hash is wrong. I made sure of this by using md5deep on the file, and then by using sha256sum on the file under Linux.
I also verified it was not the code by compiling and running the same code on my Linux box with the same file; the hash it produced was identical to the ones produced by md5deep and sha256sum.
I also adapted Aaron Gifford's sha256 implementation into a different version of my simple program and performed the test again on both Windows and Linux and ended up with the same result.
Could it be possible that the issue is being caused by compiler flags that have not been switched on?
My knowledge of C isn't amazing and my knowledge of compiler options is even worse, so any help would be kindly appreciated.
The code for the simple program is below:
#include <stdio.h>
#include "sha256.h"
#define BUFLEN 16384
int main(int argc, char *argv[]) {
sha256_context ctx256;
sha256_starts(&ctx256);
int kl, l, fd;
unsigned char buf[BUFLEN];
FILE *file = (FILE*) 0;
char *filepath;
fd = fileno(stdin);
filepath = argv[1];
file = fopen(filepath, "r");
fd = fileno(file);
while ((l = read(fd, buf, BUFLEN)) > 0) {
kl += l;
sha256_update(&ctx256, buf, l);
}
fclose(file);
uint8 sha256sum[32];
sha256_finish(&ctx256, sha256sum);
int i;
for (i = 0; i < 32; i++) {
printf("%02x", sha256sum[i]);
}
printf("\n");
return 0;
}
Binary mode gets ignored on Linux, but it applies in Windows. For reference on what it does, see http://msdn.microsoft.com/en-us/library/yeby3zcb%28VS.71%29.aspx. In short, \r\n gets translated to \n in non-binary mode.

C read binary stdin

I'm trying to build an instruction pipeline simulator and I'm having a lot of trouble getting started. What I need to do is read binary from stdin, and then store it in memory somehow while I manipulate the data. I need to read in chunks of exactly 32 bits one after the other.
How do I read in chunks of exactly 32 bits at a time? Secondly, how do I store it for manipulation later?
Here's what I've got so far, but examining the binary chunks I read further, it just doesn't look right, I don't think I'm reading exactly 32 bits like I need.
char buffer[4] = { 0 }; // initialize to 0
unsigned long c = 0;
int bytesize = 4; // read in 32 bits
while (fgets(buffer, bytesize, stdin)) {
memcpy(&c, buffer, bytesize); // copy the data to a more usable structure for bit manipulation later
// more stuff
buffer[0] = 0; buffer[1] = 0; buffer[2] = 0; buffer[3] = 0; // set to zero before next loop
}
fclose(stdin);
How do I read in 32 bits at a time (they are all 1/0, no newlines etc), and what do I store it in, is char[] okay?
EDIT: I'm able to read the binary in but none of the answers produce the bits in the correct order — they are all mangled up, I suspect endianness and problems reading and moving 8 bits around ( 1 char) at a time — this needs to work on Windows and C ... ?
What you need is freopen(). From the manpage:
If filename is a null pointer, the freopen() function shall attempt to change the mode of the stream to that specified by mode, as if the name of the file currently associated with the stream had been used. In this case, the file descriptor associated with the stream need not be closed if the call to freopen() succeeds. It is implementation-defined which changes of mode are permitted (if any), and under what circumstances.
Basically, the best you can really do is this:
freopen(NULL, "rb", stdin);
This will reopen stdin to be the same input stream, but in binary mode. In the normal mode, reading from stdin on Windows will convert \r\n (Windows newline) to the single character ASCII 10. Using the "rb" mode disables this conversion so that you can properly read in binary data.
freopen() returns a filehandle, but it's the previous value (before we put it in binary mode), so don't use it for anything. After that, use fread() as has been mentioned.
As to your concerns, however, you may not be reading in "32 bits" but if you use fread() you will be reading in 4 chars (which is the best you can do in C - char is guaranteed to be at least 8 bits but some historical and embedded platforms have 16 bit chars (some even have 18 or worse)). If you use fgets() you will never read in 4 bytes. You will read in at least 3 (depending on whether any of them are newlines), and the 4th byte will be '\0' because C strings are nul-terminated and fgets() nul-terminates what it reads (like a good function). Obviously, this is not what you want, so you should use fread().
Consider using SET_BINARY_MODE macro and setmode:
#ifdef _WIN32
# include <io.h>
# include <fcntl.h>
# define SET_BINARY_MODE(handle) setmode(handle, O_BINARY)
#else
# define SET_BINARY_MODE(handle) ((void)0)
#endif
More details about SET_BINARY_MODE macro here: "Handling binary files via standard I/O"
More details about setmode here: "_setmode"
I had to piece the answer together from the various comments from the kind people above, so here is a fully-working sample that works - only for Windows, but you can probably translate the windows-specific stuff to your platform.
#include "stdafx.h"
#include "stdio.h"
#include "stdlib.h"
#include "windows.h"
#include <io.h>
#include <fcntl.h>
int main()
{
char rbuf[4096];
char *deffile = "c:\\temp\\outvideo.bin";
size_t r;
char *outfilename = deffile;
FILE *newin;
freopen(NULL, "rb", stdin);
_setmode(_fileno(stdin), _O_BINARY);
FILE *f = fopen(outfilename, "w+b");
if (f == NULL)
{
printf("unable to open %s\n", outfilename);
exit(1);
}
for (;; )
{
r = fread(rbuf, 1, sizeof(rbuf), stdin);
if (r > 0)
{
size_t w;
for (size_t nleft = r; nleft > 0; )
{
w = fwrite(rbuf, 1, nleft, f);
if (w == 0)
{
printf("error: unable to write %d bytes to %s\n", nleft, outfilename);
exit(1);
}
nleft -= w;
fflush(f);
}
}
else
{
Sleep(10); // wait for more input, but not in a tight loop
}
}
return 0;
}
For Windows, this Microsoft _setmode example specifically shows how to change stdin to binary mode:
// crt_setmode.c
// This program uses _setmode to change
// stdin from text mode to binary mode.
#include <stdio.h>
#include <fcntl.h>
#include <io.h>
int main( void )
{
int result;
// Set "stdin" to have binary mode:
result = _setmode( _fileno( stdin ), _O_BINARY );
if( result == -1 )
perror( "Cannot set mode" );
else
printf( "'stdin' successfully changed to binary mode\n" );
}
fgets() is all wrong here. It's aimed at human-readable ASCII text terminated by end-of-line characters, not binary data, and won't get you what you need.
I recently did exactly what you want using the read() call. Unless your program has explicitly closed stdin, for the first argument (the file descriptor), you can use a constant value of 0 for stdin. Or, if you're on a POSIX system (Linux, Mac OS X, or some other modern variant of Unix), you can use STDIN_FILENO.
fread() suits best for reading binary data.
Yes, char array is OK, if you are planning to process them bytewise.
I don't know what OS you are running, but you typically cannot "open stdin in binary". You can try things like
int fd = fdreopen (fileno (stdin), outfname, O_RDONLY | OPEN_O_BINARY);
to try to force it. Then use
uint32_t opcode;
read(fd, &opcode, sizeof (opcode));
But I have no actually tried it myself. :)
I had it right the first time, except, I needed ntohl ... C Endian Conversion : bit by bit

Resources