New to the community, but not new to programming.
I've been trying to get a collection of hash functions up/running and I succeeded. However I found some weird results and haven't been able to put my finger on it yet. RFC4634 contains a C implementation for SHA-1 and SHA-2 family, which also can accept a file being passed on for hashing. RFC3174 contains a C implementation, but doesn't process file streams. I've been using the C implementation from RFC4634 to verify files, yet the verification process is returning non similar results when I compare them against SHA-1 provided hashes.
Any idea what the reasons could be?
Did you check if you opened the files in ASCII or binary mode? Line end translation may be performed before the hash is being calculated.
Update:
I just compiled the RFC4634 shatestand tried on a sample text file. As long as there isn't a line break, all tools agree. Once you insert a line break, results depend: if the text file uses CR and LF (DOS mode), then shatest produces a different result. If the line end is only LF (UNIX), it still agrees with the other tools.
Update 2:
In the file shatest.c of RFC4634, in function hashfile(...), set fopen to binary mode:
FILE *hashfp = (strcmp(hashfilename, "-") == 0) ? stdin :
fopen(hashfilename, "rb");
/* ^ HERE */
Related
I have an application, written in C, which generates various data parameters that I am logging into a text file named debug_log.txt. Whenever this log file reaches 1 MB, I am renaming the filename with timestamp ex debug_log_20200106_133000.txt & moving it in same directory. I am then reopening debug_log.txt to log new parameters.
if(stat("/home/log/debug_log.txt", &statFiledbg) == 0)
{
if(statFiledbg.st_size >= 1048576) // 1MB
{
current_time = time(0);
strftime(time_buffer, sizeof(time_buffer), "%Y%m%d_%H-%M-%S", gmtime(¤t_time));
sprintf(strSysCmddbg, "mv /home/log/debug_log.txt /home/log/debug_log%s.txt", time_buffer);
system(strSysCmddbg);
fp_dbglog = freopen("/home/log/debug_log.txt", "w", fp_dbglog);
}
}
The code works most of the time until it doesn't. After running the application for couple days, I see that debug_log.txt grows beyond 1 MB while the last moved & renamed log file is empty.
What could be the reason?
Use the rename function from the C standard library (in stdio.h) and check errno if it failed to know the exact reason why it is failing.
When working with files, and I/O in general, there are many, many things that can go wrong.
One of my senior developer in the company told me so. Is there anything wrong with using system()?
Yes: it is unnecessary (C and POSIX provide you with a function for basic usages like this), nonportable (it assumes you are in a system that has a "mv"), slower (it needs to spawn another process) and wrong for many use cases (eg. here there is no way to know what exactly failed unless you save the textual output of mv).
See questions and answers like Moving a file on Linux in C for an in-depth explanation.
We are migrating a bunch of COBOL programs from z/OS to AIX 7. We are using the IBM COBOL Compiler (5.1) on AIX. Now I don't understand how the file access and the file system work for COBOL on AIX.
The COBOL code is straight forward with
SELECT :FILE: ASSIGN TO :FILE:
ORGANIZATION IS SEQUENTIAL
ACCESS MODE IS SEQUENTIAL
STATUS IS S-STATUS.
and then doing an OPEN INPUT
This compiled fine on AIX:
PP 5724-Z87 IBM COBOL for AIX 5.1.0 in progress ...
LineID Message code Message text
91 IGYGR1216-I A "RECORDING MODE" of "F" was assumed for file
"CUSTOMERS".
94 IGYGR1216-I A "RECORDING MODE" of "F" was assumed for file
"LIST1".
97 IGYGR1216-I A "RECORDING MODE" of "F" was assumed for file "UINPUT".
Messages Total Informational Warning Error Severe Terminating
Printed: 3 3
End of compilation 1, program xxxxx, highest severity: Informational.
Now the problem is, that when running the program the file is not found. It gives Status code: 37
I know that I have to provide a file system for the file on the shell (ksh), as such:
export CUSTOMERS="STL-CUSTOMERS". The file is in the same directory as the program.
My question is this: Which file system to use? I tried "STL" which seemed to me like the "standard" AIX file system (which is JFS2). But that doesn't work. The other option are (from the COBOL on AIX Programming Guide 5.1):
DB2
SdU (SMARTdata Utilities)
SFS (Encina Structured File Server)
STL (standard language)
QSAM (queued sequential access method)
RSD (record sequential delimited)
We tried all and the file system that worked was "QSAM". The file is a text file (ASCII), that was transfered from the mainframe. But not directly, it was first copied by FTP and then converted to ASCII on Windows (we had to fix the line breaks). When playing around with it to make it work, we edited the file with vi to make the lines 80 characters each. So it was edited on AIX and looks like a ordinary text file on AIX.
Why would COBOL still want QSAM as "file system"? What does the term "file system" mean here any way? It seems that it is not in the sense of a real file system such as JFS.
I can see where this would be confusing... Especially coming from a Non-POSIX environment.
I'll start with the answer, and then provide more information.
For a normal text file, you want RSD, and to make sure that the \n is column 81.
A record length of 80 is just the data portion, not including the delimiter.
QSAM (fixed length) would appear to work, but would return the \n as part of the data!
Your FS=37 means that the attributes of the file don't match what the program is asking for (more esoteric than, say, FS=39 - invalid fixed attributes). In this case, it means the file you wanted to open was not, really, a STL file.
By filesystem we mean how the data is physically stored on the platter, SSD, RAM, ... More accurately, how we format the record before handing it off to the next lower level of I/O.
There are two basic types of filesystems:
Native (on top of JFS2, JFS, NFS, CIFS, ...) RSD, QSAM, LSQ, STL, SdU. These filesystems can be operated on by standard OS utilities
Non-Native (on top of another product) DB2 (DB2) and SFS (TxSeries/CICS). These filesystems are invisible to standard OS utilities
Then, there are groupings by type of COBOL organization (preferred):
Sequential: All filesystems support Sequential...z/OS: QSAM, VSAM
Relative: STL, SdU, SFS, DB2...........z/OS: VSAM
Indexed: STL, SdU, SFS, DB2...........z/OS: VSAM
Of the Native filesystems, QSAM(variable), STL and SDU contain metadata, making them not really viewable in vi,cat,... Ok... you can open them in vi, but it will look like utter garbage.
QSAM is a faithful implementation of z/OS:
Fixed length records: Raw data, no BDW/RDW, no line delimiters (\n).
Variable length records: RDW + raw data (no \n)... However there is no BDW.
RSD is a normal stream (text) file; each record is terminated with \n, which is not counted in the record length (the program will never see them).
LSQ (Line Sequential) is the same as on z/OS - messy semantics.
VSAM is an alias for a filesystem that provides all the features of z/OS VSAM.
Unfortunately, for historical reasons, it points to SdU...
STL is, by far and away, better at everything than is SdU.
SdU was the first filesystem to cover QSAM and VSAM, but is old and decrepit, compared to STL.
I am new to Linux OS and i am trying to compile a simpe c program
, I wrote it in using text editor
#include<stdio.h>
void main(){
printf("Hello!");
}
I typed gcc -o main main.c
and the following issue shows up
main.c:1:1: error: stray '\377' in program
# i n c l u d e < s t d i o . h >
main.c:1:2: error: stray '\376' in program
This happens whenever i run c or c++ program
\377 and \376 are an octal representation of the bytes that constitute the value 0xFEFF, the UTF-16 byte order marker. Your compiler doesn't expect those characters in your source code.
You need to change the encoding of your source file to either be UTF-8 or ASCII. Given the number of text editors that exist and the lack of that information in your question I cannot list every possibility for how to do that.
You could just do this in a bash shell:
cat > program.c
// File content here
^D
This will create a file called "program.c" with "// File content here" as its content, in UTF-8.
Your text editor is saving the program in the wrong character encoding. Save it as ASCII plain text and try again.
There is no text but encoded text.
With your editor, you have chosen to save your text file with the UTF-16LE character encoding (presumably).
Any program that reads a text file must know the character encoding of the text file. It could accept one documented character encoding (only or default) and/or allow you to tell it which you used.
This could work
gcc -finput-charset=UTF16-LE main.c
but since you have include files, the include files must use the same character encoding. On my system, they use UTF-8 (and include ©, which is good because gcc chokes on the bytes for that, letting me know that I've messed up).
Note: It's not very common to save a C source file (or most any text file) with UTF-16. UTF-8 is very common for all types of text files. (ASCII is also not very common, either. You might not find it as an option in many text editors. Historically, MS-DOS did not support it and Windows only got it very late and only for the sake of completeness.)
I'm trying to check the given file is binary or not.
I refer the link given below to find the solution,
How can I check if file is text (ASCII) or binary in C
But the given solutions is not working properly, If I pass the .c file as argument, Its not working, It gives wrong output.
The possible files I may pass as argument:
a.out
filename.c
filename.txt
filename.pl
filename.php
So I need to know whether there is any function or way to solve the problem?
Thanks...
Note : [ Incase of any query, Please ask me before down vote ]
You need to clearly define what a binary file is for you, and then base your checking on that.
If you just want to filter based on file extensions, then create a list of the ones you consider binary files, and a list of the ones you don't consider binary files and then check based on that.
If you have a list of known formats rather then file extensions, attempt to parse the file in the formats, and if it doesn't parse / the parse result makes no sense, it's a binary file (for your purposes).
Depending on your OS, binary files begin with a header specifying that they are an executable and contain several informations about them (architecture, size, byte order etc.). So by trying to parse this header, you should know if a file is binary or not. If you are on mac, check the Mach-O file format, if you are on Linux, it should be ELF format. But beware, it's a lot of documentation.
I just created a simple PDF Document containing a word "Test" in it and created a byte stream out of it in C# Console Application:
buff = File.ReadAllBytes(<Path of File>);
The size of the file is around 9,651 bytes. I also created a Win32 C dll that exports a function which takes the file byte array and the length of the byte array as an argument, declared in C# using this:
[DllImport("<path to dll>", CallingConvention = CallingConvention.Cdecl)]
public static extern int file_data(byte[] byteArray, int length);
The method in C dll is exported as below:
#define FILEDATA_API __declspec(dllexport)
FILEDATA_API int file_data(char *byteArray, int size);
I then invoked ret = file_data(buff, buff.length); and in the C code, wrote the character pointer received directly to a temp file character by character as below:
while (length> 0)
{
fprintf(outFile, "%c", *fileData); //fileData is the actual byte array received from C# Code
fileData++;
length--;
}
But the problem arises here, the C code that dumps out the byte array to a file character by character generates a file of size 9,755 bytes. Most of the content inside it seems to look correct, except some new lines that gets introduced(as far as i know and may be some additional data), which causes the PDF file to get corrupted and this dumped out version does not open in Adobe. Can someone please provide some pointers on where I might be going wrong? I cannot use %s in fprint since some combination of the byte array in the PDF results in null terminated string in C which then dumps out even lesser data than I expect.
Thanks.
UPDATE:
The desired behavior is that the file byte array as received from C#
and when written using C code to a file should make the file open
successfully in Adobe.
The code present in the problem should be
sufficient for someone to generate a win32 dll, that simply writes
out the char pointer to a file, but I have added few more details.
You're probably calling fopen without the b mode flag. Append b to your mode specifier:
FILE *outFile = fopen("file.txt", "wb")
From this site (emphasis mine):
Text files are files containing sequences of lines of text. Depending on the environment where the application runs, some special character conversion may occur in input/output operations in text mode to adapt them to a system-specific text file format. Although on some environments no conversions occur and both text files and binary files are treated the same way, using the appropriate mode improves portability.
In my experience, this "conversion" on Windows changes \n to \r\n at least.