I want to convert data in other encodings to UTF-8. I'm stranded with following problems:
Executing the attached code gives me: pointer being freed was not allocated in iconv().
Why does iconv play with my memory?
When I don't free(dst) it doesn't crash but nothing is printed. Not even gibberish.
What's wrong?
void utf8(char **dst, char **src, const char *enc)
{
iconv_t cd;
size_t len_src,
len_dst;
len_src = strlen(*src);
len_dst = len_src * 8; // is that enough for ASCII to UTF8?
cd = iconv_open("UTF-8", enc);
*dst = (char *)calloc(len_dst+1, 1);
iconv(cd, src, &len_src, dst, &len_dst);
iconv_close(cd);
}
int main(int argc, char **argv)
{
char *src = "hello world";
char *dst;
utf8(&dst, &src, "ASCII");
printf("%s\n", dst);
free(dst);
return 0;
}
Quote from iconv() description at POSIX.1-2008
size_t iconv(iconv_t cd, char **restrict inbuf,
size_t *restrict inbytesleft, char **restrict outbuf,
size_t *restrict outbytesleft);
The variable pointed to by outbuf shall be updated to point to the byte following the last byte of converted output data.
You need to save and restore *dst (and possibly *src) inside your utf8() function.
Related
I am writing a re-implementation of strlcat as an exercise. I have perform several tests and they produce similar result. However on one particular case, my function gives an segmentation fault error while the original does not, could you explain to me why? I am not allowed to use any of the standard library function, that is why I have re-implemented strlen().
Here is the code I have written :
#include <stdio.h>
#include <string.h>
int ft_strlen(char *s)
{
int i;
i = 0;
while (s[i] != '\0')
i++;
return (i);
}
unsigned int ft_strlcat(char *dest, char *src, unsigned int size)
{
size_t i;
int d_len;
int s_len;
i = 0;
d_len = ft_strlen(dest);
s_len = ft_strlen(src);
if (!src || !*src)
return (d_len);
while ((src[i] && (i < (size - d_len - 1))))
{
dest[i + d_len] = src[i];
i++;
}
dest[i + d_len] = '\0';
return (d_len + s_len);
}
int main(void)
{
char s1[5] = "Hello";
char s2[] = " World!";
printf("ft_strcat :: %s :: %u :: sizeof %lu\n", s1, ft_strlcat(s1, s2, sizeof(s1)), sizeof(s1));
// printf("strlcat :: %s :: %lu :: sizeof %lu\n", s1, strlcat(s1, s2, sizeof(s1)), sizeof(s1));
}
The output using strlcat is : strlcat :: Hello World! :: 12 :: sizeof 5. I am on macOS and I am using clang to compile if that can be of some help.
ft_strlcat() is not so bad, but it expects pointers to strings. main() is troublesome: s1 lacks a null character: so s1 is not a string.
//char s1[5] = "Hello";
char s1[] = "Hello"; // Use a string
s1[] too small for the concatenated string "HelloWorld"
char s1[11 /* or more */] = "Hello"; // Use a string
"%lu" matches unsigned long. size_t from sizeof matches "%zu".
Some ft_strlcat() issues:
unsigned, int vs. size_t
unsigned, int too narrow for long strings. Use size_t to handle all strings.
Test too late
if (!src || ...) is too late as prior ft_strlen(src); invokes UB when src == NULL.
const
ft_strlcat() should use a pointer to const to allow usage with const strings with src.
Advanced: restrict
Use restrict so the compiler can assume dest, src do not overlap and emit more efficient code - assuming they should not overlap.
Corner cases
It does not handle some pesky corner cases like when d_len >= size, but I will leave that detailed analysis for later.
Suggested signature
// unsigned int ft_strlcat(char *dest, char *src, unsigned int size)
size_t ft_strlcat(char * restrict dest, const char * restrict src, size_t size)
Some untested code for your consideration:
Tries to mimic strlcat().
Returns sum of string lengths, but not more that size.
Does not examine more than size characters to prevent reading out of bounds.
Does not append a null character when not enough room.
Does not check for dst, src as NULL. Add if you like.
Does not handle overlapping dest, src. To do so is tricky unless library routines available.
Use unsigned char * pointer to properly handle rare signed non-2's complement char.
size_t my_strlcat(char * restrict dst, const char * restrict src, size_t size) {
const size_t size_org = size;
// Walk dst
unsigned char *d = (unsigned char*) dst;
while (size > 0 && *d) {
d++;
size--;
}
if (size == 0) {
return size_org;
}
// Copy src to dst
const unsigned char *s = (const unsigned char*) src;
while (size > 0 && *s) {
*d++ = *s++;
size--;
}
if (size == 0) {
return size_org;
}
*d = '\0';
return (size_t) (d - (unsigned char*) dst);
}
If the return value is less than size, success!
s1 is not even long enough to accommodate the "Hello"
Use the correct type for sizes.
size_t ft_strlcat(char *dest, const char *src, size_t len)
{
char *savedDest = dest;
if(dest && src && len)
{
while(*dest && len)
{
len--;
dest++;
}
if(len)
{
while((*dest = *src) && len)
{
len--;
dest++;
*src++;
}
}
if(!len) dest[-1] = 0;
}
return dest ? dest - savedDest : 0;
}
Also your printf invokes undefined behaviour as order of function parameters evaluation is not determined. It should be:
int main(void)
{
char s1[5] = "Hello"; //will only work for len <= sizeof(s1) as s1 is not null character terminated
char s2[] = " World!";
size_t result = ft_strlcat(s1, s2, sizeof(s1));
printf("ft_strcat :: %s :: %zu :: sizeof %zu\n", s1, result, sizeof(s1));
}
https://godbolt.org/z/8hhbKjsbx
I'm using the following function on two different computers. One one computer is running Ubuntu and the other OS X. The function works on OS X, but not Ubuntu.
#include <stdio.h>
#define MAXBUF 256
char *safe_strncat(char *dest, const char *src, size_t n) {
snprintf(dest, n, "%s%s", dest, src);
return dest;
}
int main(int argc, const char * argv[]){
char st1[MAXBUF+1] = "abc";
char st2[MAXBUF+1] = "def";
char* st3;
printf("%s + %s = ",st1, st2);
st3 = safe_strncat(st1, st2, MAXBUF);
printf("%s\n",st3);
printf("original string = %s\n",st1);
}
Compile and run on Ubuntu
gcc concat_test.c -o concat_test
./concat_test
abc + def = def
original string = def
Compile and run in Xcode in OS X
abc + def = abcdef
original string = abcdef
Why does this work on mac and not on Ubuntu?
Should it work on Ubuntu?
Should it work on Mac?
I could swear it used to work in Ubuntu until recently, but I don't know what would have changed to make it stop working?
Could compiler settings have anything to do with this working or not?
Your code invokes undefined behavior because you pass the destination buffer as one of the source strings of your snprintf() format. This is not supported:
7.21.6.5 The snprintf function
Synopsis
#include <stdio.h>
int snprintf(char * restrict s, size_t n,
const char * restrict format, ...);
Description
The snprintf function is equivalent to fprintf, except that the output is written into an array (specified by argument s) rather than to a stream. If n is zero, nothing is written, and s may be a null pointer. Otherwise, output characters beyond the n-1st are discarded rather than being written to the array, and a null character is written at the end of the characters actually written into the array. If copying takes place between objects that overlap, the behavior is undefined.
(emphasis mine).
The implementation of snprintf differs between Ubuntu (glibc) and OS/X (Apple libc, based on BSD sources). The behavior differs and cannot be relied upon as it is undefined in all cases.
You can implement safe_strcat() this way:
#include <string.h>
char *safe_strcat(char *dest, size_t size, const char *src) {
char *p = memchr(dest, '\0', size);
if (p != NULL) {
strncat(p, src, size - (p - dest) - 1);
}
return dest;
}
Notes:
do not call this function safe_strncat(), it is really a safe version of strcat().
pass the size of the destination array after the destination pointer itself, not after the source pointer.
returning the destination pointer does not allow the caller to detect truncation. You could instead return the length of the result if the destination array had been large enough, like snprintf(), you it would still not tell the caller if the destination was not null terminated before the call (for safe_strcat and safe_strncat).
You can use the same model for safe versions of strcpy, strncat and strncpy (but not implementing strncpy()'s counter-intuitive semantics):
char *safe_strcpy(char *dest, size_t size, const char *src) {
if (size > 0) {
*dest = '\0';
strncat(dest, src, size - 1);
}
return dest;
}
char *safe_strncat(char *dest, size_t size, const char *src, size_t n) {
char *p = memchr(dest, '\0', size);
if (p != NULL) {
if (n > size - (p - dest) - 1)
n = size - (p - dest) - 1;
strncat(p, src, n);
}
return dest;
}
char *safe_strncpy(char *dest, size_t size, const char *src, size_t n) {
if (size > 0) {
if (n > size - 1)
n = size - 1;
*dest = '\0';
strncat(dest, src, n);
}
return dest;
}
snprintf(dest, n, "%s%s", dest, src);
This line caused undefined behaviour because you are overwriting the buffer dest. Because it's not defined, it's pointless to work out why it works on one machine and not the other.
More details can be found here: Is sprintf(buffer, "%s […]", buffer, […]) safe?
So, here is the function:
int strinsert(char *dst, int len, const char *src, int offset)
I need to insert a copy of my src string into the string called dst from the position offset.
The argument len specifies the number of characters reserved for the array dst.
the important part of the code:
int strinsert(char *dst, int len, const char *src, int offset)
{
strncpy(dst, src+offset, len);
char buf[100];
strcpy(buf+len, src);
len += strlen(src) ;
strcpy(buf+len, dst+offset);
strcpy(dst, buf);
return 1;
}
Still feels kind of off...
Edit: Before someone misunderstood, I am just teaching myself how to program in C and I found this exercise. Btw, I didn't really found some good learning material for one- and two-dimensional arrays, could someone be so kind and post some?
Its a bit painful, but you really have to contruct a new string as you can't really shuffle bits of memory around so easily and I don't think there is a library function to do this (is there??). Somthing like this:
int strinsert(char *dst, int len, const char *src, int offset)
{
char *new_string = new char[len];
int remaining = len;
// Check offset is not to long (+1 for null)
if (offset >= remaining)
offset = remaining;
// copy the pre-string from dest
strncpy(new_string, dest, offset);
// Calulate the remaining space
remaining -= offset;
// Add the insert string (with max chars remaining)
strncat(new_string, src, remaining);
// calc remaining space
remaining -= strlen(src);
// Add the post-string from dest (with max chars remaining)
strncat(new_string, dest, remaining);
// Finally copy the new_string into dest
strncpy(dest, new_string, len);
// free the memory
delete [] new_string;
}
Note: You probably need to do a better job of calculating the remaining space incase it goes negative...
Edit: replaced variable length array (illegal ... oops) with mem allocation
Does this solve the problem?
int strinsert(char *dst, int len, const char *src, int offset)
{
char temp [MAX_SIZE];
strncpy(temp, dest+offset, strlen(dest+offset));
strncpy(dest+offset, src, strlen(src));
strcat (dest+offset+strlen(src), temp);
return 0;
}
Obviously, the above code doesn't have any error checking, and the temp can be malloced etc.
You can use strncpy(), which is a function of string.h library. It copies first num characters of source string to destination string.
char * strncpy ( char * destination, const char * source, size_t num );
But in your case, you need a third string because you can not expand your destination array. You can copy substring (from beginning to offset) of destination array into this third array and concatenate the rest. So you need to do something like this:
char *dst = "ThisIsMyHomework";
char *src = "Not";
char finalString[50];
int offset = 6; //Assume we want to insert "Not" string between "ThisIs" and "MyHomework". So we want src array to start from 6th index of dst array.
strncpy(finalString, dst, offset); //finalString="ThisIs"
finalString[offset] = '\0'; //For finalString, you have to add "\0" manually because strcat will append other strings from 50. index of array if you don't
strcat(finalString, src); //finalString="ThisIsNot"
strcat(finalString, dst + offset); //finalString="ThisIsNotMyHomework"
void strinsert(char *dst, size_t len, const char *src, size_t offset)
{
size_t iLenDst = strlen(dst),
iLenSrc = strlen(src);
// Some error handling
if (iLenDst+iLenSrc+1>len)
{
ASSERT(FALSE):
return;
}
// restrict to max length
if (offset>iLenDst)
offset = iLenDst;
// Make room incl. trailing \0
memmove(dst+offset+iLenSrc,dst+offset,iLenDst-offset+1);
// Insert new
memcopy(dst+offset,src,iLenSrc);
}
Folks,
Trying to troubleshoot an issue with the base64 function below. About 2-3% of the requests that pass through this process return an incorrect (too short) base64output.
static const char *header_request_gce(request_rec *r, char *a)
{
char *tim = apr_palloc(r->pool, APR_RFC822_DATE_LEN);
apr_rfc822_date(tim, r->request_time);
char *uri = apr_psprintf(r->pool, "%s", r->uri);
char encode[32768];
//encode = malloc(strlen(tim)+strlen(uri)); /* make space for the new string (should check the return value ...) */
strcpy(encode, "GET\n\n\n");
strcat(encode, tim);
strcat(encode, "\n");
strcat(encode, uri);
unsigned int encode_length = strlen(encode);
unsigned char* result;
unsigned char* key = (unsigned char*) "2kcXHh+K+XLtI61/KIV3d1tVzOooTdeOqFii9osz";
static char res_hexstring[8192];
result = HMAC(EVP_sha1(), key, 40, encode, encode_length, NULL, NULL);
char *base64(const unsigned char *input, int length);
char *base64output = base64(result, strlen(result));
return base64output;
}
char *base64(const unsigned char *input, int length)
{
BIO *bmem, *b64;
BUF_MEM *bptr;
b64 = BIO_new(BIO_f_base64());
bmem = BIO_new(BIO_s_mem());
b64 = BIO_push(b64, bmem);
BIO_write(b64, input, length);
BIO_flush(b64);
BIO_get_mem_ptr(b64, &bptr);
char *buff = (char *)malloc(bptr->length);
memcpy(buff, bptr->data, bptr->length-1);
buff[bptr->length-1] = 0;
BIO_free_all(b64);
return buff;
}
The key above has been modified ofcourse, but kept in the correct character format
This line is incorrect:
char *base64output = base64(result, strlen(result));
The data (output from sha1) that you are encoding can contain the NUL byte which means strlen returns a number that is too small (with a probability of 1 - (255/256)^20 which is approximately 7.5%). Rather than call strlen you should just pass in the size as a constant. I believe that if you are just encoding a sha1 hash, the length will always be 20:
char *base64output = base64(result, 20);
There is probably a better way to get that length from an HMAC function or something (so that it updates automatically if you change the hashing algorithm), but I am, admittedly, not very familiar with the hashing functions you're using.
I'm trying to write strcpy on my own using pointers and I get an error during runtime.
void str_cpy(char **destination, const char *source) {
// char *s1 = *destination;
while (*source != '\0') {
**destination++ = *source++; //Get an error here
}
**destination = '\0';
}
I call the function as follows:
char *str = NULL;
str_cpy(&str, "String");
Is it not OK?
Thanks!
No, it's not okay. Why? Because str is a NULL pointer. It's pointing to nothing. When you try to write values into it, where will they go? It's not pointing to any allocated memory!
You first have to allocate memory for str. You can do:
char *str = malloc(strlen("String") + 1); // + 1 for the '\0' character at the end of C-style strings
Or you can do:
char str[256]; // make str large enough to hold 256 chars. Note that this is not as safe as the above version!
Also, destination should be a single pointer, not a double pointer. Well, it's not technically wrong to use a double pointer, it's just unnecessary.
Optionally, you can allocate the memory in the str_cpy function, like so:
void str_cpy(char **destination, const char *source) {
*destination = malloc(strlen(source) + 1);
// ... continue as normal
For simplicity's sake, this can be done in one line in a function.
void mystrcpy(char *dest, const char *src) {
while (*dest++ = *src++);
}
This being said, you do need to allocate memory for dest beforehand using malloc or just simply by having a character array like char dest[256].
I don't see any need to pass a pointer-to-pointer:
void str_cpy(char *dst, const char *src) {
while (*src != '\0') {
*dst++ = *src++;
}
*dst = '\0';
}
And you need to allocate memory for dst before passing:
const char *src = "String";
char *str = malloc(strlen(src)+1); //plus one for null byte
str_cpy(dst, src);
You should likely allocate some memory for that pointer before passing it off to a function that fills what it points to (which in this case, is NULL).
Example:
char *str = malloc(128);
if (str)
{
str_cpy(&str, "String");
free(str);
str = NULL;
}
I advise not doing this without also providing target-buffer size information (i.e. if you're writing your own, then boundary-check the target buffer, otherwise your version has the same security flaws as strcpy() which are bad enough as it is).
Note: Unless you're planning on changing the address held by the pointer passed as the target, you need not use a double pointer either. The double pointer usage you have prevents the traditional strcpy() usage pattern of:
char str[128];
str_cpy(&str, "Hello"); // error.
An array address cannot be passed as a pointer-to-pointer, so your code cannot fill a static array without an intermediate pointer:
char str[128];
char *p = str;
str_cpy(&p, "Hello"); //ok. passing address of pointer.
If this is not intentional (and I don't see why it could be unless you have ideas of internally emulating strdup() on a NULL pointer passage) You should address this.
Here is a complete implementation.
Good article from here. Describes timing and performance. I did not measure myself though.
http://www.howstuffworks.com/c35.htm
char* mystrcpy(char *dst, const char *src) {
char *ptr = dst;
while ((*dst++ = *src++) ) ;
return ptr;
}
int main(int argc, char *argv[]) {
const char *src = "This is C.\0";
char *dst = malloc(sizeof(char)*(strlen(src)+1)); //+1 for the null character
dst = mystrcpy(dst, src);
printf("%s",dst);
return 1;
}
Recently I faced same problem of above one using double pointer strcpy implementation
It might helpful to others below code
void strcpy_i( char **dst, const char *src )
{
*dst=(char *)malloc((strlen(src)+1)*sizeof(char));
char *tmp=*dst;
if(tmp == NULL || src == NULL)
return ;
while((*tmp++=*src++)!='\0');
}
int main()
{
char v[]="Vinay Hunachyal";
char *d=NULL;
strcpy_i(&d,v);
printf("%s",d);
return 0;
}
#include<stdio.h>
void main()
{
void mystrcpy(char *,char *);
char s1[100],s2[100];
char *p1;
char *p2;
p1=s1;
p2=s2;
printf("Enter the string to copy to s2...?\n");
scanf("%s",p1);
mystrcpy(p2,p1);
printf("S2 after copying = %s",p2);
}
void mystrcpy(char *p2,char *p1)
{
while(*p1!='\0')
{
*p2=*p1;
p2++;
p1++;
}
*p2='\0';
}
Its my solution..Simple to understand..