InternetReadFile's output to string gives a "ÌÌÌÌÌÌ" sequence in C - c
I am trying to read the text content of a HTTP Post Response body using InternetReadFile. However, all it contains is a string of "Ì" (-52 when converted to int).
Could this be encoding related? Is it that what is being returned is not a string at all?
Am I missing a step required to read the output?
Please note that I know for a fact that this message body contains plain text (based on logs).
Here is the code:
Ptr = (char *)OutBuffer;
while(TRUE)
{
// read the server response
//
if(!InternetReadFile(RequestHandle,Ptr,Length ,&BytesRead))
{
Rc = GetLastError();
InternetCloseHandle(RequestHandle );
SetLastError(Rc);
return(ACE_HTTP_ERROR);
}
if(BytesRead == 0) // end of data
break;
TotalLength += BytesRead;
Ptr += BytesRead;
if(TotalLength >= *OutBufferLength)
{
InternetCloseHandle(RequestHandle );
SetLastError(ERROR_INSUFFICIENT_BUFFER);
*OutBufferLength = TotalLength;
return(ACE_HTTP_NO_ENOUGH_SPACE);
}
}
*OutBufferLength = TotalLength;
At this point, Ptr, when read as a char array, contains nothing but a sequence of 'Ì'.
Related
Failed to send Http request based on MFC
I'm new to HTTP and MFC. I suppose to upload a binary file to a given URL, by sending PUT request. I'm sure the requesting URL and the header are well formed and the file works well, because I tried them by Postman then get a SUCCESS returned.
But when I try to do this by the following MFC code, it always returns a 403 error. Please help me.
bool UploadFile(LPCTSTR strUrl, LPCTSTR filenameFullpath, CString header)
{
BOOL bResult = FALSE;
DWORD dwType = 0;
CString strServer;
CString strObject;
INTERNET_PORT wPort = 0;
DWORD dwFileLength = 0;
BYTE * pFileBuff = NULL;
CHttpConnection * pHC = NULL;
CHttpFile * pHF = NULL;
CInternetSession cis;
bResult = AfxParseURL(strUrl, dwType, strServer, strObject, wPort);
if (!bResult)
return FALSE;
CFile file;
try
{
// Read the file
if (!file.Open(filenameFullpath, CFile::shareDenyNone | CFile::modeRead))
return FALSE;
dwFileLength = file.GetLength();
if (dwFileLength <= 0)
return FALSE;
pFileBuff = new BYTE[dwFileLength];
memset(pFileBuff, 0, sizeof(BYTE) * dwFileLength);
file.Read(pFileBuff, dwFileLength);
// Set up internet connection
const int nTimeOut = 5000;
cis.SetOption(INTERNET_OPTION_CONNECT_TIMEOUT, nTimeOut);
cis.SetOption(INTERNET_OPTION_CONNECT_RETRIES, 2); // Retry once if failed
pHC = cis.GetHttpConnection(strServer, dwType == AFX_INET_SERVICE_HTTP ? NORMAL_CONNECT : SECURE_CONNECT, wPort); // Get a HTTP connection
pHF = pHC->OpenRequest(CHttpConnection::HTTP_VERB_PUT, strObject);//strObject
pHF->AddRequestHeaders(header , HTTP_ADDREQ_FLAG_ADD_IF_NEW);
// I set the header separately instead of passing it as the first parameter of the next call.
// If I don't do so, another error occurs
if (!pHF->SendRequest(NULL, 0, pFileBuff, dwFileLength))
{
delete[]pFileBuff;
pFileBuff = NULL;
pHF->Close();
pHC->Close();
cis.Close();
return FALSE;
}
DWORD dwStateCode = 0;
pHF->QueryInfoStatusCode(dwStateCode);
if (dwStateCode == HTTP_STATUS_OK)
bResult = TRUE;
}
catch (CInternetException * pEx)
{
char sz[256] = "";
pEx->GetErrorMessage(sz, 25);
CString str;
str.Format("InternetException occur!\r\n%s", sz);
AfxMessageBox(str);
}
delete[]pFileBuff;
pFileBuff = NULL;
file.Close();
pHF->Close();
pHC->Close();
cis.Close();
return bResult;
}
And the calling of this function is just something like this:
CHttpClient hc;
hc.UploadFile(csUrl, "E:\\blah blah blah\\blah.tma", csUploadHeader);
while csUrl and csUploadHeader are well formed CString;
I got it! I should give pHF the header items (the key-value pairs) one by one, by calling the AddRequestHeaders() method several times, instead of wrapping and passing them together to pHF.
Then the following codes works very well:
CString sHeader1, sHeader2, sHeader3;
sHeader1.Format(_T("%s : %s"), sKey1, sValue1); // sKey_, sValue_ are strings
sHeader2.Format(_T("%s : %s"), sKey2, sValue2);
sHeader3.Format(_T("%s : %s"), sKey3, sValue3);
pHF->AddRequestHeaders(sHeader1, HTTP_ADDREQ_FLAG_ADD_IF_NEW); // Keep the second paeramter as these
pHF->AddRequestHeaders(sHeader2, HTTP_ADDREQ_FLAG_COALESCE);
pHF->AddRequestHeaders(sHeader3, HTTP_ADDREQ_FLAG_COALESCE);
And a very important notification: the string we pass to AddRequestHeaders() should not have the quotation("") around either the key or the value. In other words, string like "MyKey":"MyValue" have to be modified as MyKey:MyValue before giving to AddRequestHeaders();
Multiline 'values' in json using the Jansson API
Something in here is not allowing '\n' to be interpreted as a newline control character. Can someone help?
The output, when printed, seems to print
"I want this statement\nto be in newlines,\nbecause I want it\n that way.",
AND NOT
"I want this statement
to be in newlines,
because I want it
that way."
static void
append_object_to_array(json_t* my_json_array)
{
const char* KEY = "KEY";
const char* VALUE = "I want this statement\nto be in newlines,\nbecause I want it\n that way.";
json_t* my_json_obj = json_object();
json_object_set_new(my_json_obj , KEY, json_string(VALUE));
json_array_append_new(my_json_array, my_json_obj);
}
int main()
{
json_t* json_array = json_array();
append_object_to_array(json_array);
json_decref(my_json_array);
char* output = NULL;
output = json_dumps(my_json_array, JSON_ENCODE_ANY);
/* My other failed attempts at getting this to work
* output = json_dumps(my_json_array, JSON_ESCAPE_SLASH);
* output = json_dumps(my_json_array, JSON_INDENT(n));
*/
char* expected_output = "[{\"KEY\": \"I want this statement\nto be in newlines,\nbecause I want it\n that way.\"}]";
if (strcmp(expected_output, output) != 0) {
printf("Expected: %s", expected_output);
printf("\n");
printf("Actual: %s", output);
} else {
printf("Success");
}
return 0;
}
P.S. I have checked the following possible duplicates (I may have missed something), but they don't seem to fit what I am looking for.
Are multi-line strings allowed in JSON?
Can a JSON value contain a multiline string
JNI function returning illegal UTF characters at android
Im trying to return string from JNI to android but its returning illegal UTF characters like this:
JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8:
illegal start byte 0x80
04-12 16:08:09.899 18210-18372 A/art:art/runtime/runtime.cc:427]
string: '���� ���!��"��,"���"���#���$��%���
%��`&��'��H(���)��D*���*��X+�� ,���,���-��4.��|.��P/��t/���/��01��x1��
2��D2���2���3���4���5��06���6��9���9��;���;��H<�� =��0=���=���>��8?��
Here is the code which I am using:
JNIEXPORT jbyteArray Java_pakdata_com_qurantextc_MainActivity_get(
JNIEnv *pEnv,
jobject this,
jint pageNo, jint lang) {
char* buffer=(char*)malloc(10000); // this buffer contains the ayat
register unsigned int pageNumber = pageNo - 1;
char * header=(char*)malloc(1000);
sprintf(header,"[{\"OFFSET\":%d,\"DATA\":\"",pageNumber+1);
strcpy(buffer,header);
// to get the last ayat of the page
// this loop will fetch all ayats of the page
for (int i = start_ayat; i <= end_ayat; i++) {
sprintf(buffer+strlen(buffer),"<div class=\\\"qr0\\\" data-ayat=\\\"%d\\\" id=\"%d\\\"><span>",i+1,i+1);
get(lang, i, buffer + strlen(buffer)); // len is equal to length of buffer ( strlen() )
strcpy(buffer+strlen(buffer),"<\\/span><\\/div>");
}
// char* footer;
sprintf(buffer+strlen(buffer),"<div class=\\\"pagebreak\">%d<a id=\\\"%d\\\"next\\\"href=\\\"\\/page\\/%d\\\"></a><\\/div <\\/div>\"}]",pageNumber+1,pageNumber+1,pageNumber+1);
__android_log_print(ANDROID_LOG_DEBUG, "LOG_TAG","string: '%s'" , buffer);
int l = strlen(buffer);
char c[l];
strcpy(c,replace(buffer,"\r","<br>"));
jbyteArray ret = (*pEnv)->NewByteArray(pEnv,l);
(*pEnv)->SetByteArrayRegion (pEnv,ret, 0, l, c);
const char * errorKind = NULL;
uint8_t utf8 = checkUtfytes(c, &errorKind);
if (errorKind != NULL) {
free(buffer);
return ret;
} else {
free(buffer);
return ret;
}
I have tried using this too:
return = (*pEnv)->NewStringUTF(pEnv,buffer)
but it still contain illegal UTF characters..
Here is my android side code
byte[] ss = get(a, pos);
s= new String(ss,"UTF-8");
Still getting illegal UTF character error.
I have tried encoding on java side but its no help either,
I am posting here because all other methods that are written here i have already tried but it didn't worked.
PLEASE HELP!!!
May be I am late but your code seems to be correct but according to JNI documentation they doesn't support these characters. You have to handle it from server side. Hope it helps.
Reading and parsing text file exception-C#
I am parsing big text files and it's working fine for some time but after few minutes it give me exception (An unhandled exception of type 'System.UnauthorizedAccessException' occurred in System.Core.dll
Additional information: Access to the path is denied.)
I get exception on below mention line.
accessor = MemoryMapped.CreateViewAccessor(offset, length, MemoryMappedFileAccess.Read);
Below is my function
public static void CityStateZipAndZip4(string FilePath,long offset,long length,string spName)
{
try
{
long indexBreak = offset;
string fileName = Path.GetFileName(FilePath);
if (fileName.Contains(".txt"))
fileName = fileName.Replace(".txt", "");
System.IO.FileStream file = new System.IO.FileStream(#FilePath, FileMode.Open,FileAccess.Read, FileShare.Read );
Int64 b = file.Length;
MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateFromFile(file, fileName, b, MemoryMappedFileAccess.Read, null, HandleInheritability.Inheritable, false);
using (MemoryMapped)
{
//long offset = 182; // 256 megabytes
//long length = 364; // 512 megabytes
MemoryMappedViewAccessor accessor = MemoryMapped.CreateViewAccessor(offset, length, MemoryMappedFileAccess.Read);
byte byteValue;
int index = 0;
int count = 0;
StringBuilder message = new StringBuilder();
do
{
if (indexBreak == index)
{
count = count + 1;
accessor.Dispose();
string NewRecord = message.ToString();
offset = offset + indexBreak;
length = length + indexBreak;
if (NewRecord.IndexOf("'") != -1)
{ NewRecord = NewRecord.Replace("'", "''"); }
// string Sql = "insert into " + DBTableName + " (ID, DataString) values( " + count + ",'" + NewRecord + "')";
string Code = "";
if (spName == AppConfig.sp_CityStateZip)
{
Code = NewRecord.Trim().Substring(0, 1);
}
InsertUpdateAndDeleteDB(spName, NewRecord.Trim (), Code);
accessor = MemoryMapped.CreateViewAccessor(offset, length, MemoryMappedFileAccess.Read);
message = new StringBuilder();
index = 0;
//break;
}
byteValue = accessor.ReadByte(index);
if (byteValue != 0)
{
char asciiChar = (char)byteValue;
message.Append(asciiChar);
}
index++;
} while (byteValue != 0);
}
MemoryMapped.Dispose();
}
catch (FileNotFoundException)
{
Console.WriteLine("Memory-mapped file does not exist. Run Process A first.");
}
}
Somewhere deep in resource processing code we have something like this:
try {
// Try loading some strings here.
} catch {
// Oops, could not load strings, try another way.
}
Exception is thrown and handled already, it would never show up in your application. The only way to see it is to attach debugger and observe this message.
As you could see from the code, it has nothing to do with your problem. The real problem here is what debugger shows you something you should not see.
Run the solution without debugging mode and it works fine.
This exception means that your program does not get Read access to the file from Windows. Have you made sure that this file is not locked when your program tries to read it ? For example, it could be a file that your own program is currently using. If not, try to run your program as an Administrator and see if it makes a difference.
Parsing code for GPS NMEA string
i am trying to parse the incoming GPGGA NMEA GPS string using Arduino uno and below code.
What i am trying to do is that i am using only GPGGA NMEA string to get the values of Latitude, longitude and altitude.In my below code, i had put certain checks to check if incoming string is GPGGA or not, and then store the further string in a array which can be further parsed suing strtok function and all the 3 GPS coordinates can be easily find out.
But i am unable to figure out how to store only GPGGA string and not the further string.I am using a for loop but it isn't working.
I am not trying to use any library.I had came across certain existing codes like this.
Here is the GPGGA string information link
i am trying to have following functionlity
i) Check if incoming string is GPGGA
ii) If yes, then store the following string upto EOL or upto * (followed by checksum for the array) in a array, array length is variable(i am unable to find out solution for this)
iii) Then parse the stored array(this is done, i tried this with a different array)
#include <SoftwareSerial.h>
SoftwareSerial mySerial(10,11); // 10 RX / 11 TX
void setup()
{
Serial.begin(9600);
mySerial.begin(9600);
}
void loop()
{
uint8_t x;
char gpsdata[65];
if((mySerial.available()))
{
char c = mySerial.read();
if(c == '$')
{char c1 = mySerial.read();
if(c1 == 'G')
{char c2 = mySerial.read();
if(c2 == 'P')
{char c3 = mySerial.read();
if(c3 == 'G')
{char c4 = mySerial.read();
if(c4 == 'G')
{char c5 = mySerial.read();
if(c5 == 'A')
{for(x=0;x<65;x++)
{
gpsdata[x]=mySerial.read();
while (gpsdata[x] == '\r' || gpsdata[x] == '\n')
{
break;
}
}
}
else{
Serial.println("Not a GPGGA string");
}
}
}
}
}
}
}
Serial.println(gpsdata);
}
Edit 1:
Considering Joachim Pileborg, editing the for loop in the code.
I am adding a pic to show the undefined output of the code.
Input for the code:
$GPGGA,092750.000,5321.6802,N,00630.3372,W,1,8,1.03,61.7,M,55.2,M,,*76
$GPGSA,A,3,10,07,05,02,29,04,08,13,,,,,1.72,1.03,1.38*0A
$GPGSV,3,1,11,10,63,137,17,07,61,098,15,05,59,290,20,08,54,157,30*70
$GPGSV,3,2,11,02,39,223,19,13,28,070,17,26,23,252,,04,14,186,14*79
$GPGSV,3,3,11,29,09,301,24,16,09,020,,36,,,*76
$GPRMC,092750.000,A,5321.6802,N,00630.3372,W,0.02,31.66,280511,,,A*43
$GPGGA,092751.000,5321.6802,N,00630.3371,W,1,8,1.03,61.7,M,55.3,M,,*75
$GPGSA,A,3,10,07,05,02,29,04,08,13,,,,,1.72,1.03,1.38*0A
$GPGSV,3,1,11,10,63,137,17,07,61,098,15,05,59,290,20,08,54,157,30*70
$GPGSV,3,2,11,02,39,223,16,13,28,070,17,26,23,252,,04,14,186,15*77
$GPGSV,3,3,11,29,09,301,24,16,09,020,,36,,,*76
$GPRMC,092751.000,A,5321.6802,N,00630.3371,W,0.06,31.66,280511,,,A*45
After a quick check of the linked article on the NMEA 0183 protocol, this jumped out at me: <CR><LF> ends the message. This means, that instead of just read indiscriminately from the serial port, you should be looking for that sequence. If found, you should terminate the string, and break out of the loop. Also, you might want to zero-initialize the data string to begin with, to easily see if there actually is any data in it to print (using e.g. strlen).
You could use some functions from the C library libnmea. Theres functions to split a sentence into values by comma and then parse them.
Offering this as a suggestion in support of what you are doing...
Would it not be useful to replace all of the nested if()s in your loop with something like:
EDIT added global string to copy myString into once captured
char globalString[100];//declare a global sufficiently large to hold you results
void loop()
{
int chars = mySerial.available();
int i;
char *myString;
if (chars>0)
{
myString = calloc(chars+1, sizeof(char));
for(i=0;i<chars;i++)
{
myString[i] = mySerial.read();
//test for EOF
if((myString[i] == '\n') ||(myString[i] == '\r'))
{
//pick this...
myString[i]=0;//strip carriage - return line feed(or skip)
//OR pick this... (one or the other. i.e.,I do not know the requirements for your string)
if(i<chars)
{
myString[i+1] = mySerial.read() //get remaining '\r' or '\n'
myString[i+2]=0;//add null term if necessary
}
break;
}
}
if(strstr(myString, "GPGGA") == NULL)
{
Serial.println("Not a GPGGA string");
//EDIT
strcpy(globalString, "");//if failed, do not want globalString populated
}
else
{ //EDIT
strcpy(globalString, myString);
}
}
//free(myString) //somewhere when you are done with it
}
Now, the return value from mySerial.available() tells you exactly how many bytes to read, you can read the entire buffer, and test for validity all in one.
I have a project that will need to pull the same information out of the same sentence.
I got this out of a log file
import serial
import time
ser = serial.Serial(1)
ser.read(1)
read_val = ("nothing")
gpsfile="gpscord.dat"
l=0
megabuffer=''
def buffThis(s):
global megabuffer
megabuffer +=s
def buffLines():
global megabuffer
megalist=megabuffer.splitlines()
megabuffer=megalist.pop()
return megalist
def readcom():
ser.write("ati")
time.sleep(3)
read_val = ser.read(size=500)
lines=read_val.split('\n')
for l in lines:
if l.startswith("$GPGGA"):
if l[:len(l)-3].endswith("*"):
outfile=open('gps.dat','w')
outfile.write(l.rstrip())
outfile.close()
readcom()
while 1==1:
readcom()
answer=raw_input('not looping , CTRL+C to abort')
The result is this:
gps.dat
$GPGGA,225714.656,5021.0474,N,00412.4420,W,0,00,50.0,0.0,M,18.0,M,0.0,0000*5B
Using "malloc" every single time you read a string is an enormous amount of computational overhead. (And didn't see the corresponding free() function call. Without that, you never get that memory back until program termination or system runs out of memory.) Just pick the size of the longest string you will ever need, add 10 to it, and declare that your string array size. Set once and done. There are several C functions for getting substrings out of a string, strtok() using the coma is probably the least overhead. You are on an embedded microcontroller. Keep it small, keep overhead down. :)
#include <stdio.h>
#include <string.h>
#define GNSS_HEADER_LENGTH 5
#define GNSS_PACKET_START '$'
#define GNSS_TOKEN_SEPARATOR ','
#define bool int
#define FALSE 0
#define TRUE 1
//To trim a string contains \r\n
void str_trim(char *str){
while(*str){
if(*str == '\r' || *str == '\n'){
*str = '\0';
}
str++;
}
}
/**
* To parse GNSS data by header and the index separated by comma
*
* $GPGSV,1,1,03,23,39,328,30,18,39,008,27,15,33,035,33,1*5A
* $GNRMC,170412.000,V,,,,,,,240322,,,N,V*2D
* $GNGGA,170412.000,,,,,0,0,,,M,,M,,*57
*
* #data_ptr the pointer points to gps data
* #header the header for parsing GPGSV
* #repeat_index the header may repeat for many lines
* so the header index is for identifying repeated header
* #token_index is the index of the parsing data separated by ","
* the start is 1
* #result to store the result of the parser input
*
* #result bool - parsed successfully
**/
bool parse_gnss_token(char *data_ptr, char *header, int repeat_index, int token_index, char *result) {
bool gnss_parsed_result = FALSE; // To check GNSS data parsing is success
bool on_header = FALSE;
// For header
int header_repeat_counter = 0;
int header_char_index = 0; // each char in header index
// For counting comma
int counted_token_index = 0;
// To hold the result character index
bool data_found = FALSE;
char *result_start = result;
char header_found[10];
while (*data_ptr) {
// 1. Packet start
if (*data_ptr == GNSS_PACKET_START) {
on_header = TRUE;
header_char_index = 0; // to index each character in header
data_found = FALSE; // is data part found
data_ptr++;
}
// 2. For header parsing
if (on_header) {
if (*data_ptr == GNSS_TOKEN_SEPARATOR || header_char_index >= GNSS_HEADER_LENGTH) {
on_header = FALSE;
} else {
header_found[header_char_index] = *data_ptr;
if (header_char_index == GNSS_HEADER_LENGTH - 1) { // Now Header found
header_found[header_char_index + 1] = '\0';
on_header = FALSE;
if (!strcmp(header, header_found)) {
// Some headers may repeat - to identify it set the repeat index
if (header_repeat_counter == repeat_index) {
//printf("Header: %s\r\n", header_found );
data_found = TRUE;
}
header_repeat_counter++;
}
}
header_char_index++;
}
}
// 3. data found
if (data_found) {
// To get the index data separated by comma
if (counted_token_index == token_index && *data_ptr != GNSS_TOKEN_SEPARATOR) {
// the data to parse
*result++ = *data_ptr;
gnss_parsed_result = TRUE;
}
if (*data_ptr == GNSS_TOKEN_SEPARATOR) { // if ,
counted_token_index++; // The comma counter for index
}
// Break if the counted_token_index(token_counter) greater than token_index(search_token)
if (counted_token_index > token_index) {
break;
}
}
// Appending \0 to the end
*result = '\0';
// To trim the data if ends with \r or \n
str_trim(result_start);
// Input data
data_ptr++;
}
return gnss_parsed_result;
}
int main()
{
char res[100];
char *nem = "\
$GNRMC,080817.000,A,0852.089246,N,07636.289920,E,0.00,139.61,270322,,,A,V*04\r\n\\r\n\
$GNGGA,080817.000,0852.089246,N,07636.289920,E,1,5,1.41,11.246,M,-93.835,M,,*5E\r\n\
$GNVTG,139.61,T,,M,0.00,N,0.00,K,A*2F\r\n\
$GNGSA,A,3,30,19,17,14,13,,,,,,,,1.72,1.41,0.98,1*0A\r\n\
$GNGSA,A,3,,,,,,,,,,,,,1.72,1.41,0.98,3*02\r\n\
$GNGSA,A,3,,,,,,,,,,,,,1.72,1.41,0.98,6*07\r\n\
$GPGSV,3,1,12,06,64,177,,30,60,138,15,19,51,322,18,17,42,356,27,1*68\r\n\
$GPGSV,3,2,12,14,36,033,17,07,34,142,17,13,32,267,17,02,21,208,,1*6C\r\n\
$GPGSV,3,3,12,15,05,286,,01,05,037,,03,03,083,,20,02,208,,1*6B\r\n\
$GAGSV,1,1,00,7*73\r\n\
$GIGSV,1,1,00,1*7D\r\n\
$GNGLL,0852.089246,N,07636.289920,E,080817.000,A,A*43\r\n\
$PQTMANTENNASTATUS,1,0,1*4F\r\n";
printf("Parsing GNRMC\r\n");
printf("===============\r\n");
for(int i=1;i<=16;i++){
parse_gnss_token(nem, "GNRMC", 0, i, res);
printf("Index: %d, Result: %s\r\n", i, res);
}
printf("Parsing GNVTG (First Parameter)\r\n");
printf("================================");
// GNVTG - Header, 0 - Repeat Index(if header is repeating), 1 - Value Index,
parse_gnss_token(nem, "GNVTG", 0, 1, res);
printf("\r\nGNVTG: %s\r\n", res);
return 0;
}