Special characters not returned by SQLite3 - c

I have a problem with the SQLite3 C API. I'm trying to read some information from the places.sqlite database that Firefox uses to store many things. What I'm interested in is to retrieve the last visited website and the window title associated.
I wrote this piece of code:
query = "SELECT `url`, `title` FROM `moz_places` WHERE `id`=(SELECT `place_id` FROM `moz_historyvisits` ORDER BY `id` DESC LIMIT 1)";
stmt = NULL;
if (sqlite3_prepare_v2(db, query.c_str(), strlen(query.c_str()) + 1, &stmt, NULL) != SQLITE_OK)
cerr << sqlite3_errmsg(db) << endl;
else
{
while ((ret = sqlite3_step(stmt)) == SQLITE_ROW)
cout << sqlite3_column_text(stmt, 0) << endl << sqlite3_column_text(stmt, 1) << endl;
if (ret != SQLITE_DONE)
cerr << sqlite3_errmsg(db) << endl;
}
When there are no special characters it works fine, but if the window title contains "é", "è" or "•" for example, I get those wonderful characters "é", "è" or "•".
I did some research and found that some people actually got non-UTF8 encoded databases. So I checked the places.sqlite database encoding with both SQLite Manager and the "PRAGMA encoding" request, both say that it is UTF-8 encoded.
Then I created my own database, encoded in UTF-8, entered some special characters in a table and it didn't work either. So I thought that maybe QtCreator (which I'm using) can't show special characters, I tried to get the foreground window title with GetForegroundWindow() and print it in QtCreator application output. It successfully showed the special characters of some window titles.
What am I missing here ?
Thank you.

Weel, seems like I managed to fix that problem by using this function I wrote:
wchar_t *char_to_wchar(const char *str)
{
wchar_t *wbuf;
int wsz;
if (!str)
return (NULL);
wbuf = NULL;
wsz = MultiByteToWideChar(CP_UTF8, 0, str, -1, wbuf, 0);
if (wsz)
wbuf = (wchar_t*)malloc(wsz * sizeof(*wbuf));
else
cerr << "MultiByteToWideChar fail getting required size: " << GetLastError() << endl;
if (wbuf)
{
if (!MultiByteToWideChar(CP_UTF8, 0, str, -1, wbuf, wsz))
cerr << "MultiByteToWideChar fail: " << GetLastError() << endl;
}
return (wbuf);
}
Still, I found it weird that with a function like GetWindowText() I manage to correctly print "é" and other caracters from a simple char *, but here I must use a wchar_t *...
Well, it works at least, let's hope it was the best solution :)

Related

what is wrong with the following while loop?

I have a c winsock code section where a client receives a comma delimited stream of file fingerprints as shown below. I need to extract the fingerprints from the stream using strtok_s() in a while loop. My problem is most of the time the client does not extract the exact number of fingerprints sent from the server, even though the data received(observed by debugging) is exactly what the server sent.
What am I missing here?
recv_size = recv(clnt_sock, fp_buf, BUF_LEN, 0);
received_fp_size += recv_size;
if (0 != (last_string_len = recv_size % 33))
strncpy(last_string, &fp_buf[(recv_size - last_string_len)], last_string_len);//
while (recv_size > 0)
{
unique_fp = strtok_s(fp_buf, ",", &strtk);
k:
while (unique_fp != NULL)
{
memcpy(unique_fp_buf[unique_files_count], unique_fp, 32);
unique_fp = strtok_s(NULL, ",", &strtk);
unique_files_count++;
}
recv_size = recv(clnt_sock, fp_buf, BUF_LEN, 0);
received_fp_size += recv_size;
if (last_string_len > 0)
{
unique_fp = strtok_s(fp_buf, ",", &strtk);
strncat_s(last_string, unique_fp, strlen(unique_fp));
memcpy(unique_fp, last_string, 32);
last_string_len = 0;
goto k;
}
}
The reason behind the if (0 != (last_string_len = recv_size % 33)) line is; The server sends a multiple of 33 byte strings(32 for the fingerprint and 1 for the coma demlimiter)
One problem is that you never check that fp_buf actually contains a complete token. For instance if the first call only receives 20 bytes, your code will fail by copying a partial fingerprint.
I think another problem is here:
memcpy(unique_fp, last_string, 32);
Seems you are copying into the receive buffer and therefore overwrites some data that you haven't processed yet. Further, you may overwrite a token.
Maybe you actually wanted:
memcpy(unique_fp_buf[unique_files_count], last_string, 32);
^^^^^^^^^^^
unique_fp = strtok_s(NULL, ",", &strtk);
unique_files_count++;
Besides that I think you are making the code much more complicated than needed. The use of a goto kind of tell you that your design is wrong.
Instead of using a last_string you could do:
1) Call recv
2) Process all complete fingerprints
3) Copy the remainder (i.e. the last partial fingerprint) to the start of `fp_buf`
4) Call `recv` with an offset into `fp_buf`
5) Repeat from step 2 (i.e. use a while loop - don't use goto
Step 3 could be something like:
recv_size = recv(clnt_sock, fp_buf + length_of_remainder , BUF_LEN - length_of_remainder, 0);
In that way you don't have to handle the last_string stuff

How to call on a renamed file in another function? (Visual C++)

So, I'm trying to create a text file that can be written to and used to seed a random number generator, but I need to be able to write to and call on the file in all of my functions. The file has to be named with the user's first and last name so I found some code that would create a text file and rename it, I'm just having trouble calling the file in my other functions. I included the beginning part of one of my other functions after the code for the text file creation. I'm going to need to send a time stamp to the file and use that time to seed my RNG.
int user_file_name()
{
string tstamp = get_timestamp();
//Creating input/output file using user's name
ofstream user_file;
string filename;
cout << "What is your first and last name?\n" << endl;
getline(cin, filename);
filename += ".txt";
cout << "Thank you, " << filename << "." << endl << endl;
user_file.open(filename.c_str());
user_file << tstamp;
user_file.close();
return 0;
}
int addition()
{
char DIFFICULTY;
difficulty_menu();
cin >> DIFFICULTY;
get_timestamp();
string tstamp = get_timestamp();
Why not just have user_file_name return the string filename instead of just an int? You can then store that filename somewhere so that other areas of your program can access it.

Opening registry key in c

I have create a key using regedit, now I want to get its value. It doesn't give any error but it isn't showing anything.
Code :
int main() {
HKEY hKey;
RegOpenKeyEx(HKEY_LOCAL_MACHINE, L"Ehsan Akbari", 0, KEY_ALL_ACCESS, &hKey);
TCHAR sz[50];
DWORD size = 50,type;
RegEnumValue(hKey, 0, L"test", &size, NULL, &type, (LPBYTE)sz, &size);
RegCloseKey(hKey);
getch();
return 0;
}
An image of regedit :picture
What am I doing wrong?
Edit
When I debugged I saw that hKey is NULL, but GetLastError doesn't report anything.
Here are the immediate problems that I can see:
You detect no errors because you don't check for errors. Read the documentation for each function. The error code is returned in the return value.
You ask for KEY_ALL_ACCESS which won't be granted under HKLM. You need to request just read access KEY_READ.
Your screenshot shows the key has been created under HKCU, and you're trying to open it under HKLM.
RegEnumValue expects the size of the data buffer in bytes. You pass the length, the number of characters.
You are mixing Unicode literals and TCHAR. This is pointless. Your code won't compile targeting MBCS and in any case you don't care about Win98 any more. Stop using TCHAR and use wchar_t instead.
The lpValueName parameter must be a modifiable buffer. You pass a literal. Remember that this function enumerates values. It does not read specific named values as perhaps you expect.
The lpcchValueName parameter contains the size of the buffer you passed to lpcchValue in characters. You pass the length of the data buffer.
The data returned may not be null terminated. You must protect against this as described in the documentation.
For a C program which ignores its arguments, the correct main is int main(void).
I expect there are more errors but I stopped looking at this point. I recommend you spend some quality time with the documentation.
To open the path "Ehsan Akbari" in HKEY_CURRENT_USER you could try this:
HKEY hKey;
long result = RegOpenKeyEx(HKEY_CURRENT_USER , TEXT("\\Ehsan Akbari"), 0, KEY_ALL_ACCESS, &hKey);
if ( result == ERROR_SUCCESS )
{
cout << "OK" << endl;
}
else
{
cout << "Error " << result << endl;
}

Creating an XML file in OpenCV

I have created an XML file, but when I try to read it in I get an error about a duplicate key. When I opened the XML file I noticed that nothing was nested and everything was treated as it's own individual node. I was wondering if there was a way to properly create the XML file and be able to parse through it as well. Below I have included my code for writing an XML file out, my XML output, and what I would like it to be.
XML Code:
// Open up the file so that we can write to it
cv::FileStorage fs;
fs.open(szFileName, cv::FileStorage::WRITE);
// Check if we actually created the file
if(fs.isOpened())
{
fs << "Total_Images" << (int)vImages.size();
//cv::write(fs, "Total Images", (int)vImages.size());
for(int i = 0; i < (int)vImages.size(); i++)
{
char szSize[8];
string szNode;
szNode = "image_";
itoa(i, szSize, 10);
szNode += szSize;
fs << "Node" << szNode;
fs << "Width" << vImages[i]->GetWidth();
fs << "Height" << vImages[i]->GetHeight();
fs << "Keypoint_Size" << (int)vImages[i]->GetKeyPoints().size();
for(int j = 0; j < vImages[i]->GetKeyPoints().size(); j++)
{
char szSize[256];
fs << "Angle" << vImages[i]->GetKeyPoints()[j].angle;
fs << "Class" << vImages[i]->GetKeyPoints()[j].class_id;
fs << "Octave" << vImages[i]->GetKeyPoints()[j].octave;
fs << "Pt_X" << vImages[i]->GetKeyPoints()[j].pt.x;
fs << "Pt_Y" << vImages[i]->GetKeyPoints()[j].pt.y;
fs << "Response" << vImages[i]->GetKeyPoints()[j].response;
fs << "Size" << vImages[i]->GetKeyPoints()[j].size;
}
fs << "Descriptors" << vImages[i]->GetDescriptors();
}
fs.release();
}
XML Output:
<?xml version="1.0"?>
<opencv_storage>
<Total_Images>1</Total_Images>
<Node>image_0</Node>
<Width>500</Width>
<Height>266</Height>
<Keypoint_Size>268</Keypoint_Size>
<Angle>8.7332672119140625e+001</Angle>
<Class>0</Class>
<Octave>-1</Octave>
<Pt_X>1.7401049804687500e+002</Pt_X>
<Pt_Y>6.5084960937500000e+001</Pt_Y>
<Response>0.</Response>
<Size>1.1386331558227539e+001</Size>
<Angle>9.3859291076660156e+001</Angle>
<Class>0</Class>
<Octave>-1</Octave>
<Pt_X>1.7674386596679687e+002</Pt_X>
<Pt_Y>6.4983604431152344e+001</Pt_Y>
<Response>0.</Response>
<Size>1.2262072563171387e+001</Size>
<Angle>9.8604904174804688e+001</Angle>
<Class>0</Class>
<Octave>-1</Octave>
<Pt_X>1.1859683990478516e+002</Pt_X>
<Pt_Y>6.6855834960937500e+001</Pt_Y>
<Response>0.</Response>
<Size>1.1112688064575195e+001</Size>
...
Expected Output
<?xml version="1.0"?>
<opencv_storage>
<Total_Images>1</Total_Images>
<Node>image_0
<Width>500</Width>
<Height>266</Height>
<Keypoint_Size>268</Keypoint_Size>
<KeyPoint>
<Angle>8.7332672119140625e+001</Angle>
<Class>0</Class>
<Octave>-1</Octave>
<Pt_X>1.7401049804687500e+002</Pt_X>
<Pt_Y>6.5084960937500000e+001</Pt_Y>
<Response>0.</Response>
<Size>1.1386331558227539e+001</Size>
</KeyPoint>
<KeyPoint>
<Angle>9.3859291076660156e+001</Angle>
<Class>0</Class>
<Octave>-1</Octave>
<Pt_X>1.7674386596679687e+002</Pt_X>
<Pt_Y>6.4983604431152344e+001</Pt_Y>
<Response>0.</Response>
<Size>1.2262072563171387e+001</Size>
</KeyPoint>
<KeyPoint>
<Angle>9.8604904174804688e+001</Angle>
<Class>0</Class>
<Octave>-1</Octave>
<Pt_X>1.1859683990478516e+002</Pt_X>
<Pt_Y>6.6855834960937500e+001</Pt_Y>
<Response>0.</Response>
<Size>1.1112688064575195e+001</Size>
</KeyPoint>
...
</Node>
Thanks
You can embedd a node inside another node.
Do it like this (using "{" and "}" ):
FileStorage fs; // Open it and check that it is opened;
fs << "SimpleData" << 1;
fs << "Structure << "{";
fs << "firstField" << 1;
fs << "secondField" << 2;
fs << "}"; // End of structure node
fs << "SimpleData2" << 2;
Actually we can see the "Duplicated key" error in opencv/modules/core/src/persistence.cpp line 711
The key seems have sth to do with the node name in your xml file: once you have many nodes with the same name like below,it will has duplicated key.And even you haven't read but just open this file,it will occur.So, just modify your node names,how about add an index "node_1"?
<opencv_storage>
<mynodes>
<node>1</node>
<node>2</node>
<node>3</node>
<node>4</node>
...
<node>10</node>
</mynodes>
You can also try to use JSON which is more fash and simple to interpret
File storage is not designed for this type of use, however you can use a XML-parser/writter, the easiest, fastest and most fashion one is Xerces-C. You can download it here
Actually, the filestorage object is made to store simple datas like calibration from one runtime to the other... From what i know there is no way to structure the nodes between each others...
Julien,
EDIT :
READ DATA FROM A MAT
for(int i=0;i<img.rows;i++) {
unsigned char* row = img.data + i * img.step;
for(int j=0;j<img.cols;j++) {
unsigned char* channB = (row + 3*j);
unsigned char* channG = (row + 3*j + 1);
unsigned char* channR = (row + 3*j + 2);
}
}
WRITE WITH XERCES-C
The "DOMPrint" example shows how to write in a document with a serializer...
However, let me tell you that i think that OpenCV is enough for your example, is it a big problem that yuor datas are in nodes next to each other ? I'll try to find an easier way, let me think of it, i'll post back here (still, you sould try Xerces-C it's great and usefull)

Missing characters from input stream from fastcgi request

I'm trying to develop simple RESTful api using FastCGI (and restcgi). When I tried to implement POST method I noticed that the input stream (representing request body) is wrong. I did a little test and looks like when I try to read the stream only every other character is received.
Body sent: name=john&surname=smith
Received: aejh&unm=mt
I've tried more clients just to make sure it's not the client messing with the data.
My code is:
int main(int argc, char* argv[]) {
// FastCGI initialization.
FCGX_Init();
FCGX_Request request;
FCGX_InitRequest(&request, 0, 0);
while (FCGX_Accept_r(&request) >= 0) {
// FastCGI request setup.
fcgi_streambuf fisbuf(request.in);
std::istream is(&fisbuf);
fcgi_streambuf fosbuf(request.out);
std::ostream os(&fosbuf);
std::string str;
is >> str;
std::cerr << str; // this way I can see it in apache error log
// restcgi code here
}
return 0;
}
I'm using fast_cgi module with apache (not sure if that makes any difference).
Any idea what am I doing wrong?
The problem is in fcgio.cpp
The fcgi_steambuf class is defined using char_type, but the int underflow() method downcasts its return value to (unsigned char), it should cast to (char_type).
I encountered this problem as well, on an unmodified Debian install.
I found that the problem went away if I supplied a buffer to the fcgi_streambuf constructor:
const size_t LEN = ... // whatever, it doesn't have to be big.
vector<char> v (LEN);
fcgi_streambuf buf (request.in, &v[0], v.size());
iostream in (&buf);
string s;
getline(in, s); // s now holds the correct data.
After finding no answer anywhere (not even FastCGI mailing list) I dumped the original fastcgi libraries and tried using fastcgi++ libraries instead. The problem disappeared. There are also other benefits - c++, more features, easier to use.
Use is.read() not is >> ...
Sample from restcgi documentation:
clen = strtol(clenstr, &clenstr, 10);
if (*clenstr)
{
cerr << "can't parse \"CONTENT_LENGTH="
<< FCGX_GetParam("CONTENT_LENGTH", request->envp)
<< "\"\n";
clen = STDIN_MAX;
}
// *always* put a cap on the amount of data that will be read
if (clen > STDIN_MAX) clen = STDIN_MAX;
*content = new char[clen];
is.read(*content, clen);
clen = is.gcount();

Resources