The problem
1- I'd like to create a MIME message. Something like this:
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="---12345"
This is a multipart message in MIME format.
---12345
Content-Type: text/plain
This is the plain text
---12345
Content-Type: application/pdf
>> PDF binary code here <<
---12345
2- Then i need to pass it to the OpenSSL functions in the form of BIO * data.
PKCS7 *PKCS7_sign(..., ..., ..., BIO *data, ...);
The first approach
Load the plain text and attachment data from the filesystem and assign it to char * data, manipulate the data to add the respective MIME headers, finally assign it to BIO * memoryBIO with BIO_puts(memoryBIO, data);.
But this approach doesnt work because the binary data contains "\0"(NULL) which wont go well with char type.
The second approach
Assign plain text and attachment to a BIO each and then "concatenate" them. But couldn't find a way to do this.
Conclusion
I'd like to know if there is a way to accomplish such feat.
I'd also like to avoid using intermediary files and build everything in-memory.
After following jww's suggestion, what i ended up using is BIO_write();.
BIO * inBIO = NULL;
std::vector<unsigned char> data = GetData();
inBIO = BIO_new(BIO_s_mem());
BIO_write(inBIO, data.data(), data.size());
Again thanks to jww for this answer and for all the other answers regarding OpenSSL in SO, you've helped me a great deal.
Related
Is it possible to encode a C char array (char*) in JSON in order to send it through a TCP socket? If so how can I make it and which would be a simple JSON library to use?
Thanks,
You probably don't need to use JSON. If you just want to send a simple message (e.g. "Hi, how are ya"), then you can just send the char array. You cannot send JSON, only a char array of JSON (JSON is a data format, not a transfer protocol).
Example
const char *json = "{\"id\": 12837, \"name\": \"Massimo Isonni\"}"
// You would then send json.
If you need to serialize a struct into a JSON string, I would recommend this.
I'm encrypting a file and saving it in AWS. What Media Type (FKA MIME-type) is encrypted text? "text/plain" doesn't seem right.
Use
application/octet-stream
which is intended for arbitrary binary data - perfect for an encrypted file that by definition can't be interpreted. (RFC2046)
I am using react with fetch for sending an image to the server.
I have tried using Content-type = application/x-www-form-urlencoded to pass my data to the server but it replaces special characters with spaces (i.e. + becomes a space).
I have switched the header to be Content-type: multipart/form-data but that throws the error
Failed to load resource: the server responded with a status of 500
(Internal Server Error).
I have added a boundary to the Content-type as boundary=abcdefg.
That did not change anything and I am not sure what my boundary would be.
Finding a clear answer with straight forward examples about boundaries have been impossible to get.
The data that I am sending is a large string.
If needed I can post that as well.
Here is a sample of the code that is causing the problem:
SaveTest4(data: string) {
const options = {
method: 'post',
headers: {
"Content-type": "multipart/form-data; boundary=abcdefg"
},
body: 'data=' + data
}
fetch('api/DataPoint/AddTest4', options);
}
Based on part of your analysis, it sounds like you're trying to send base64-encoded data. A content type of application/x-www-form-urlencoded will result in the server performing URL decoding, which will replace each instance of + in the content body with a space character.
When you used a content type of multipart/form-data, the server fails with status 500 because the data you provided wasn't a properly constructed MIME document.
My psychic debugging powers tell me that you're trying to post a base64-encoded file to a ASP.NET MVC WebAPI endpoint that's expecting a JSON document. You might have a controller method that looks like this:
[HttpPost("api/DataPoint/AddTest4")]
public void AddTest4([FromBody] string data) { ... }
If you send with a content type of application/json, this endpoint will expect a document that looks like this:
"{base64-encoded-data}"
Note that there are quotes around the data, because a quoted string is a proper JSON document. You'd just use JSON.stringify() to create the quoted string in this case, which would escape any quotes within the string correctly.
If you send with application/x-www-form-urlencoded, you'd need to send a document that looks like this:
data={base64-encoded-data}
But, as you note, you'd have to make sure you escape all of the special characters in the payload; you can do this using window.encodeURIComponent(), which would translate each "+" to "%2B", among other things.
If the files that you're uploading to this endpoint are large, it would be significantly better to use an instance of FormData. This would allow the browser to stream the file to the server in chunks instead of reading it into memory and base64-encoding it in JavaScript.
Hello I've got a problem trying to create a twitter console app. When getting a response I've got some thrash in it.
Here my code is:
char new_request[1024] = "GET /1.1/statuses/user_timeline.json?count=4&screen_name=twitterapi HTTP/1.1\r\nHost: api.twitter.com\r\nUser-Agent: twitter-terminal-app$
strcat(new_request, bearer);
strcat(new_request, "\r\nAccept-Encoding: gzip\r\n\r\n\0");
BIO_write(bio, new_request, strlen(new_request));
printf("%s\n", new_request);
p = BIO_read(bio, ans, 2047); // Getting header
ans[p] = 0;
printf("%s\n", ans);
char ans2[100000] = "";
p = BIO_read(bio, ans2, 10000); // Getting body
BIO_should_retry(bio);
FILE *file = fopen("result.txt", "w+");
fputs(ans2, file);
printf("%s\n%i\n", ans2, p);
The answer I have in body looks like that:
▒▒is▒HǿJ▒_
▒▒▒▒V▒▒▒▒.▒▒T▒▒f
▒Tm▒m
▒P▒▒1▒|▒}▒%▒▒▒▒Q^▒▒▒▒?▒cx{Չ
F%▒C*;▒▒ŴD/#▒L▒▒▒▒?L▒A▒▒▒N▒▒ĝ▒▒▒▒$▒El▒▒▒▒X▒▒B0▒~%▒▒5▒˲#Y)GY▒ctz▒h&-▒▒▒O>AG▒▒▒l6b▒z▒:K
▒T▒\▒▒2+▒b▒H▒&B ▒▒▒B▒qV▒▒▒▒▒▒
▒▒a_▒l▒?#▒o▒▒▒W▒▒u%▒▒▒;▒1M▒v▒▒L▒▒
Maybe the answer is encoded somehow and that's the problem. Tried to serf dev.twitter.com but didn't find any answer. If I use BIO_gets() instead of BIO_read() the answer is -2. Any ideas?
strcat(new_request, "\r\nAccept-Encoding: gzip\r\n\r\n\0");
...
p = BIO_read(bio, ans, 2047); // Getting header
...
p = BIO_read(bio, ans2, 10000); // Getting body
... Maybe the answer is encoded somehow
You've declared in your HTTP request that you support data compressed with gzip and that's why the server has send you the data compressed. If you would not only read and ignore the HTTP response header but actually take a look at it you would probably notice:
Content-Encoding: gzip
Apart from explicitly allowing compressed data you are doing a HTTP/1.1 request. This means that you would also need to be able to deal with chunked transfer encoding. HTTP/1.1 also implies that by default that connections are persistent so you would need to properly parse the header to find out where the response really ends instead of relying on connection end. You also cannot rely on a fixed size of the header or that header and body can be read with separate BIO_read calls. For example you might need multiple reads for the body or the body might already be included in the single read you do for the header.
Unless you really want to deal with all these problems yourself I recommend you better use an existing library which implements this properly and thus gets you the correct response reliably and not by chance.
If you instead want to learn how this is done I recommend you start with learning more about HTTP, i.e. by reading the wikipedia entry and then continue with all the standards referenced there. I suggest to start with HTTP/1.0 since this is simpler than HTTP/1.1.
I have a simple Restlet service hosted on AppEngine. This performs basic CRUD operations with strings and is working well with all sorts of UTF-8 characters when I test it with curl (for all the verbs).
This is consumed by a simple restlet client hosted in a servlet on another AppEngine app:
// set response type
resp.setContentType("application/json");
// Create the client resource
ClientResource resource = new ClientResource(Messages.SERVICE_URL + "myentity/id");
// Customize the referrer property
resource.setReferrerRef("myapp");
// Write the response
resource.get().write(resp.getWriter());
The above is pretty much all I have in the servlet. Very plain.
The servlet is invoked via jquery ajax, and the json that I get back is well formed and everything, but the problem is that UTF-8 encoded strings are coming back scrambled, for example:
Université de Montréal becomes Universit?? de Montr??al.
I tried adding this line in the servlet (before everything else):
resp.setCharacterEncoding("UTF-8");
But the only diference is that instead of getting ?? I get Universitᅢᄅ de Montrᅢᄅal (I don't even know what kind of characters those are, asian I suppose).
I am 100% sure the restlet service is OK, because other than debugging it line by line I am able to test it from cmd line with curl and it's returning well formed strings.
By looking at the http header of the response from firefox (when calling the servlet via javascript) I can see the encoding is indeed UTF-8, as expected. After hours of struggling reading every possible related article I came across this restlet discussion and noticed that indeed I do have Transfer-Encoding: chunked on the http header of the response. I tried the proposed solutions (override ClientResource.toRepresentation, didn't do any good so I tried restlet 2.1 as susggested with ClientResource.setRequestEntityBuffering(true), no luck there either) but I am not convinced my issue is related to Transfer-Encoding: chunked at all.
At this point I am out of ideas, and I would really appreciate any suggestions! O_o
UPDATE:
I tried doing a manual GET with a classic UrlConnection and the string is coming back alright:
URL url = new URL(Messages.SERVICE_URL + "myentity/id");
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
resp.getWriter().print(writer.toString());
So much for being all RESTful and fancy ...but still I have no clue why the original version doesn't work! :/
I tried doing a manual GET with a classic UrlConnection and the string is coming back alright:
URL url = new URL(Messages.SERVICE_URL + "myentity/id");
URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
resp.getWriter().print(writer.toString());
So much for being all RESTful and fancy ...but still I have no clue why the original version doesn't work! :/
Does your response contain the appropriate "Content-Type" header? It should be something like "Content-Type: application/json; charset=UTF-8" (note the charset).
Try starting your development server and retrieving your resource from the command line using cURL and inspecting the headers, e.g. curl -i http://localhost:8080/myentity/id. In theory browsers should assume UTF-8 for JSON, but I wouldn't trust on that.