Writing a GSM modem driver? - c

I've been working on an application which uses a GSM modem for one of two things; check its status using the built in HTTP stack by sending a GET request to the server, or sending data to the server (using UDP). I have tried several different methods to keep this as reliable as possible, and I'm finally ready to ask for help.
My application is written for the SIMCOM908 module and the PIC18 platform (I'm using a PIC18 Explorer for development).
So the problem is sometimes the modem is busy doing something, and misses a command. As a human, I would see that and just resend the command. Adding a facility for my MCU to timeout and resend isn't an issue.
What is an issue is that the modem sends unsolicited responses after different events. When the modem changes registration status (with the cell tower) it would respond with +CGREG: 1, ... or when the GPS is ready GPS Ready. These responses can happen at any time, including in the middle of a command (like creating an IP connection).
This is a problem, because I haven't thought of a way to deal with this. My application needs to send a command (to connect to the server for example, AT+CIPSTART="UDP","example.com",5000) This command will response with 'OK', and then when the command has finished 'CONNECT OK'. However, I need to be able to react to the many other possible responses, and I haven't figured out a way of doing this. What do I need to do with my code to; wait for a response from the modem, check the response, perform an action based on that response?
I am code limited (being an 8-bit microcontroller!) and would like the keep repetition to a minimum. How can I write a response function that will take a response from the GSM module (solicited or now) and then let the rest of my program know what is happening?
Ideally, I'd like to do something with those responses. Like keep an internal state (when I hear GPS Ready, I know I can power the GPS etc.
Maybe there are some things I should think about, or maybe there's an open source project that already solves this problem?
Here's what I have so far:
/* Command responses */
enum {
// Common
OK = 0,
ERROR,
TIMEOUT,
OTHER,
// CGREG
NOT_REGISTERED,
// CGATT
NOT_ATTACHED,
// Network Status
NO_NETWORK,
// GPRS status
NO_ADDRESS,
// HTTP ACTION
NETWORK_ERROR,
// IP Stack State
IP_INITIAL,
IP_STATUS,
IP_CONFIG,
UDP_CLOSING,
UDP_CLOSED,
UDP_CONNECTING
} gsmResponse;
int gsm_sendCommand(const char * cmd) {
unsigned long timeout = timer_getCurrentTime() + 5000;
uart_clearb(GSM_UART); // Clear the input buffer
uart_puts(GSM_UART, cmd); // Send the command to the module
while (strstr(bf2, "\r") == NULL) { // Keep waiting for a response from the module
if (timeout < timer_getCurrentTime()) { // Check we haven't timed out yet
printf("Command timed out: %s\r\n", cmd);
return TIMEOUT;
}
}
timer_delay(100); // Let the rest of the response be received.
return OK;
}
int gsm_simpleCommand(const char * cmd) {
if (gsm_sendCommand(cmd) == TIMEOUT)
return TIMEOUT;
// Getting an ERROR response is quick, so if there is a response, this will be there
if (strstr(bf2, "ERROR") != NULL)
return ERROR;
// Sometimes the OK (meaning the command ran) can take a while
// As long as there wasn't an error, we can wait for the OK
while (strstr(bf2, "OK") == NULL);
return OK;
}
A simple command is any AT command that is specifically looking for OK or ERROR in response. Something like AT. However, I also use it for more advanced commands like AT+CPIN? because it means I will have captured the whole response, and can further search for the +CPIN: READY. However, none of this actually response to the unsolicited responses. In fact, the gsm_sendCommand() function will return early when the unsolicited response is received.
What a good way to manage complex, occasionally unsolicited, status messages like this? Please take note that this application is written in C, and runs on an 8bit microcontroller!

Having to handle both unsolicited messages as well as responses to requests in the same data stream is difficult since you will need to demultiplex the incoming stream and dispatch the results to the appropriate handler. It's a bit like an interrupt handler in that you have to drop what you were doing and handle this other bit of information which you were not necessarily expecting.
Some modules have a secondary serial port which can also be used for messages. If this is possible you could have unsolicited messages only appear on a single serial port while the main port is for your AT commands. This may not be possible, and some GSM modules will not support the complete command set on a secondary port.
Perhaps a better approach is to just disable unsolicited messages. Most commands all the state to be requested. eg While waiting for registration, instead of waiting for an unsolicited registration message to appear, simply poll the module for the current registration state. This allows you to always be in control, and you only have to handle the responses for the command just sent. If you're waiting for multiple events you can poll in a loop for each item in turn. This will generally make the code simpler as you only have to handle a single response at a time. The downside is that your response times are limited by your polling rate.
If you're set on continuing with the unsolicited message approach, I'd suggest implementing a small queue for unsolicited messages. While waiting for responses to a command, if the response does not match the command, just push the response on a queue. Then, when you've either received a response to your AT command or timed out you can process the unsolicited message queue afterwards.

Related

RxFrameNtf, TxFrameNtf and Ntf.data in unetpy

I am using Unetstack software along with Unetpy. I wish to retrieve transmit and recieve notifications when I run .py file which imports Unetpy python library. I followed this tutorial
I am successfully able to connect to the localhost and print values like phy.MTU and so on. When I transmit a packet I also receive a reply saying AGREE on the command prompt.output_of_my_script
my_script
Can you please help me in receiving Txframentf and rxframentf along with data payload.
I have made changes posted in bug reports suggested in this linkeven.
Please guide me on how to print notifications for rxframe and txframe.
Thank you``
Your script is fine until the last line:
print(phy << org_arl_unet_phy.TxFrameNtf())
Here you are trying to send a TxFrameNtf to the physical agent. This does not make sense, as it is the physical agent who sends you such a notification when a transmission is completed.
By the time you reach this line, you should have already received the notification as txntf as long as the transmission was completed within 5 seconds (timeout=5000). To print out the notification, all you need to do is:
print(txntf)
I just tested this against the 3-node-network.groovy sample. I am using unetpy-1.3b5 and fjagepy-1.4.2b3. Here's the modified code:
from unetpy import *
modem = UnetGateway('localhost', 1102)
phy = modem.agentForService(Services.PHYSICAL)
print(phy.MTU)
print(phy.basebandRate)
print(phy << org_arl_unet_phy.TxFrameReq(to=3, data=[1,2,3,4]))
txntf = modem.receive(timeout=5000)
print(txntf)
and the output:
16
4096
AGREE
TxFrameNtf:INFORM[type:1]
You can see that the TxFrameNtf is correctly received.
For reception, you need to subscribe to the agent's notifications and then receive a frame:
modem.subscribe(phy)
rxntf = modem.receive(org_arl_unet_phy.RxFrameNtf, timeout=5000)
print(rxntf)
Assuming you receive a frame within the 5 second timeout specified (in this example, on node 3), this should print out something like:
RxFrameNtf:INFORM[type:CONTROL from:1 to:3 protocol:0 rxTime:34587658 (4 bytes)]
You sent a datagram through some agent that supports the DATAGRAM service. There may be many agents that support this service (not just the physical layer). In any case, that datagram would be received on a different node, and so you wouldn't expect to receive DatagramNtf on the transmitting node.
The RangeReq should yield a RangeNtf if successful, but that might take more than the default receive timeout of 1 second, depending on how far node 2 is. So you might want to try a longer receive timeout to see if you get your notification.
To access the data from payload from the rxntf, you can try print(rxntf.data).

Akka http file streaming not writing bytes more than 'n' size from different clients

`Akka http 10.0.6;
max-content-length is set to 6000m`
I am using Akka file streaming to upload huge files( sent as octet-stream to my service ) to accept incoming bytes and write it to a file sink. This is what I observe from my experimentations.. My limited understand from reading the documents is the client should be able keep sending the data unless we tell explicitly from akka http for back pressure mechanisms.. Been searching online to understand this behavior, not able to get an understanding yet on this to explain the following behavior.. Is there something I am missing in the code? How can I debug more on this? Also, via scalatest, able to do it. If someone can throw more might on what’s the difference in behavior w.r.t scalatest and via curl/http clients.. Thanks
Through "curl", I can stream max of 1KB sized file. Anything more than this, it will just hang and this message is given after timeout no matter how long I wait(20 seconds, 5 mins, 10 mins, etc) "Sending an 2xx 'early' response before end of request was received... Note that the connection will be closed after this response. Also, many clients will not read early responses! Consider only issuing this response after the request data has been completely read!"
 
Through "Apache http client", max of 128KB sized file streaming happens. Anything more than this, it will hang and same message as in #1 from akka http service
 
Through "python client", max of ~26KB sized file streaming happens. Anything more than this, it will hang and same message as in #1 from akka http service
 
Through scalatest inside the service, was able to upload files like 200MB, 400MB and more also
Here’s the code..
`put {
withoutSizeLimit {
extractDataBytes { bytes =>
 
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
// Tried system dispatcher also
implicit val executionContext = system.dispatchers.lookup("dispatcher")
val sink = FileIO.toPath(Paths.get("/file.out"))
val action = bytes.runWith(sink).map {
case ior if ior.wasSuccessful => {
complete(StatusCodes.OK, s"${ior.count} bytes written")
}
case ior => complete(StatusCodes.EnhanceYourCalm, ior.getError.toString)
}
Await.result(action, 300.seconds)
}
}
}`

Apache C module how to send response to client

I have written an Apache module that handles receiving a file from a client. I now want to send a response back to the client. I want the response to contain a string representing the file path to the file sent to the module. Since I am new to writing Apache modules I am unsure as to whether there is a response struct of some sort I need to use or if everything I need is in the request_rec passed into my handler. I noticed that ap_rprintf sends data to the client. Should I just use that? And if so, how is it sent back to the client (i.e. how can my client extract the string from things sent back to it)?
Thanks!
Edit:
I just stumbled across apr_socket_send() but I don't know if that works in this case. request_rec stores the connection, so could I create a socket to the client and send the data back that way?
Have you checked out the source code for mod_example?
Basically:
r->content_type = "text/html";
ap_send_http_header(r);
ap_rputs(DOCTYPE_HTML_3_2, r);
ap_rputs("<HTML>\n", r);
...
ap_rprintf(r, "Stuff that you want to send in the body");
...

G-WAN persist request data in handler

Solution:
int session = (int)get_env(argv, SESSION_ID); to get identifier unique to connection
US_VHOST_DATA (vhost) or US_HANDLER_DATA (listener) or US_SERVER_DATA (server) for data persistent > current connection
Missing pieces:
either persistent data for connection only
or some way to execute code when current connection is closed by client (econnreset etc.) or server (e.g. kalive_tmo reached)
This should be solvable as soon as a new HDL_BEFORE_CLOSE state for handlers is added, which makes this question answered for me.
Original Question:
Is it possible in a G-WAN handler to store information persistent to a request/connection (don't really know if "request" applies here)?
To better illustrate what I mean, this is what I got now:
Client (browser, javascript) sends websocket handshake
Handler starts, gets into:
HDL_AFTER_ACCEPT - here i call gc_init for US_REQUEST_DATA, and get no error
HDL_AFTER_READ - here i check for US_REQUEST_DATA which is not yet set, so I do websocket handshake and gc_malloc + set US_REQUEST_DATA, increase KALIVE_TMO, and then return 2 to send data
Client sees websocket connection as being established, so I (manually triggered some seconds afterwards) send a message
Handler goes to HDL_AFTER_READ again, BUT US_REQUEST_DATA is not set
What I've also tried:
returning 1 instead of 2 in HDL_AFTER_READ -> client gets 404 and handshake does not work
At the moment I'm only using US_REQUEST_DATA to identify if websocket connection is already established and next incoming data should be in websocket message format, so if there is a different (maybe better?) solution, I'm open to that as well of course.
Thanks!
Edit: Added clarification about request/connection
I am not sure why US_REQUEST_DATA does not seem to keep your allocated block of memory.
Can you try the persistence.c example to see if it works as expected for you?
Other than G-WAN persistent pointers, you can use OS services like the Linux shared memory API, etc.
But the G-WAN API should work fine once you copy & paste the example above.
Other values have different scopes:
US_VHOST_DATA (scope:vhost)
US_HANDLER_DATA (scope:listener)
US_SERVER_DATA (scope:server)
Use the session ID below which is unique to each CONNECTION:
int session = (int)get_env(argv, SESSION_ID);

Why are these deferred tasks not being executed in the order in which they were added?

I'm using Twilio to send sms's with appengine. Twilio doesn't accept sms's longer than 160 characters so I have to split them. I am splitting the sms's and sending them as follows:
def send_sms_via_twilio(mobile_number, message_text):
client = TwilioRestClient(twilio_account_sid , twilio_auth_token)
message = client.sms.messages.create(to=mobile_number, from_=my_twilio_number, body=message_text)
split_list = split_sms(long_message)
for each_message in split_list:
send_sms_via_twilio(each_message)
However I found that the order of sending varied. For example sometimes I'd recieve message 2/5 then 1/5 then 4/5 etc and other times the order would be correct. The order of the split_list is definately correct. To overcome the incorrect order of the sms's I tried
for each_message in split_list:
deferred.defer(send_sms_via_twilio, each_message, _countdown=1)
However I encountered the same problem. I then tried
for each_message in split_list:
deferred.defer(send_sms_via_twilio, each_message, _countdown=1, _queue="send-text-message")
and defined my queue as
- name: send-text-message
rate: 1/s
bucket_size: 10
max_concurrent_requests: 1
retry_parameters:
task_retry_limit: 5
Thinking that the issue was concurrency (running in python27) and that if I limited max_concurrent_requests this issue would be solved. However the issue is still present i.e. the texts still get sent in the wrong order. I checked the logs but couldnt see any notification of task failure - they just seem to be executing in the wrong order.
Is there something I am missing? How can I fix this issue.
Note that the SMS messaging (specifically the underlying protocols like SMPP) are asynchronous by definition. It means there is no way you can specify the order of distinct SMS messages.
There is a way to specify the order of SMS packets by using the UDH (user defined headers) in the binary body of those messages. But this works only for long SMS messages -- those that are too long to be sent in one message. For example, if your msg exceeds 160 GSM-7 characters or 80 UTF-16 characters it will be send as more than one message with UDH.
In that case the mobile phone won't show message parts as they arrive. It will collect them in memory until the last one comes and then assembles them in the right order. For the end user this is just a message longer than usual and you don't have to write "1/3", "2/3", ... in the message.
Disclaimer: I work for a company that enables you to send and receive both multiple binary messages with user-specified headers (UDH) and/or standard long messages.
If you are not tied to Twilio try using SMSified. They automatically split the message for you, insure it is in the correct order, and add "1/2, 2/2..." to the end of the message. In other words you just send the complete message to their REST API, no matter the length, and they handle the rest. Since they also use a REST API you can continue to use Python.

Resources