I need to create a TCP connection in AppEngine (on the server side) and keep it alive indefinitely (or at least ~30 minutes). I've created a background thread, opened a java.net.Socket and attempted to read a line with a wrapped BufferedReader. After about 3 seconds of inactivity, I get the following exception:
java.net.SocketException: Socket operation timed out: The API call remote_socket.Receive() took too long to respond and was cancelled.
I'll include a skeleton of the code I'm using below. Any help would be greatly appreciated (including workarounds or information on the limitations of sockets in GAE). Thanks!
ThreadFactory tm = ThreadManager.backgroundThreadFactory();
thread = tm.newThread(new Runnable() {
#Override
public void run() {
Socket socket = null;
try {
socket = new Socket("localhost", 8000);
socket.setSoTimeout(0);
socket.setKeepAlive(true);
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
String in = reader.readLine();
// execution never gets past here
//...
} catch (IOException e) {
e.printStackTrace();
} finally {
// ... <close socket>
}
}
});
thread.start();
In case it's relevant, here's the use case: I've got a TCP server that used to work with an embedded Applet on the client side. I'd like to redesign the Applet as a webapp, but the server still works fine, so I'd rather not change it. I'd like to write a simple servlet that keeps the TCP connection alive and feeds messages back and forth between the TCP server and the GWT client.
Sockets have lots of restrictions in GAE due to security reasons, though GAE provides many ready to use services which requires use of sockets like for email, xmpp etc.
First your app needs to be paid app, means billing must be enabled for using sockets on GAE
You can have a look here for how to use sockets in java on GAE
Below is the excerpt from google app engine documentation for socket limitations:
App Engine supports sockets without requiring you to import any
special App Engine libraries or add any special App Engine code.
However, there are certain limitations and behaviors you need to be
aware of when using sockets:
Sockets are available only for paid apps.
You cannot create a listen socket; you can only create outbound sockets.
java.net.URL is still configured to use the URL Fetch API; there is currently no way around this.
Most classes in javax.net.ssl are supported.
You can only use TCP or UDP; arbitrary protocols are not allowed.
You cannot bind to specific IP addresses or ports.
Port 25 (SMTP) is blocked; you can still use authenticated SMTP on the submission port 587.
Private, broadcast, multicast, and Google IP ranges (except those whitelisted below), are blocked:
Google Public DNS: 8.8.8.8, 8.8.4.4, 2001:4860:4860::8888, 2001:4860:4860::8844 port 53
Gmail SMTPS: smtp.gmail.com port 465 and 587
Gmail POP3S: pop.gmail.com port 995
Gmail IMAPS: imap.gmail.com port 993
Socket descriptors are associated with the App Engine app that created them and are non-transferable (cannot be used by other apps).
Sockets may be reclaimed after 2 minutes of inactivity; any socket operation keeps the socket alive for a further 2 minutes.
You cannot Select between multiple available sockets because that requires java.nio.SocketChannel which is not currently supported.)
Related
I want to connect a client React app to the mosquitto test server. The "meat" React code involved:
import mqtt from 'mqtt';
const mqttConnect = ("mqtt://192.168.1.157",{clientId:"mqttjs01"}) => {
setConnectStatus('Connecting');
console.log(mqttOptions)
setClient(mqtt.connect(host, mqttOptions));
};
The error on the web page is:
ws.js:108 WebSocket connection to 'ws://test.mosquitto.org/' failed:
so one challenge I see is the insistence of a web socket. (ws protocol). Then another challenge is my ultimate goal is to talk to a mosquitto broker running on a Raspberry Pi (port 1883, which I thought the mqqt:// was addressing). The Raspberry Pi is not using websockets.
It appears clear to me that I am clueless. Any guidance / pointing in the right direction...greatly appreciated. Thank you.
With React you have to use WebSockets (because it effectively runs in the broswers) so your URL should start with ws:// not mqtt://.
Secondly you need to specify a port because there is no default port for MQTT over Websockets (and the WebSockets library will default to port 80)
So having checked the table of port numbers on test.mosquitto.org you should be using:
ws://test.mosquitto.org:8080/
p.s your client id's also probably need to have a lot more entropy (be random) otherwise you'll only ever have 1 connected client.
As for your local setup, you will need to configure mosquitto to listen on 2 different ports:
One for normal MQTT (default 1883)
One for MQTT over WebSockets (any port you want)
e.g. a conf file something like:
allow_anonymous true
listener 1883
listener 9001
protocol websockets
Mosquitto will share the same topic space between both listeners.
This documentation page says that:
Applications in the Java 8 runtime can use native Java sockets with no restrictions
However a socket to port 25 get blocked somewhere. The code bellow works fine on my local JVM but not in Google app engine Java8 standard env.
Socket skt = new Socket("gmail.com", 25);
BufferedReader rdr = new BufferedReader(new InputStreamReader(skt.getInputStream()));
I have been going through the docs for days now, but cannot find anything explaining why its not working. Nothing on my logs neither.
Can I open a socket to port 25 on GAE or not? please help
Port 25 is always blocked for mail related security reasons.
In the Java sockets documentation that you shared is stated that:
Port 25 (SMTP) is blocked; you can still use authenticated SMTP on the submission port 587.
It's true that the documentation is somewhat misleading about if this port limitation applies only to Java 7 runtime or affects Java 8 too. I'll send feedback about this documentation page to prevent future confusion.
we wrote in C++ a screen sharing application based on sending screenshots.
It works by establishing a TCP connection btw the server and client, where the server forwards every new screenshot received for a user through the connection, and this is popped-up by the client.
Now, we are trying to host this on google app engine, and therefore need 'servlet'-ize and 'sandbox' the server code, so to implement this forwarding through HTTP requests.
I immagine the following:
1. Post request with the screenshot as multiple-data form (apache uploads ..).
But now the server needs to contact the specified client (who is logged in) to send it/forward the screenshot.
I'm not sure how to 'initiate' such connection from the servlet to the client. The client doesn't run any servlet environment (of course).
I know HTTP 1.1 mantains a TCP connection, but it seems gapps won't let me use it.
1 approaches that comes to mind is to send a CONTINUE 100 to every logged in user at login, and respond with the screenshot once it arrives. Upon receival the client makes another request, and so on.
an alternative (insipired from setting the refresh header for a browser) would be to have the app pool on a regular basis (every 5 secs).
You're not going to be able to do this effectively on GAE.
Problem 1: All output is buffered until your handler returns.
Problem 2: Quotas & Limits:
Some features impose limits unrelated
to quotas to protect the stability of
the system. For example, when an
application is called to serve a web
request, it must issue a response
within 30 seconds. If the application
takes too long, the process is
terminated and the server returns an
error code to the user. The request
timeout is dynamic, and may be
shortened if a request handler reaches
its timeout frequently to conserve
resources.
Comet support is on the product roadmap, but to me your app still seems like a poor fit for a GAE application.
Long Polling is the concept used for such asynchronous communications between server and client.
In Long Polling, servlet keeps a map of client and associated messages. Key of Map being client id and value being list of messages to be sent to the client. When a client opens a connection with server (sends request to a servlet), the servlet checks the Map if there are any messages to be sent to it. If found, it sends the messages to the client exits from the method. On receiving messages, the client opens a new connection to the server. If the servlet does not find any messages for given client, it waits till the Map gets updated with messages for given client.
This is a late reply, I'm aware, but I believe that Google have an answer for this requirement: the Channel API.
Is there a way to open a TCP Socket back to a non-standard Silverlight port such as port 80?
I don't quite understand the restrictions on Silverlight ports.
I would like to open a connection back to the server of origin using any port. I can serve a policy file from port 943 if needed.
Microsoft restricted the ports to a range well outside the "well known ports." This prevents Silverlight from communicating directly to most web-based resources like HTTP, POP, SMTP, etc.
The most common way to get around this is to set up a "proxy" service on your domain. The proxy accepts requests, forwards them through the port on the service you're using, and returns the result.
The allowed port range (after the policy server check) is 4502 through 4532 to my knowledge. Using port 80 would be an HttpWebRequest or the like.
I've read that Silverlight 2.0 imposes by design an asynchronous model when communicating with the web server. I haven't had a chance to experiment with Silverlight, but I assume that it uses a thread-pool to manage threads like in the .NET Framework.
Now, since some browsers, most notably Internet Explorer, have an hard-coded limit of maximum two concurrent HTTP connections that can be made on the web server, what happens if I make a bunch of asynchronous requests from Silverlight?
Does Silverlight bypass this limitation in the web browser and open as many HTTP connections as there are threads available, or do the asynchronous requests queue up and wait for one of the two connections to become available?
In IE (haven't tested others) Silverlight is restricted to 2 connections at a time.
The behavior in Silverlight is to simply not make the request. So if you make 5 Async web service requests right in a row, the first 2 will happen, the other three won't. No exception is thrown that i've seen...
Fiddler is a big help here :)
Create a messaging manager interface for your client. Any outgoing request are posted to a queue that this manager processes against. It would serially process queued messages (i.e., when the call back of the last message sent to the server is invoked, can then safely proceed to process the next queued message).
You can consume the other connection resource by keeping a Comet connection open to the server. The server would push any return messages to the client via this Comet connection. You'll need to stamp out-going messages with a unique number that can be embedded as a property on in-coming messages - so that results can be correlated to request. The messaging manager would dispatch a result message to the appropriate handler for that result.
Essentially you end up using two connection resources to establish bi-directional messaging. But there is no artificial limit on the number of requesters on the client (though request will get serially transmitted to the server). The act of sending is always fast, though, because you don't wait for any result to be computed - you just need to deliver the message reliably to the server and return. Results come back asynchronously on the other Comet connection.
We do something along these lines with our Flex client apps in conjunction to Adobe BlazeDS running in our Tomcat web server:
A Flex-based asynchronous stack
Firefox is also limited to two connections, in addition to IE as stated already.
Note that the limit is per hostname.
If you add entries to your hosts file, or use dns aliases you can get more connections. For example in testing, add lines like '127.0.0.1 test1' to your hosts file, and then you can open two connections to http://localhost and two more to http://test1
I guess, being a .NET application Silverlight 2 has an independent from browser limit.
I would assume It is maxconnection attribute in Machine.config as mentioned in http://support.microsoft.com/kb/828219
Firstly the Machine.config file would not be used as the Silverlight control is sandboxed with its own version of the CoreCLR.
I believe that the Silverlight control actually makes use of the underlying browser to make the asynchronous HTTP requests. This is most likely the case considering how the Silverlight control can't gain access to SOAP fault information as the SOAP specification requires that the server returns an HTTP 500 response code and the Silverlight control doesn't get that from the browser hosting the control.
This post here serves to confirm this.
As to the limit of concurrent HTTP connections, I believe IE5 and later limit the number of connections to the same site based on HTTP protocol version - HTTP/1.0 it limits to 4 connections and HTTP/1.1 to 3 connections. Most of the time the web server will limit the number of connections to 2 per client, queueing or discarding the remainder.