GAE/Java LocalChannelFailureException at development server - google-app-engine

I'm using Channel API (Java) with Google App Engine for my web application. I have implemented a Token-reusing-mechanism for not exceeding the Channel API Quotas that fast.
This means, that my implementation reuses an existing channel for a user that refreshes the page as long as the expiration time of the token received by the ChannelService.createChannel() call, is not over.
When refreshing my page I get the following exception (with x starting at 0 and increasing for every refresh). However, my page continues to work as intended. Is there a way to avoid the exception being thrown? Or can I just ignore the exception?
com.google.appengine.api.channel.dev.LocalChannelFailureException: Client connection with ID connection-x not found.
at com.google.appengine.api.channel.dev.Channel.getClientMessageQueue(Channel.java:79)
at com.google.appengine.api.channel.dev.ChannelManager.getNextClientMessage(ChannelManager.java:300)
at com.google.appengine.api.channel.dev.LocalChannelServlet.doGet(LocalChannelServlet.java:120)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
...
Im reusing tokens with the following classes:
When calling ChannelService.createChannel() I save the expiration date and the generated token in an Entity called "Channel"
public class Channel {
private String id;
private String token;
private Date expiration;
}
Then I have a ChannelService class that returns a valid Channel with its get() method. The channelDAO is a class that just uses a Map for storing Channels. So there is no database persistence, which would keep a token alive over a server restart.
public Channel get(String clientId) {
Calendar calendar = Calendar.getInstance();
Channel channel = channelDAO.get(clientId);
if (channel == null || calendar.getTime().after(channel.getExpiration())) {
com.google.appengine.api.channel.ChannelService channelService = ChannelServiceFactory
.getChannelService();
calendar.add(Calendar.MINUTE, CHANNEL_UPTIME);
String token = channelService.createChannel(player.toString(), CHANNEL_UPTIME);
channel = new Channel(clientId, token, calendar.getTime());
channelDAO.persist(channel);
}
return channel;
}

I fixed the problem by further investigations on the source of the exception. The Channel API works with polling requests that are executed every 500ms. I used Firefox's console to track these. Here is an example poll:
[20:40:15.978] GET http://localhost:8080/_ah/channel/dev?command=poll&channel=920a60f9b27ece1a1ba43d251fdacf2e-channel-eqt3xi-1385927324758-{clientId}&client=connection-2 [HTTP/1.1 200 OK 0ms]
In my question I stated, that the exception occurs on page reload, so the problem with this was: When the page is reloaded, something (I don't know what exactly, but i assume it has something to do with sockets getting closed and reopened on page refresh) happens which causes the client (last parameter of the GET request) to no longer be available. However, a new client is available: the client "connection-{i+1}". So when you enter the page initially, the client is "connection-0". After page refresh it is "connection-1". But as the old page used a delayed execution for the poll, a false request (still connection-0) is sent to the server, that, as a result, throws the Exception.
I fixed the problem by manually cancelling the delayed execution, when leaving the page with jQuery.
var channel = new goog.appengine.Channel('${channel.token}');
var socket = channel.open(handler);
$(window).on('beforeunload', function() {
clearTimeout(socket.pollingTimer_);
});

Your token re-use scheme should be carefully checked for bugs as that exception shouldn't occur each page reload.
There is a known issue after local server restarts but as stated it should only be only if the development server restarted.

I had the same issue using GWT and gwt-gae-channel. The solution would be something like:
Socket socket = channel.open(new SocketListener() {...});
Window.addWindowClosingHandler(new ClosingHandler() {
#Override
public void onWindowClosing(ClosingEvent event) {
socket.close();
}
});

Related

How to manually ack/nack a PubSub message in Camel Route

I am setting up a Camel Route with ackMode=NONE meaning acknowlegements are not done automatically. How do I explicitly acknowledge the message in the route?
In my Camel Route definition I've set ackMode to NONE. According to the documentation, I should be able to manually acknowledge the message downstream:
https://github.com/apache/camel/blob/master/components/camel-google-pubsub/src/main/docs/google-pubsub-component.adoc
"AUTO = exchange gets ack’ed/nack’ed on completion. NONE = downstream process has to ack/nack explicitly"
However I cannot figure out how to send the ack.
from("google-pubsub:<project>:<subscription>?concurrentConsumers=1&maxMessagesPerPoll=1&ackMode=NONE")
.bean("processingBean");
My PubSub subscription has an acknowledgement deadline of 10 seconds and so my message keeps getting re-sent every 10 seconds due to ackMode=NONE. This is as expected. However I cannot find a way to manually acknowledge the message once processing is complete and stop the re-deliveries.
I was able to dig through the Camel components and figure out how it is done. First I created a GooglePubSubConnectionFactory bean:
#Bean
public GooglePubsubConnectionFactory googlePubsubConnectionFactory() {
GooglePubsubConnectionFactory connectionFactory = new GooglePubsubConnectionFactory();
connectionFactory.setCredentialsFileLocation(pubsubKey);
return connectionFactory;
}
Then I was able to reference the ack id of the message from the header:
#Header(GooglePubsubConstants.ACK_ID) String ackId
Then I used the following code to acknowledge the message:
List<String > ackIdList = new ArrayList<>();
ackIdList.add(ackId);
AcknowledgeRequest ackRequest = new AcknowledgeRequest().setAckIds(ackIdList);
Pubsub pubsub = googlePubsubConnectionFactory.getDefaultClient();
pubsub.projects().subscriptions().acknowledge("projects/<my project>/subscriptions/<my subscription>", ackRequest).execute();
I think it is best if you look how the Camel component does it with ackMode=AUTO. Have a look at this class (method acknowledge)
But why do you want to do this extra work? Camel is your fried to simplify integration by abstracting away low level code.
So when you use ackMode=AUTO Camel automatically commits your successfully processed messages (when the message has successfully passed the whole route) and rolls back your not processable messages.

Pass byte array from WPF to WebApi

tl;dr What is the best way to pass binary data (up to 1MBish) from a WPF application to a WebAPI service method?
I'm currently trying to pass binary data from a WPF application to a WebAPI web service, with variable results. Small files (< 100k) generally work fine, but any larger and the odds of success reduce.
A standard OpenFileDialog, and then File.ReadAllBytes pass the byte[] parameter into the client method in WPF. This always succeeds, and I then post the data to WebAPI via a PostAsync call and a ByteArrayContent parameter.
Is this the correct way to do this? I started off with a PostJSONAsync call, and passed the byte[] into that, but thought the ByteArrayContent seemed more appropriate, but neither work reliably.
Client Method in WPF
public static async Task<bool> UploadFirmwareMCU(int productTestId, byte[] mcuFirmware)
{
string url = string.Format("productTest/{0}/mcuFirmware", productTestId);
ByteArrayContent bytesContent = new ByteArrayContent(mcuFirmware);
HttpResponseMessage response = await GetClient().PostAsync(url, bytesContent);
....
}
WebAPI Method
[HttpPost]
[Route("api/productTest/{productTestId}/mcuFirmware")]
public async Task<bool> UploadMcuFirmware(int productTestId)
{
bool result = false;
try
{
Byte[] mcuFirmwareBytes = await Request.Content.ReadAsByteArrayAsync();
....
}
Web Config Settings
AFAIK these limits in web.config should be sufficient to allow 1MB files through to the service?
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="1073741824" />
</requestFiltering>
</security>
<httpRuntime targetFramework="4.5" maxRequestLength="2097152"/>
I receive errors in WebAPI when calling ReadAsByteArrayAsync(). These vary, possibly due to the app pool in IIS Express having crashed / getting into a bad state, but they include the following (None of which have lead to any promising leads via google):
Specified argument was out of the range of valid values. Parameter name: offset
at System.Web.HttpInputStream.Seek(Int64 offset, SeekOrigin origin)\r\n
at System.Web.HttpInputStream.set_Position(Int64 value)\r\n at System.Web.Http.WebHost.SeekableBufferedRequestStream.SwapToSeekableStream()\r\n at System.Web.Http.WebHost.Seek
OR
Message = "An error occurred while communicating with the remote host. The error code is 0x800703E5."
InnerException = {"Overlapped I/O operation is in progress. (Exception from HRESULT: 0x800703E5)"}
at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)\r\n
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityCoreSync(Byte[] buffer, Int32 offset, Int32 size)\r\n
at System.Web.Hosting.IIS7WorkerRequ...
Initially I thought this was most likely down to IIS Express limitations (running on Windows 7 on my dev pc) but we've had the same issues on a staging server running Server 2012.
Any advice on how I might get this working would be great, or even just a basic example of uploading files to WebAPI from WPF would be great, as most of the code I've found out there relates to uploading files from multipart forms web pages.
Many thanks in advance for any help.
tl;dr It was a separate part of our code in the WebApi service that was causing it to go wrong, duh!
Ah, well, this is embarrassing.
It turns out our problem was down to a Request Logger class we'd registered in WebApiConfig.Register(HttpConfiguration config), and that I'd forgotten about.
It was reading the request content via async as StringContent, and then attempting to log it to the database in an ncarchar(max) field. This itself is probably OK, but I'm guessing all the weird problems started occurring when the LoggingHandler as well as the main WebApi controller, were both trying to access the Request content via async?
Removing the LoggingHandler fixed the problem immediately, and we're now able to upload files of up to 100MB without any problems. To fix it more permanently, I guess I rewrite of the LoggingHandler is required to set a limit on the maximum content size it tries to log / to ignore certain content types.
It's doubtful, but I hope this may be of use for someone one day!
public class LoggingHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
LogRequest(request);
return base.SendAsync(request, cancellationToken).ContinueWith(task =>
{
var response = task.Result;
// ToDo: Decide if/when we need to log responses
// LogResponse(response);
return response;
}, cancellationToken);
}
private void LogRequest(HttpRequestMessage request)
{
(request.Content ?? new StringContent("")).ReadAsStringAsync().ContinueWith(x =>
{
try
{
var callerId = CallerId(request);
var callerName = CallerName(request);
// Log request
LogEntry logEntry = new LogEntry
{
TimeStamp = DateTime.Now,
HttpVerb = request.Method.ToString(),
Uri = request.RequestUri.ToString(),
CorrelationId = request.GetCorrelationId(),
CallerId = callerId,
CallerName = callerName,
Controller = ControllerName(request),
Header = request.Headers.ToString(),
Body = x.Result
};
...........

java.sql.SQLRecoverableException: Connection is already in use

In my java code, I am processing huge amount of data. So I moved the code as servlet to Cron Job of App Engine. Some days it works fine. After the amount of the data increases, the cron job is not working and shows the following error message.
2012-09-26 04:18:40.627
'ServletName' 'MethodName': Inside SQLExceptionjava.sql.SQLRecoverableException:
Connection is already in use.
I 2012-09-26 04:18:40.741
This request caused a new process to be started for your application, and thus caused
your application code to be loaded for the first time. This request may thus take
longer and use more CPU than a typical request for your application.
W 2012-09-26 04:18:40.741
A problem was encountered with the process that handled this request, causing it to
exit. This is likely to cause a new process to be used for the next request to your
application. If you see this message frequently, you may be throwing exceptions during
the initialization of your application. (Error code 104)
How to handle this problem?
This exception is typical when a single connection is shared between multiple threads. This will in turn happen when your code does not follow the standard JDBC idiom of acquiring and closing the DB resources in the shortest possible scope in the very same try-finally block like so:
public Entity find(Long id) throws SQLException {
Connection connection = null;
// ...
try {
connection = dataSource.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
return entity;
}
Your comment on the question,
#TejasArjun i used connection pooling with servlet Init() method.
doesn't give me the impression that you're doing it the right way. This suggests that you're obtaining a DB connection in servlet's init() method and reusing the same one across all HTTP requests in all HTTP sessions. This is absolutely not right. A servlet instance is created/initialized only once during webapp's startup and reused throughout the entire remaining of the application's lifetime. This at least confirms the exception you're facing.
Just rewrite your JDBC code according the standard try-finally idiom as demonstrated above and you should be all set.
See also:
Is it safe to use a static java.sql.Connection instance in a multithreaded system?

Windows Phone 7 Push Notifications Not Showing Up On My Phone

UPDATE: The plot thickens. I changed my channel name and it is suddenly working (which means it wasn't a problem with my push service, since I'm getting the same HTTP response from the Microsoft Push Notification server).
To me, however, this is not a solution. How will I be able to test this and KNOW my users are getting their push notifications if I'm getting the same response when it's not working as I do when it is?
[ORIGINAL POST]
I've been trying to get push notifications sent to my Windows Phone 7 device, but I'm having very big problems that I can't find any answers for. I'll start with the c# code.
I set up push notifications using the following C# code.
private HttpNotificationChannel channel;
private static string PUSH_CHANNEL = "MySpecialPushChannel";
private Uri PushUri = null;
private bool IsPushRegistered = false;
public void StartPushSubscription()
{
try
{
channel = HttpNotificationChannel.Find(PUSH_CHANNEL);
}
catch
{}
if (channel != null)
{
PushUri = channel.ChannelUri;
if (!channel.IsShellTileBound)
channel.BindToShellTile();
}
else
{
channel = new HttpNotificationChannel(PUSH_CHANNEL);
channel.ChannelUriUpdated += new EventHandler<NotificationChannelUriEventArgs>(channel_ChannelUriUpdated);
channel.HttpNotificationReceived += new EventHandler<HttpNotificationEventArgs>(channel_HttpNotificationReceived);
channel.ErrorOccurred += new EventHandler<NotificationChannelErrorEventArgs>(channel_ErrorOccurred);
try
{
channel.Open();
channel.BindToShellTile();
}
catch (Exception err)
{
channel = null;
IsPushRegistered = false;
// Code to try again
}
}
}
void channel_ChannelUriUpdated(object sender, NotificationChannelUriEventArgs e)
{
PushUri = e.ChannelUri;
IsPushRegistered = true;
}
I'm following the standard WP7 push structure:
Find the HttpNotificationChannel (or start a new one)
Register event handler to get the push notification uri back
Open the channel
Bind to the tile
Handle the channel Uri (which we send to our service to await the happy day when we send the push notification
OK... so far so good. No errors, I get my Uri, send it to my service just fine. I pin my app to the start screen and my service sends a push request to the Uri (sending just the count so that I get a little push count number in the upper right hand corner). I get back an HTTP 200 status with the following:
DeviceConnectionStatus => Connected
NotificationStatus => Received
SubscriptionStatus => Active
And then... nothing. No push status shows up on my app. I've now tried it on my device, in the emulator, on another device, and with multiple servers and the result is always the same. Everything looks like it is working except for the fact that it doesn't work.
To me, however, this is not a solution. How will I be able to test this and KNOW my users are getting their push notifications if I'm getting the same response when it's not working as I do when it is?
The answer is, you can't. It's a limitation of how WP7 handles notifications.
For structured notifications like Tile and Toast, if you get the Connected/Active/Received/200 response, then you can know that MPNS accepted your notification request. However, this does not mean that you have sent a valid XML payload.
The component that handles parsing XML is the Push Client, the process running on the phone that accepts push notifications and deals them out to appropriate applications, displays the toast, etc.
If you have sent invalid XML, there is absolutely no indication that you've done so. At most, if you try to send the notification again to that same push channel URI, you'll get a 404 in response. Apparently getting an invalid XML payload for a specific application makes that application's push channel close, requiring you to go through the whole procedure again.
I've discovered this while debugging with our server team, and through trying to get the phone to display an alternate live tile. The only advice I can offer you is to quadruple-check your XML.
You will get errors in your error event handler for your push notification channel for Toast notifications that have invalid XML, since you are able to send/receive toast notifications while the application is active.
If anyone from Microsoft is reading this, PLEASE provide more thorough documentation on possible error states in the push notification system. We also need an event handler for Tile notifications, or at least allow us to receive tile notifications while the app is in the foreground and fire the notification channel error event so that we can be aware that our XML payload is invalid.
Especially if your web service isn't built with WCF, .NET, Azure, and whatever, working with Push Notifications on WP7 is like wandering blind.
Documentation for an exception message reading "InvalidOperationException(Failed to open channel)" should not read: "This exception is raised when the notification channel failed to open. Try opening the notification channel again." (reference)
are you getting the URL from each device? you need to get a URL from the push notification sevice for each device everytime your device connects,
when it does you need to find a way of retrieving the url from each client,
once you do that and your still not receiving push notifications then I would write to microsoft to see if they can see anything to do with the push notifications

What happens if an application calls more than 10 asynchronous URL Fetch on Google App Engine?

Reading the Google App Engine documentation on asynchronous URL Fetch:
The app can have up to 10 simultaneous
asynchronous URL Fetch calls
What happens if an application calls more than 10 async fetch at a time?
Does Google App Engine raise an exception or simply queue the remain calls waiting to serve them?
Umm, Swizzec is incorrect. Easy enough to test:
rpc = []
for i in range(1,20):
rpc.append(urlfetch.createrpc())
urlfetch.make_fetch_call(rpc[-1],"http://stackoverflow.com/questions/3639855/what-happens-if-i-call-more-than-10-asynchronous-url-fetch")
for r in rpc:
response = r.get_result().status_code
This does not return any exceptions. In fact, this works just fine! Note that your results may vary for non-billable applications.
What Swizec is reporting is a different problem, related to maximum simultaneous connections INTO your application. For billable apps there is no practical limit here btw, it just scales out (subject to the 1000ms rule).
GAE has no way of knowing that your request handler will issue a blocking URL fetch, so the connection 500's he is seeing are not related to what his app is actually doing (that's an oversimplification btw, if your average request response time is > 1000ms your likelyhood of 500's increases).
This is an old question, but I believe the accepted answer is incorrect or outdated and may confuse people. It's been a couple of months that I actually tested this, but in my experience Swizec is quite right that GAE will not queue but rather fail most asynchronous URL fetches exceeding the limit of around 10 simultaneous ones per request.
See https://developers.google.com/appengine/docs/python/urlfetch/#Python_Making_requests and https://groups.google.com/forum/#!topic/google-appengine/EoYTmnDvg8U for a description of the limit.
David Underhill has come up with a URL Fetch Manager for Python, which queues asynchronous URL fetches that exceed the limit in application code.
I have implemented something similar for Java, which synchronously blocks (due to the lack of a callback function or ListenableFutures) additional requests:
/**
* A URLFetchService wrapper that ensures that only 10 simultaneous asynchronous fetch requests are scheduled. If the
* limit is reached, the fetchAsync operations will block until another request completes.
*/
public class BlockingURLFetchService implements URLFetchService {
private final static int MAX_SIMULTANEOUS_ASYNC_REQUESTS = 10;
private final URLFetchService urlFetchService = URLFetchServiceFactory.getURLFetchService();
private final Queue<Future<HTTPResponse>> activeFetches = new LinkedList<>();
#Override
public HTTPResponse fetch(URL url) throws IOException {
return urlFetchService.fetch(url);
}
#Override
public HTTPResponse fetch(HTTPRequest request) throws IOException {
return urlFetchService.fetch(request);
}
#Override
public Future<HTTPResponse> fetchAsync(URL url) {
block();
Future<HTTPResponse> future = urlFetchService.fetchAsync(url);
activeFetches.add(future);
return future;
}
#Override
public Future<HTTPResponse> fetchAsync(HTTPRequest request) {
block();
Future<HTTPResponse> future = urlFetchService.fetchAsync(request);
activeFetches.add(future);
return future;
}
private void block() {
while (activeFetches.size() >= MAX_SIMULTANEOUS_ASYNC_REQUESTS) {
// Max. simultaneous async requests reached; wait for one to complete
Iterator<Future<HTTPResponse>> it = activeFetches.iterator();
while (it.hasNext()) {
if (it.next().isDone()) {
it.remove();
break;
}
}
}
}
}
500 errors start happening. Silently.
You only find out about these when you look at your log under all requests (don't get listed as errors). It simply says "The request was aborted because you reached your simultaneous requests limit".
So when you're making lots of asynchronous calls, make sure you can handle some of them spazzing out.
See if this answers your question:
http://groups.google.com/group/google-appengine/browse_thread/thread/1286139a70ef83c5?fwc=1

Resources