getServingUrl occasionally returning relative URL in appengine development server - google-app-engine

I'm programmatically creating Blobs using the experimental files api. I then call getServingUrl() to get the URL of the image that I programmatically put in the blobstore.
Occasionally, this method returns a relative path like: /_ah/img/D4HVTcZm0wvYCcDNhV892A instead of 127.0.0.1/_ah/img/D4HVTcZm0wvYCcDNhV892A
This only happens occasionally and breaks some of my client code. Is this a known issue (quick googleing didnt find anything)? I'd prefer not to have any hacks to fix it (i.e. when on development server, prepend the host if it doesn't exist).
Any ideas?

Related

GraphQL/Apollo application with file download from server

I'm a little new to GraphQL and this question falls under "It cannot possibly be this hard. I have to be missing something."
I have a fairly standard GraphQL/Apollo/React application split into client and server. Everything is working well with the client making API calls and getting data back from the server. The client is even able to upload files to the server. However, I now need the server to stream back files saved on disk. That's it.
This is the "I have to be missing something" part. Everything I've seen in the docs and on Stackoverflow is some variation of pushing the file back from the server and through the GraphQL query as a base64-endocded string and then doing some very hacky stuff on the client, often involving a hidden href tag and a simulated click. To this I say, "What???"
Seriously. There are files on disk that the server knows how to find. The client needs to show a button to the user that they can click on to download the file. That's it. Every other framework in every other language has an easy way to do this. Can someone show me the incredibly simple thing that I'm missing here?
Thanks,
Alex
What you're missing is that GraphQL really shouldn't be used for this purpose.
While GraphQL itself does not specify a specific format for serializing responses, the de facto format is JSON. And the only way to get the file inside a JSON response is if it's serialized as a string.
If you want to serve static content, you should set up Nginx, Apache or another web server that's been built with that in mind. Alternatively, if you're already using some existing web server library like Express, it most likely has tools for serving static content as well.
Just because you have a GraphQL endpoint does not necessarily mean it should be the only way your client communicates with your backend.

Cefpython app with html/js files in local filesystem

I'm trying to make a hybrid python-js application with cefpython.
I would like to have:
JS and HTML files local to the cef python app (e.g. in './html', './js', etc)
Load one of the HTML files as the initial page
Avoid any CORS issues with files accessing each other (e.g. between directories)
The following seems to work to load the first page:
browser = cef.CreateBrowserSync(url='file:///html/index.html',
window_title="Rulr 2.0")
However, I then hit CORS issues.
Do I need to run a webserver also? Or is there an effective pattern for working with local files?
Try passing "disable-web-security" switch to cef.Initialize or set BrowserSettings.web_security_disabled.
Try also setting BrowserSettings.file_access_from_file_urls_allowed and BrowserSettings.universal_access_from_file_urls_allowed.
There are a few options in CEF for loading custom content and that can be used to load filesystem content without any security restrictions. There is a resource handler, a scheme handler and a resource manager. In CEF Python only resource handler is currently available. There is the wxpython-response.py example on README-Examples.md page.
Resource manager is a very easy API for loading various content, it is to be implemented in Issue #418 (PR is welcome):
https://github.com/cztomczak/cefpython/issues/418
For scheme handler see Issue #50:
https://github.com/cztomczak/cefpython/issues/50
Additionally there is also GetResourceResponseFilter in upstream CEF which is an easier option than resource handler, to be implemented via Issue #229:
https://github.com/cztomczak/cefpython/issues/229
You could also run an internal web server inside your app (easy to do with Python) and serve files that way. Upstream CEF also has a built-in web server functionality, however I don't think this will be exposed in cefpython, as it's already easy to set up web server in Python.

How to pass deployment settings to application?

I am trying to deploy a Qooxdoo web application backed by CherryPy-hosted web services onto a server. However, I need to configure the client-side Qooxdoo application with the hostname of the server on which the application resides so that that the Ajax callbacks resolve to the right host. I have a feeling I can use the capabilities of the generate.py Qooxdoo script to generate client-side code with this appropriately set, but reading through the docs hasn't helped make it clear how yet. Anyone have any tips?
(FWIW, I know how I'd approach this using something like PHP and a different client-side framework like Echo 3--I'd have the index file be a PHP file that reads a local system configuration file prior to sending back client-side code. In this case, however, the generate.py file is a necessary part of the toolchain, so I can't see how to do it so simply.)
You can use qx.core.Enviroment class to add/get configuration for your project. The recommend way is only during compilation time, but there is a hack if you want to configure your application during run time.
Configuration during compilation time
If you want to configure the environment during compilation time see this.
In both cases after you add any environmental variable to your application, it can be accessed using the qx.core.Environment.get method.
On run time
WARNING this method isn't supported/documented from qooxdoo. Basically it's a hack
If you want to make available some environment configuration on run time you have to do this before qooxdoo loads. In order to this you could add some javascript into your webpage e.g.
window.qx = { };
window.qx.$$environment = {
"myawsomeapp.hostname": "example.org",
};
This should be added somewhere in your page before the qooxdoo start loading otherwise it will not have the desirable effect. The advantage of this method is that you can push configuration to the client e.g. some api keys that may be different between instances of your application.
The easiest way will be to compose your AJAX URL on the fly from window.location; ideally, you would be able to use window.location.origin which for this StackOverflow website would be "https://stackoverflow.com" but there are issues with that on IE.
A cross platform solution is:
var urlRoot = window.location.protocol + "//" +
window.location.hostname + (window.location.port ? ':' +
window.location.port: '');
This means your URL will always be correct, even if the server name changes (eg your on a test server instead of production).
See here for more details:
https://tosbourn.com/a-fix-for-window-location-origin-in-internet-explorer/

Dealing with large zip uploads and extracting using google cloud

I am trying to create a site for e-learning courses (zips html/css/js/media) to be uploaded to.
I am using go on google app engine with google cloud storage to store the zips and extracted courses.
I will explain the development dead ends I have encountered.
My first thought was to use the resumable upload functionality of cloud storage to send the zip file, then read it using go on app engine, unzip the files and write them back to cloud storage.
This took a while to read and understand the documentation and worked perfectly for my 2MB test zip. It failed when I tried it with a modest 67MB zip. I had encountered a hidden limitation when accessing cloud storage from app engine. No matter the client I used there was a 10MB/32MB limit.
I tried both the old and new libraries as well as blobstore.
I also looked into creating a custom oauth2 supporting client library using sockets but hit too many dead ends.
Giving up on that approach I thought even though it would mean more uploading, perhaps just extracting on the client (browser) side then uploading each file with it's own resumable upload would make the most sense. After exploring a few libraries I had extracting in browser working ready to upload.
I wrote my handler that created the datastore entry for the upload, selected a location for the upload and created all the upload urls.
When testing this I was finding that it would take a while to go through generating the long lists of files (anything over 100). I decided that it would make sense since I was using to to make the requests concurrently. I spend a day or two getting that working. After dealing with some CORS issues that weirdly did not show up earlier I had everything working.
Then I started getting errors when stress testing my approach with a large (500mb) zip/course. The uploads would fail and I discovered that when trying to send 300+ files to generate upload urls I was getting the following error
Post http://localhost:62394: dial tcp [::1]:62394: connectex: No connection could be made because the target machine actively refused it.
now I have no idea how to diagnose this. I don't know if I am hitting a rate limit and if I am I don't know how to avoid it.
This seems like creating this should be simple, but it is anything but.
I have a few options I can pursue
Try to create the resumable uploads with a batch operation(https://cloud.google.com/storage/docs/json_api/v1/how-tos/batch)
batch operations to /upload are not supported.
Maybe make requesting each url a one by one api call.
Make requesting the url happen over a channel (https://cloud.google.com/appengine/docs/go/channel/reference)
spend the next week or more adding layers of retries and fallback error handling.
Try another solution.
This should be simple. How should this be done?

Prevent Silverlight xap from being cached by proxy server

The company I work for has proxies/WAN accelerators between our international sites to cache Intranet web content. I have a Silverlight application being hosted on a server at one location, but being accessed by clients in another location. When the users access the web page hosting the Silverlight app, they get the stale xap file being cached by the proxy and not the latest version from the server. Local users always get the latest xap as their requests are not going through a proxy.
I've tried the various header/metadata techniques mentioned elsewhere to prevent caching, and the containing web page itself is being served up fresh, but I still get the old .xap file. Short of getting our IT admin to disable proxy caching for my site, is there anything I can do make sure the latest xap file get retrieved from the server instead of the proxy? The containing page is ASP.NET.
What I do is just add a querystring at the end of the path to the xap file. Then when you change the querystring variable, the proxies etc. should see it as a request to a new file. So far this has worked fine for me.
So basically, when embedding an .xap in a straight-up HTML file, you would do this:
<param name="source" value="ClientBin/SilverlightApplication1.xap?cachepreventer=whatevervalue"/>
And then when you deploy a new version, just change "whatevervalue" to something else.
EDIT
If you need to use this technique in many places in your app I would read the querystring value from config and just write it to the page using asp.net. That way you only need to update it in one place when you deploy.
If you want to make sure every time the xap file is retrieved and you don't want to worry about it - just use
<param name="source" value="ClientBin/YourSilverlightapp.xap?<%=Guid.NewGuid().ToString() %>"/>
of course - this lends itself to a heavier cache load. I do like the helper method above though if you only want changes to be propagated to the client.

Resources