I'm trying to implement wowza streaming engine with google cloud using this tutorial.
The problem I am having is that when i try to add the Wowza Streaming Engine image via the developer console it fails saying " Required 'read' permission for 'rawDisk.source' ". I have also tried adding the image via the command-line tool with no success.
From my research on this error, all I have been able to surmise is that maybe this particular image is no no longer publicly shared, but this seem highly unlikely.
So I tried exactly what you said, followed the same tutorial and got the same error. While the tutorial is somewhat out-dated (as certain commands are no longer the same and so on), the image they provide should still be fine. I wasn't able to create my image via the Web UI but it WORKED via CLI.
Here's what you can do as a workaround:
gcloud compute images create [IMAGE_NAME] --project=[YOUR_PROJECT] --source-uri http://storage.googleapis.com/wowzamediasystems/wse/WowzaStreamingEngine-4.1.0.v2.image.tar.gz
As long as you're properly authenticated (for example gcloud auth login) the above command should work. I tested, and was also able to create my instance using the image it creates. SSHed to it without a problem.
Related
My python application on the Google App Engine (standard environment) writes some logs to stdout.
I used to be able to see them on the Logs Viewer but not anymore.
The default options under the "Log name" drop down menu do not show stdout, and I have tried to manually add logName="projects/airlib-main/logs/stdout" into the Query Builder but it doesn't find anything.
I solved my problem by NOT using basicConfig to configure the logs and rather instantiate my own loggers and use them throughout the application. The python documentation says that < This function does nothing if the root logger already has handlers configured > and maybe the Google App Engine does that now. My application used to produce logs properly so there might have been a change recently in the way the Google App Engine works in this environment.
I would recommend you to take a look at the post from the Community:
How to group related request log entries GAE python 3.7 standard env. This post provides you with possible solutions to your stdout logging.
For example, as mentioned on this answer here - that it's in Python language - there are some methods that you can use to set the stdout as the pattern for logs. So, checking this post should assist you in configuring the use of stdout on your App Engine.
Let me know if the information helped you!
So it seems from a few SO questions I've seen that this is a problem among other users. Recently one of our head dev's left and I inherited a lot of his projects. One of which, is a website that what seems like lives on an app engine from google cloud platforms. From the App Engine documentation, to download source code you use the appcfg.py download_app command. Which I did, however the only results I get back from that call is:
Fetching file list...
Fetching files...
And then it just ends. No error message or any kind of message at all, and of course, it did not download the source code into the output dir I specified.
Scratching my head and looking at various SO posts, someone mentioned something about going into the google cloud vm directly and doing the same command, and to my surprise finding the same exact behavior that I did in my local terminal.
This made me realize it must be something else at play. I took a look at my versions tab in the App Engine dashboard on GCP. I see my instance running, it correctly says Serving and if I click the link it brings me to the website which loads fine. However, under Size it says 0 B which made me think perhaps this is why the download_app isn't downloading anything, because the version is 0 B?
What I'm trying to figure out is why it says 0 B for the version, when clearly the site runs fine and how I can get the source code for this. Here's a screenshot for reference
And screenshot of my terminal (local). Obviously I omitted the -A and -V flags, but they are correctly set and if I purposely make them incorrect I do indeed get an error message.
EDIT
Just so everyone is aware, I also made sure my user had the correct permissions. Owner, App Engine Owner... and some others. I don't think that's the problem.
When you deploy an App Engine Flexible application, the source code is uploaded to Cloud Storage on your project in a bucket named staging.<project-id>.appspot.com. You can navigate in this bucket and download the source code for a specific version as a .tar file.
Alternatively, you can find the exact Cloud Storage URL for your source code by going to Dev Console > Container Registry > Build History and select the build for your version. You'll find the link to your source code under Build Information.
One thing to note however is that the staging... bucket is created by default with a Lifecycle rule that deletes files older than 15 days automatically. You can delete this rule if you want so that all versions' source code is kept indefinitely.
In your case I believe that may not have helped since files may have been deleted already but it's worth knowing you can get the source code from there (source code isn't pushed to Source Repository by default, your developer had to configure it manually).
Posting this since none of the listed methods on the web didn't take me to the code (by June 2021)
Note: appcfg.py is deprecated by Google
You could try accessing your source code through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download code automatically but you can copy-paste the code.
Advice: Push your code to a Git repository to avoid this hassle next time.
Hope you will find this helpful.
In the developer console you can select the respective project and check:
on the Services page - which services, AKA modules - as they used to be (and still are) called in various places, you app has deployed
on the Versions page - which versions for each of the services are deployed
This information is what appcfg.py download_app expects. See also:
the various appcfg.py options using its --help flag
How do I download a specific service's source code off of AppEngine?
You can also access the deployed source code live (if everything else fails it could still be a last resort method to get the code, but tedious), see my answer to Google Cloud DataStore automatic indexing
Update:
I just now noticed in your screenshot that it's a flexible environment app. The appcfg.py docs are in the standard environment section, I suspect it's not applicable to the flexible environment, for which what's deployed is actually a docker image built during the deployment operation. From Deploying your application:
Deploy your app to App Engine using the gcloud app deploy
command. This command automatically builds a container image by using
the Container Builder service and then deploys that image to the
App Engine flexible environment. The container will include any local
modifications that you've made to the runtime image.
It might be possible to access the code on the actual GCE instance running the app, by connecting to the running instance and starting a shell in your app container, see Connecting to the instance
I recently wrote a tool that checks if certain requests on a web page are fired. For example is jquery from a google server loaded when I visit StackOverflow? The tool is based on webdriver and a chrome browser.
The tool works like a charm but it also generates google analytics traffic that should not be there.
Is there a possibility to tell Google that the tool is a bot and should not be counted? All solutions I found focus on the page under test and how to not collect data via Google. I need a solution that works without changing the website's code.
Possibly the easiest way would be to send a custom user agent string along and filter by that via a view filter.
Untested, but I think if you use the same user agent string as the Google crawler I'm pretty certain it will be filtered out automatically (after all Google should be smart enough to filter their own crawler). According to this page the Google crawler UA always contains the string "Googlebot" (else youD have to go through your server logs to find a likely UA string).
We solved our problem with a chrome extension. That extension redirects all requests against the google collect endpoint to example.com. Works like a charm.
Here is the Answer to your Question:
It sounds interesting to know about a tool which checks if certain requests on a web page are fired for e.g. jquery from a google server loaded.
Practically, there is a lot of difference when a page e.g. StackOverflow.com is opened Manually (through keying in the URL manually) and when chromedriver exe is used to initiate the new Google Chrome session.
As an example, manually when we open a new Google Chrome session, the command line parameters which gets passed along with chrome.exe are --flag-switches-begin and --flag-switches-end only.
But when we try to use chromedriver exe, we generally take help of ChromeOptions class and DesiredCapabilities class to add various config parameters to the Google Chrome session. Apart from all these one parameter is added i.e. --test-type=webdriver.
Hence any website when accessed by chromedriver exe is always aware that it's a Automated Bot.
Let me know if this Answers your Question.
I'm working on a project with Google App Engine. I am using continuous integration via Travis, and wish to be able to deploy directly from it. Due to a bug that will not be resolved directly, I can't rely on Travis' built-in GAE deployment, so I basically have to use mvn appengine:update manually. This requires me navigate to a generated URL and manually paste to the terminal an authentication code, which I can't do in automated builds.
It was suggested to me, however, that I do some Unix magic instead. While I can easily pick out the URL I need to navigate to from grep, I still need to log in to Google with my credentials in order to actually get the authentication code (which I can then grep out and pipe to the deployment program).
Given that, how do I log in to Google with my credentials, using only curl or similar command-line utilities?
I've accomplished similar things in the past using Service Accounts. These are likely a good fit for your problem.
Service Accounts will allow you to authenticate and upload your app without manual intervention.
Overview
A Service Account will allow you to do "passwordless" authentication like you may already do with ssh, and git, etc. by setting up your keys. This will remove the requirement that you log in manually, or follow the road to madness by trying to do a "manual" login automatically.
There are basically two steps:
Create your service account and key (with the right permissions)
Use that credential instead of what you're doing now
Resources
I think it's better to give a list of resources than concrete instructions since it's basically impossible to express concisely (even though it's a simple process, there's bound to be a lot of little things that annoy), everyone's requirements will be slightly different, and Google is likely to change the process at some point.
Using the Google Cloud Platform Console for App Engine | Permissions
Using OAuth 2.0 for Server to Server Applications
Setting up OAuth 2.0 | Service Accounts
gcloud auth activate-service-account
Hopefully that's enough to get you headed in the right direction.
Note
You'll likely have to spend some time looking at your .appcfg_oauth2_tokens_java and sorting out a variety of other annoyances, but I believe that this approach is the best way to solve your problem.
It sounds like you have a pretty straight-forward setup and that a Service Account alone will get you there, but if you need to get a little weird, the App Engine Admin API is always there.
Using Eclipse, I am experiencing an error when trying to deploy a rather basic web app with JAX-RS and JAXB. It runs okay locally, but when trying it on the remote servers I get the message shown below...
'Deploying to Google' has encountered a problem / This application does not exist
Below shows my appengine-web.xml
The XML file illustrates that I am using the same name in the xml as what's specified in the project properties...
The output window show...
------------ Deploying frontend ------------
Preparing to deploy:
Created staging directory at: '/var/folders/n8/6by626014jbfc0dwmxnb0ly00000gn/T/appcfg2754901216637807129.tmp'
Scanning for jsp files.
Scanning files on local disk.
Initiating update.
com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=hillingarincident&version=0&
404 Not Found
This application does not exist (app_id=u'hillingarincident').
Debugging information may be found in /private/var/folders/n8/6by626014jbfc0dwmxnb0ly00000gn/T/appengine-deploy447984481661870877.log
The referenced debug logs show...
Unable to update:
com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=hillingarincident&version=0&
404 Not Found
This application does not exist (app_id=u'hillingarincident').
at com.google.appengine.tools.admin.AbstractServerConnection.send1(AbstractServerConnection.java:293)
at com.google.appengine.tools.admin.AbstractServerConnection.send(AbstractServerConnection.java:253)
at com.google.appengine.tools.admin.AbstractServerConnection.post(AbstractServerConnection.java:232)
at com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.java:644)
at com.google.appengine.tools.admin.AppVersionUpload.beginTransaction(AppVersionUpload.java:449)
at com.google.appengine.tools.admin.AppVersionUpload.doUpload(AppVersionUpload.java:124)
at com.google.appengine.tools.admin.AppAdminImpl.doUpdate(AppAdminImpl.java:371)
at com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:53)
at com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy(AppEngineBridgeImpl.java:433)
at com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace(DeployProjectJob.java:148)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Any answers will be appreciated. At one point my browser was not logged in to the target google account, so I swapped to the correct one a little later, Google does render the application name as expected.
Okay, this was simple in the end! Eclipse performs an auto-login to the Google account, unfortunately I created the Eclipse project whilst being logged in to one Google account and then tried to specify the application name afterwards.
You'll see in the bottom-right (or bottom-left in some versions) a Google icon with the name of the user that you are logged in as. If that's not the account where your application is defined, then simply logout of that account, then login as the correct Google account.
Now there's no error :-)
I know this question is super old but I had this issue all day and finally I found a solution. Maybe it will help someone out in the future.
After you create a project in Google Cloud Platform, you must go to google cloud shell in your project and run the command
gcloud beta app create
After you run this command, you will get prompted to choose a region. Then go back to eclipse and try deploying it. It worked for me.
There are not just 1 way can cause this problem. For me, I have this problem when I create the project using Maven. But I don't have the same issue if I directly create the project from the Google plugin.
There might be another issue, when you register with Google App Engine, you receive email indicating your activation. If you have not received the email yet, this problem could occur too.
Another issue could be to use the gmail account for the Google App Engine to avoid any such errors.