I am trying to pull my reports from YouTube reporting API and upload them into GCS. Since I must use oauth2, I used Google's guide on creating a service account and giving it domain-wide authority. While I can get authenticated service to pull the reports at user level. I get "The caller does not have permission" when appengine runs the request. Any idea where to look next?
SCOPES = ['https://www.googleapis.com/auth/yt-analytics-monetary.readonly',
'https://www.googleapis.com/auth/youtubepartner-content-owner-readonly',
'https://www.googleapis.com/auth/devstorage.read_write']
SERVICE_ACCOUNT_FILE = 'file.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
delegated_credentials = credentials.with_subject('email#email.com')
storage_client = build('storage', 'v1', credentials=credentials)
youtube_reporting = build('youtubereporting', 'v1',
credentials=credentials)
Traceback:
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27/b93f87cfaa0115e5_unzipped/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/b93f87cfaa0115e5_unzipped/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/b93f87cfaa0115e5_unzipped/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/b93f87cfaa0115e5_unzipped/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/alloc/tmpfs/dynamic_runtimes/python27/b93f87cfaa0115e5_unzipped/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/b93f87cfaa0115e5_unzipped/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~test-project-67412/20180712t103610.411083715789214207/main.py", line 66, in get
jobs = list_jobs(youtube_reporting, ContentOwner, True)
File "/base/data/home/apps/s~test-project-67412/20180712t103610.411083715789214207/main.py", line 52, in list_jobs
includeSystemManaged=True).execute()
File "/base/data/home/apps/s~test-project-67412/20180712t103610.411083715789214207/lib/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/base/data/home/apps/s~test-project-67412/20180712t103610.411083715789214207/lib/googleapiclient/http.py", line 840, in execute
raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 403 when requesting https://youtubereporting.googleapis.com/v1/jobs?onBehalfOfContentOwner=ContentOwner&alt=json&includeSystemManaged=true returned "The caller does not have permission">
--UPDATE--
So I guess it had to do with Domain-wide authority for service account. I deleted the account, created a new one, gave it domain-wide authority, and it works just fine now.
Related
I have an older (Python 2.7) Google App Engine app that I recently deployed using Google Cloud Platform (gcloud), since appcfg was retired. I'm having issues using the images.get_serving_url not working for uploaded image blobs. The Images API is returning 'TransformationError'. This code worked previously for years in production before the switch to google cloud platform.
One quick thing to note is that Google Cloud Platform created a default storage bucket for blobs. Previously, I think GAE was using the Blobstore, so even though this change is transparent to developers, probably a lot has happened under the hood. I think for some reason my app is having issues creating the image url for the file stored on the cloud storage bucket.
I made a very simple test case to replicate the issue. Are there new requirements, or should this work?
class testUpload(webapp.RequestHandler):
def get(self):
uploadUrl = blobstore.create_upload_url('/testUploadHandler')
print("Generated upload url: %s" % uploadUrl)
self.response.out.write("""
<html><body>
<form action="{0}" method="POST" enctype="multipart/form-data">
Upload File: <input type="file" name="file"><br>
<input type="submit" name="submit" value="Submit">
</form>
</body></html>""".format(uploadUrl))
class testUploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
logging.info("testUploadHandler called.")
upload = self.get_uploads()[0]
blob_key = upload.key()
image_url = images.get_serving_url(blob_key)
self.redirect(image_url)
The deployed code returns the following error on the web server (local dev server is fine):
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~puzzleflow-dev/20210325t013659.433969250928848570/puzzleflow.py", line 4171, in post
image_url = images.get_serving_url(blob_key)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/1/google/appengine/api/images/__init__.py", line 1866, in get_serving_url
return rpc.get_result()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 615, in get_result
return self.__get_result_hook(self)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/fdc6d631da52d25b/python27/python27_lib/versions/1/google/appengine/api/images/__init__.py", line 1970, in get_serving_url_hook
raise _ToImagesError(e, readable_blob_key)
TransformationError
Also, you can try this yourself by going to this link (it should be publicly available):
https://puzzleflow-dev.appspot.com/testUpload
The issue was that I had switched my access control to Uniform, since Google says this is the 'recommended' setting. I'm not well-versed enough on the differences between Fine-Grained and Uniform, but switching it back to Fine-Grained fixed the problem.
I am trying to run django in appengine locally, and I get this erro - AssertionError: No api proxy found for service "logservice"
What does this mean? Am I missing something in the setup?
I run it by the following command:
PYTHONPATH=/lib/python2.7/site-packages/ dev_appserver.py app.yaml
Full stacktrace -
Traceback (most recent call last):
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 1302, in communicate
req.respond()
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 831, in respond
self.server.gateway(self).respond()
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/lib/cherrypy/cherrypy/wsgiserver/wsgiserver2.py", line 2115, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/wsgi_server.py", line 287, in __call__
return app(environ, start_response)
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/request_rewriter.py", line 314, in _rewriter_middleware
response_body = iter(application(environ, wrapped_start_response))
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/request_handler.py", line 165, in __call__
self._flush_logs(response.get('logs', []))
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/request_handler.py", line 303, in _flush_logs
apiproxy_stub_map.MakeSyncCall('logservice', 'Flush', request, response)
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 95, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/Users/Dev/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 322, in MakeSyncCall
assert stub, 'No api proxy found for service "%s"' % service
AssertionError: No api proxy found for service "logservice"
I'm tyring to use remote_api_shell.py to access the datastore of my Google App Engine application in the following way:
remote_api_shell.py -s url/of/server --secure
But I get the following error:
Traceback (most recent call last):
File "./remote_api_shell.py", line 133, in <module>
run_file(__file__, globals())
File "./remote_api_shell.py", line 129, in run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 160, in <module>
main(sys.argv)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 156, in main
oauth2=True)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 74, in remote_api_shell
secure=secure)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 768, in ConfigureRemoteApiForOAuth
rpc_server_factory=rpc_server_factory)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 835, in ConfigureRemoteApi
app_id = GetRemoteAppIdFromServer(server, path, rtok)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 569, in GetRemoteAppIdFromServer
response = server.Send(path, payload=None, **urlargs)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 245, in Send
url, method=method, body=payload, headers=headers)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/oauth2client/oauth2client/client.py", line 562, in new_request
redirections, connection_type)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1584, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1332, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1268, in _conn_request
conn.connect()
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1014, in connect
self.disable_ssl_certificate_validation, self.ca_certs)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 80, in _ssl_wrap_socket
cert_reqs=cert_reqs, ca_certs=ca_certs)
File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 243, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 8] _ssl.c:510: EOF occurred in violation of protocol
Any help would be appreciated.
The problem went away when I switched from using my custom domain name to my-app-id.appspot.com.
Some other considerations:
If you are using https, then your-app-id.appspot.com will not work out of the box. You have to enable it on the Google Apps page corresponding to the domain name. Yes, that means that you will actually have to have Google Apps setup for that domain.
If you have a multi-module setup, make sure you add remote_api: on to the builtins section of the app.yaml of the default module. Or if you have remote-api: on for another module, then use the url that skips the dispatch.yaml file and targets the module directly.
Below is a fragment of my code, i tried passing an instance variable from the class DiscussPage to anothe class quesAnsweredPage
class DiscussPage(webapp2.RequestHandler):
def post(self):
self.Ques_tit = "Content"
classname = questAnsweredPage()
classname.setValue(self.Ques_tit,self.Ques_tit)
class questAnsweredPage(webapp2.RequestHandler):
Can_I = "yes"
def setValue(self,que,title):
self.response.write(que)
The Code above returns the error:
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "B:\Eclipse World\Google App Engine\App Engine Dev\Jambite_Extended\JambitesCurrent\Jambitesupdated by bel\main.py", line 2686, in post
classname.setValue(self.Ques_tit,self.Ques_tit)
File "B:\Eclipse World\Google App Engine\App Engine Dev\Jambite_Extended\JambitesCurrent\Jambitesupdated by bel\main.py", line 2774, in setValue
self.response.write(que)
AttributeError: 'NoneType' object has no attribute 'write'
How do i resolve this error
You need to look at the signature of classes based on webapp2.RequestHandler ( specifically the init method)
From the docs http://webapp-improved.appspot.com/api/webapp2.html#webapp2.RequestHandler we get
class webapp2.RequestHandler(request=None, response=None)[source]
So when you instantiate an instance of a class inheriting from RequestHandler you need to initialise it correctly.
You code should read
classname = questAnsweredPage(self.request,self.response)
Now why you would use a variable name classname to hold an instance of a class is a whole separate style issue ;-)
The error is probably because self.response can only be called inside a get() or post() method.
I am trying to use this script for federated login with google's app engine. This works on the deployed app engine but with localhost it gives this error:
Traceback (most recent call last):
File "/workspace/python/app_en/google_appengine/google/appengine/ext/webapp/__init__.py", line 515, in __call__
handler.get(*groups)
File "/workspace/python/app_en/bingousie/helloworld.py", line 23, in get
p['login_url'] = users.create_login_url(federated_identity=p_url)
File "/workspace/python/app_en/google_appengine/google/appengine/api/users.py", line 216, in create_login_url
apiproxy_stub_map.MakeSyncCall('user', 'CreateLoginURL', req, resp)
File "/workspace/python/app_en/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 86, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/workspace/python/app_en/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 286, in MakeSyncCall
rpc.CheckSuccess()
File "/workspace/python/app_en/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl
self.request, self.response)
File "/workspace/python/app_en/google_appengine/google/appengine/api/apiproxy_stub.py", line 73, in MakeSyncCall
if request.ByteSize() > self.__max_request_size:
File "/workspace/python/app_en/google_appengine/google/appengine/api/user_service_pb.py", line 178, in ByteSize
n += self.lengthString(len(self.destination_url_))
TypeError: object of type 'NoneType' has no len()
Is this because the openid servers do not allow logging in to local site? If it is then it why doesn't it at least work with myopenid.com (I use it for my django openid authentication on local server).
In the create_login_url call you should specify the dest_url parameter.
dest_url can be full URL or a path
relative to your application's domain.
I had the same problem here.