I tried to pull the report from Double Click Search API and put it into Google Cloud Storage by using google app engine(Python).
The following code is used to pull the report and place in google cloud storage.
Code:
def download_files(service, report_id, report_fragment):
"""Generate and print sample report.
Args:
service: An authorized Doublelcicksearch service.
report_id: The ID DS has assigned to a report.
report_fragment: The 0-based index of the file fragment from the files array.
"""
print "Enter into download_files", report_id
filename="/awstogcs/DoubleClickSearch_Campaign"+report_id+"_"+report_fragment+".csv"
write_retry_params = _gcs.RetryParams(backoff_factor=1.1)
gcs_file=_gcs.open(filename, 'w', content_type='text/plain',retry_params=write_retry_params)
request = service.reports().getFile(reportId=report_id, reportFragment=report_fragment)
time.sleep(120)
gcs_file.write(request.execute())
gcs_file.close()
Not sure why getting this error message.
Error Message:
INFO 2017-06-26 13:23:37,417 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO 2017-06-26 13:23:37,710 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO 2017-06-26 13:23:37,976 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO 2017-06-26 13:23:38,157 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 308 -
INFO 2017-06-26 13:23:38,440 module.py:809] default: "PUT /_ah/gcs/awstogcs/DoubleClickSearch_CampaignAAAndQT-U9IDyFYX_0.csv?upload_id=encoded_gs_file%3AYXdzdG9nY3MvRG91YmxlQ2xpY2tTZWFyY2hfQ2FtcGFpZ25BQUFuZFFULVU5SUR5RllYXzAuY3N2 HTTP/1.1" 200 -
Related
Below is the request that I did. Everything works except when I try the put /stats-batch which shows this error. I don't know what is the options that is being displayed, I only noticed it today.
The /stats-batch also just prints hello when access
27.0.0.1 - - [25/May/2021 16:20:46] "OPTIONS /user HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:20:46] "POST /user HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:20:48] "OPTIONS /product HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:20:48] "GET /product HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:20:48] "OPTIONS /new-batch HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:20:48] "GET /new-batch HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:21:21] "OPTIONS /new-batch HTTP/1.1" 200 -
oks
127.0.0.1 - - [25/May/2021 16:21:21] "PUT /new-batch HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:21:21] "GET /new-batch HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:21:43] "OPTIONS /stats-batch HTTP/1.1" 200 -
127.0.0.1 - - [25/May/2021 16:21:44] "PUT /stats-batch HTTP/1.1" 500 -
Traceback (most recent call last):
File "C:\Users\USER\Desktop\Allen\santeh\env\Lib\site-packages\flask\app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
.......
.......
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\Lib\json\encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type function is not JSON serializable
I call this functions with a button click using axios like this. config has the authorization header just like all the other request
const startBatch = async () => {
const status = 'start'
await axios
.put('http://127.0.0.1:5000/stats-batch', status, config)
.then((res) => {
console.log(res)
})
.catch((err) => {
console.log(err)
})
}
The console shows the error that No Access-Control-Allow-Origin header is present on the requested resource.
I also have flask_cors installed and the other axios request is working. I don't know what cause this error if its the backend or the frontend.
I see an issue; "/new-batch" endpoint is throwing 500 exceptions, which could be related to "Object of type function is not JSON serializable".
Based on the description / Error "No Access-Control-Allow-Origin header is present on the requested resource" message what I understand is
CORS configuration has been missed for PUT VERB.
Because you wrote that rest of the endpoints work without issues, only PUT on this specific "/new-batch" endpoint fails.
Options: In CORS, a preflight request is sent with the OPTIONS method so that the server can respond if it is acceptable to send the request.
URL: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS
Hi I am trying to do SSE(Server-Sent Events) with python flask quite similar to this question
I am using SSE to plot a real-time graph in the web app.
My codes are working fine when applied in local, but when I deploy it in GAE(Google App Engine), the data does not seem to be returned(yield)
I found that I should make the response header as following
X-Accel-Buffering: no
in this guide here
So, I tried this code on "main.py"
# render "plot.html" and plot the real-time graph
# plot.html gets the value from /chart-data event stream, and update the graph
#app.route('/plot_graph', methods=["GET", "POST"])
def plot_graph():
return render_template('plot.html')
#app.route('/chart-data')
def chart_data():
def generate_random_data():
while True:
# generating random data
json_data = json.dumps(
{'time': datetime.now().strftime('%Y-%m-%d %H:%M:%S'), 'value': random() * 100})
yield f"data:{json_data}\n\n"
time.sleep(0.5)
resp = Response(generate_random_data(), mimetype='text/event-stream')
resp.headers["X-Accel-Buffering"] = "no"
return resp
Also, in "plot.html" file, it is getting values from /chart-data like this
const source = new EventSource("/chart-data");
source.onmessage = function (event) {
const data = JSON.parse(event.data);
...
}
and this also works well when executed in local machine, but not working on GAE..
for the logs and error messages, when I tried the code on local machine,
and went to the root dir, /plot_graph, /chart-data, the log looks like
127.0.0.1 - - [03/Jul/2020 14:14:19] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [03/Jul/2020 14:14:20] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [03/Jul/2020 14:14:23] "GET /plot_graph HTTP/1.1" 200 -
127.0.0.1 - - [03/Jul/2020 14:14:24] "GET /chart-data HTTP/1.1" 200 -
127.0.0.1 - - [03/Jul/2020 14:15:04] "GET /chart-data HTTP/1.1" 200 -
when I went into /plot_graph, I see a GET request from /chart-data and it works fine(/plot_graph shows a real-time graph). Also, when I go into /chart-data, I can see yielded values being displayed on the web.
In the GAE logs,
2020-07-03 05:22:23 default[20200703t141524] "GET / HTTP/1.1" 200
2020-07-03 05:22:23 default[20200703t141524] "GET /favicon.ico HTTP/1.1" 404
2020-07-03 05:22:38 default[20200703t141524] "GET /plot_graph HTTP/1.1" 200
for the GAE, even if I entered /plot_graph, the GET request does not seemed to be happening. and since it is not getting any values, the graph is not plotted(only the frame for the graph is displayed).
Also, I tried to go into /chart-data of GAE web server, but I could not enter and cannot see any GET request in the logs as I saw on the server on local machine.
Please can you help me with this problem?
I'm creating a database for a school project and doing the backend between it and the app.(I'm using Flask and SQLAlchemy for it)
So the problem is that i can't call a function User(entrances), the function User() is a construction function used to create and object ,well it's what it looks like for me at least. But the function itself works when i do it in the interpreter.
WHY can i do it in my interpreter and can't do it on the flask tiny web framework???
here's the github link:BackEndRepository
Before taking a look at the samples here's the output form the interpreter that works:
Interperter
here's the code for my route:
from app import app,db
from models import User
from flask import render_template,Flask,request,redirect,url_for
#app.route('/index')
def index():
return "Hello, World!"
#app.route('/teste')
def teste():
users = db.session.query(User).all()
return u"<br>".join([u"{0}: {1}".format(user.name, user.email) for user in users])
#app.route('/teste2')
def teste2():
teste=User(name='susan',email='susan#example.com')
#u = User(name=request.args.get('1'), email=request.args.get('2'))
return teste
And here is the code for my model:
from app import db
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(64), index=True, unique=True)
email = db.Column(db.String(120), index=True, unique=True)
def __init__(self,name,email):
self.name = name
self.email= email
def __repr__(self):
return '<User {0} {1}>'.format(self.name,self.email)
so i run in my terminal "flask run" and get the error in response:(theese is all the log i had after doing what i explained)
#Arthur:~/Public/ProjetoBackEnd(Original)$ flask run
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [21/Nov/2018 14:24:41] "GET / HTTP/1.1" 404 -
127.0.0.1 - - [21/Nov/2018 14:24:41] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [21/Nov/2018 14:24:46] "GET /index HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2018 14:24:46] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [21/Nov/2018 14:24:52] "GET /teste HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2018 14:24:53] "GET /favicon.ico HTTP/1.1" 404 -
[2018-11-21 14:24:57,004] ERROR in app: Exception on /teste2 [GET]
Traceback (most recent call last):
File "/home/arthur/.local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/home/arthur/.local/lib/python2.7/site-packages/flask/app.py", line 1816, in full_dispatch_request
return self.finalize_request(rv)
File "/home/arthur/.local/lib/python2.7/site-packages/flask/app.py", line 1831, in finalize_request
response = self.make_response(rv)
File "/home/arthur/.local/lib/python2.7/site-packages/flask/app.py", line 1982, in make_response
reraise(TypeError, new_error, sys.exc_info()[2])
File "/home/arthur/.local/lib/python2.7/site-packages/flask/app.py", line 1974, in make_response
rv = self.response_class.force_type(rv, request.environ)
File "/home/arthur/.local/lib/python2.7/site-packages/werkzeug/wrappers.py", line 921, in force_type
response = BaseResponse(*_run_wsgi_app(response, environ))
File "/home/arthur/.local/lib/python2.7/site-packages/werkzeug/test.py", line 923, in run_wsgi_app
app_rv = app(environ, start_response)
TypeError: 'User' object is not callable
The view function did not return a valid response. The return type must be a string, tuple, Response instance, or WSGI callable, but it was a User.
127.0.0.1 - - [21/Nov/2018 14:24:57] "GET /teste2 HTTP/1.1" 500 -
127.0.0.1 - - [21/Nov/2018 14:24:57] "GET /favicon.ico HTTP/1.1" 404 -
JUST highlighting the part that's bugging me out the most here:
TypeError: 'User' object is not callable
The problem is here:
#app.route('/teste2')
def teste2():
teste=User(name='susan',email='susan#example.com')
#u = User(name=request.args.get('1'), email=request.args.get('2'))
return teste
You are returning teste which is a User. That's not a valid return type here as the error tells you:
The return type must be a string, tuple, Response instance, or WSGI callable, but it was a User.
It's fine to return a string for example, so this should work:
#app.route('/teste2')
def teste2():
teste=User(name='susan',email='susan#example.com')
#u = User(name=request.args.get('1'), email=request.args.get('2'))
return teste.name
I have an application in which I would like to move some of the request handlers into separate files. I've reduced the problem to this simple app which demonstrates it.
If I browse the main page, e.g.:
http://localhost:12082/
I see
'Hello World...'
displayed
if I try to go to the init page, e.g.:
http://localhost:12082/init/
I see
'404 Not Found'
The resource could not be found.
and in the log I see this:
(init2 message is never logged, the helper method is never called.)
What am I doing wrong?
Thank you for any clues
log contents:
...
INFO 2014-01-15 20:58:23,384 admin_server.py:117] Starting admin server at: http://localhost:8005
INFO 2014-01-16 04:58:34,398 initter.py:6] init1
INFO 2014-01-16 04:58:34,398 initter.py:17] init3
INFO 2014-01-15 20:58:34,421 module.py:617] default: "GET / HTTP/1.1" 200 14
INFO 2014-01-15 20:58:34,473 module.py:617] default: "GET / HTTP/1.1" 200 14
INFO 2014-01-15 20:58:39,555 module.py:617] default: "GET /init/ HTTP/1.1" 404 154
app.yaml:
application: helloworld
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: helloworld.application
helloworld.py
import webapp2
import initter
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Hello World...')
application = webapp2.WSGIApplication([
('/', MainPage),
('/init', initter.LoadPeople),
], debug=True)
initter.py
import webapp2
import logging
logging.getLogger().setLevel(logging.DEBUG)
logging.info("init1")
class LoadPeople(webapp2.RequestHandler):
def get(self):
logging.info("init2")
names = [ 'Joe', 'Jill', 'George', 'John', 'Dave' ]
for name in names:
logging.debug("name is %s", str(name))
logging.info("init3")
http handler expressions could be tricky. There is a difference between '/init' and '/init/' (with an extra slash at the end). Try changing your WSGI app initialization to this:
application = webapp2.WSGIApplication([
('/', MainPage),
('/init*', initter.LoadPeople),
], debug=True)
This will also take care of any query-strings you provide. Example: "....localhost/init?someKey=someValue"
When I run dev_appserver.py locally I see messages like this:
INFO 2014-01-05 02:19 ... default: "GET / HTTP/1.1" 200 793
INFO 2014-01-05 02:19 ... default: "GET /static/css/style.css HTTP/1.1" 200 1253
INFO 2014-01-05 02:20 ... default: "GET / HTTP/1.1" 200 801
...
What do the numbers at the end of the lines represent? In this case 793, 1253, 801?
Following the Apache common log format convention, those numbers are the Content-Lengths of the respective responses.
If you're curious, look in the SDK in lib/cherrypy/cherrypy/_cplogging.py in access() (~line 207).