here is the model in account.models.py
from django.conf import settings
from django.db import models
class Profile(models.Model):
user = models.OneToOneField(settings.AUTH_USER_MODEL,
on_delete=models.CASCADE,unique=True)
date_of_birth = models.DateField(blank=True, null=True)
photo = models.ImageField(upload_to='users/%Y/%m/%d/',
blank=True)
def __str__(self):
return f'Profile of {self.user.username}'
here is the output after i ran makemigrations and migrate:
(.venv) PS C:\Users\lakka\OneDrive - Manipal Academy of Higher
Education\miniproject\improved> python manage.py makemigrations
account Migrations for 'account': account\migrations\0001_initial.py
- Create model Profile (.venv) PS C:\Users\lakka\OneDrive - Manipal Academy of Higher Education\miniproject\improved> python
manage.py migrate account Operations to perform: Apply all
migrations: account Running migrations: No migrations to apply.
it is creating the model but it is not migrating
Related
I am trying to split traffic in my Python3 application using Flask. I create two different versions of the app using gcloud app deploy -v1 and gcloud app deploy -v2 and then split the traffic evenly using gcloud app services set-traffic myservice --splits=v1=0.5,v2=0.5 --split-by ip. I've made several requests to myservice but I only ever see the output from v2 of my app.
The GCP documentation here suggests that the issue might be related to caching and says: "For cacheable static resources that vary between versions, change the resource's URL between versions."
I cannot figure out how to change the target url to serve these two versions in different locations. I am not sure if this is something that needs to be done in the app.yaml file or my flask application?
For reference, here is my app.yaml file:
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
service: myservice
runtime: python39
env: standard
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 3
# This sample incurs costs to run on the App Engine flexible environment.
# The settings below are to reduce costs during testing and are not appropriate
# for production use. For more information, see:
# https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
Here is v1 of my app:
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# [START gae_flex_quickstart]
import logging
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello():
"""Return a friendly HTTP greeting."""
return 'Hello world!'
#app.errorhandler(500)
def server_error(e):
logging.exception('An error occurred during a request.')
return """
An internal error occurred: <pre>{}</pre>
See logs for full stacktrace.
""".format(e), 500
if __name__ == '__main__':
# This is used when running locally. Gunicorn is used to run the
# application on Google App Engine. See entrypoint in app.yaml.
app.run(host='127.0.0.1', port=8080, debug=True)
# [END gae_flex_quickstart]
and finally v2 of my app:
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# [START gae_flex_quickstart]
import logging
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello():
"""Return a friendly HTTP greeting."""
return 'Hello Universe!'
#app.errorhandler(500)
def server_error(e):
logging.exception('An error occurred during a request.')
return """
An internal error occurred: <pre>{}</pre>
See logs for full stacktrace.
""".format(e), 500
if __name__ == '__main__':
# This is used when running locally. Gunicorn is used to run the
# application on Google App Engine. See entrypoint in app.yaml.
app.run(host='127.0.0.1', port=8080, debug=True)
# [END gae_flex_quickstart]
I am following the docs on how to backup datastore using AppEngine.
I am performing a gcloud app deploy cron.yaml command on a GCE VM that is meant to update a cronjob in AppEngine. the GCE VM and AppEngine cron are in the same project, and I have granted AppEngine admin to the GCE VM via a default Service Account. When I run this on my local machine, it updates fine. However on the GCE instance, thats where issues arise
Here are the files
app.yaml
runtime: python27
api_version: 1
threadsafe: true
service: cloud-datastore-admin
libraries:
- name: webapp2
version: "latest"
handlers:
- url: /cloud-datastore-export
script: cloud_datastore_admin.app
login: admin
cron.yaml
cron:
- description: "Daily Cloud Datastore Export"
url: /cloud-datastore-export?namespace_id=&output_url_prefix=gs://<my-project-id>-bucket
target: cloud-datastore-admin
schedule: every 24 hours
cloud_datastore_export.yaml
import datetime
import httplib
import json
import logging
import webapp2
from google.appengine.api import app_identity
from google.appengine.api import urlfetch
class Export(webapp2.RequestHandler):
def get(self):
access_token, _ = app_identity.get_access_token(
'https://www.googleapis.com/auth/datastore')
app_id = app_identity.get_application_id()
timestamp = datetime.datetime.now().strftime('%Y%m%d-%H%M%S')
output_url_prefix = self.request.get('output_url_prefix')
assert output_url_prefix and output_url_prefix.startswith('gs://')
if '/' not in output_url_prefix[5:]:
# Only a bucket name has been provided - no prefix or trailing slash
output_url_prefix += '/' + timestamp
else:
output_url_prefix += timestamp
entity_filter = {
'kinds': self.request.get_all('kind'),
'namespace_ids': self.request.get_all('namespace_id')
}
request = {
'project_id': app_id,
'output_url_prefix': output_url_prefix,
'entity_filter': entity_filter
}
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + access_token
}
url = 'https://datastore.googleapis.com/v1/projects/%s:export' % app_id
try:
result = urlfetch.fetch(
url=url,
payload=json.dumps(request),
method=urlfetch.POST,
deadline=60,
headers=headers)
if result.status_code == httplib.OK:
logging.info(result.content)
elif result.status_code >= 500:
logging.error(result.content)
else:
logging.warning(result.content)
self.response.status_int = result.status_code
except urlfetch.Error:
logging.exception('Failed to initiate export.')
self.response.status_int = httplib.INTERNAL_SERVER_ERROR
app = webapp2.WSGIApplication(
[
('/cloud-datastore-export', Export),
], debug=True)
The Error I'm getting is
Configurations to update:
descriptor: [/usr/local/sbin/pluto/<my-project-id>/datastore/cron.yaml]
type: [cron jobs]
target project: [<my-project-id>]
Do you want to continue (Y/n)?
Updating config [cron]...
failed.
ERROR: (gcloud.app.deploy) Server responded with code [403]:
Forbidden Unexpected HTTP status 403.
You do not have permission to modify this app (app_id=u'e~<my-project-id>').
I have checked other posts related to this, however they seem to deal with an old version/deployment of appengine
Service Accounts!
From Deploying using IAM roles:
To grant a user account the ability to deploy to App Engine:
Click Add member to add the user account to the project and then select all of the roles for that account by using the dropdown menu:
Required roles to allow an account to deploy to App Engine:
a. Set the one of the following roles:
Use the App Engine > App Engine Deployer role to allow the account to deploy a version of an app.
To also allow the dos.yaml or dispatch.yaml files to be deployed with an app, use the App Engine > App Engine Admin role
instead.
The user account now has adequate permission to use the Admin API to deploy apps.
b. To allow use of App Engine tooling to deploy apps, you must also give the user account the Storage > Storage Admin role
so that the tooling has permission to upload to Cloud Storage.
Optional. Give the user account the following roles to grant permission for uploading additional configuration files:
Cloud Scheduler > Cloud Scheduler Admin role: Permissions for uploading cron.yaml files.
Potentially of interest:
Deployments with predefined roles
Predefined roles comparison matrix
Okay after some tinkering. I added the project editor role to the service account linked to the GCE instance running my server. I am not fully aware if this is the role with least priviledge to enable this to work.
I am developing a website to make my own portfolio. I wanted to switch to google cloud service. I got my app working and a database up. I am able to connect via ip but if a an other instance get's create the ip has to be whitelisted first.
SQLAlchemly url is what i saw was the anwser. But i want to keep working with peewee if possible.
Can someone explain to me why my code is not connect from the app engine to the cloud sql.
My code
"""Main script for the blog website."""
from flask import Flask
from peewee import *
from livereload import Server, shell
from flask_uploads import UploadSet, IMAGES, configure_uploads
from flask_login import LoginManager, login_user
from jinja2 import Environment
import os
from playhouse.db_url import connect
import pymysql
# db = connect(host='127.0.0.1', port=3306, user='root', password='fakepassword')
# db = pymysql.connect(host='127.0.0.1', port=3306, user='root', password='fakepassword')
try:
db = connect('sqlite:///root:fakepassword#/DATABASE?unix_socket=/cloudsql/austinbakkerblog:us-west1:database')
except Exception:
print('did not connect to database')
# 'mysql+mysqldb://%s:%s#/blog?unix_socket=/cloudsql/%s'
app = Flask(__name__)
app.config.from_pyfile('config.py', silent=False)
DEBUG = app.debug
# db = MySQLDatabase('database')
# db.connect()
# db = connect('mysql://root:fakepassword#127.0.0.1:3306/database')
# db = connect('mysql://root:fakepassword#/DATABASE?unix_socket=/cloudsql/austinbakkerblog:us-west1:database')
# db = connect('mysql://root:fakepassword#/cloudsql/austinbakkerblog:us-west1:database')
# db = connect('mysql://root:fakepassword#cloudsql/austinbakkerblog:us-west1:database')
# print(db.connect())
login_manager = LoginManager()
login_manager.init_app(app)
login_manager.login_view = 'login'
photos = UploadSet('photos', IMAGES)
configure_uploads(app, photos)
import models
from views import *
if __name__ == '__main__':
models.initialize()
print('=' * 100)
app.run()
If anyone is willing to help that would amazing, i spendt the last to day's trying to get it work but no success.
So after a few days of grinding i got it to work. Not sure exactly how but it
app.yaml
works. runtime: python
env: flex
entrypoint: gunicorn -b :$PORT app:app
runtime_config:
python_version: 3
env_variables:
SQL_USER: root
SQL_PASSWORD: test
SQL_DATABASE: database
INSTANCE_CONNECTION_NAME: austinbakkerblog:europe-west2:mydatabase1
beta_settings:
cloud_sql_instances: "austinbakkerblog:europe-west2:mydatabase1"
automatic_scaling:
min_num_instances: 1
max_num_instances: 2
cpu_utilization:
target_utilization: 0.5
app.py
"""Main script for the blog's website."""
from flask import Flask
from peewee import *
from livereload import Server, shell
from flask_uploads import UploadSet, IMAGES, configure_uploads
from flask_login import LoginManager, login_user
from jinja2 import Environment
import os
import pymysql
db = MySQLDatabase(host='127.0.0.1', user='root', password='test', unix_socket='/cloudsql/austinbakkerblog:europe-west2:mydatabase1', database='database')
print(db.connect())
print(db.close())
Rather than using a connection URL, it might be easier to debug if you just setup the database connection directly.
For example,
db = MySQLDatabase('db_name', user='root', password='dbpassword',
host='googlecloud.whatever.com', port=31337)
Can you get pymysql to connect?
I am writing a new application using App Engine and, as the docs suggest to not use Blobstore API, I'm using the Google Cloud Storage client (GCS). All is good but I want to be able to return "signed urls" to clients so they can get the GCS resources without passing through the application. I believe that is what signet urls are for.
But how to test that? I can sucessfully test GCS calls from the client, but I have no idea how to test the client's HTTP calls using urlfetch.
Below is a full test case that illustrates my issue:
import base64
import mimetypes
import urllib
import urllib2
from datetime import datetime, timedelta
import time
from google.appengine.api import app_identity
from google.appengine.datastore import datastore_stub_util
from google.appengine.ext import testbed
from google.appengine.ext import ndb
import unittest
import cloudstorage
# IS THIS RIGHT ?
GCS_API_ACCESS_ENDPOINT = 'http://localhost:8000/_ah/gcs'
def sign_url(bucket_object, expires_after_seconds=60):
""" cloudstorage signed url to download cloudstorage object without login
Docs : https://cloud.google.com/storage/docs/access-control?hl=bg#Signed-URLs
API : https://cloud.google.com/storage/docs/reference-methods?hl=bg#getobject
"""
# source: https://github.com/voscausa/appengine-gcs-signed-url/blob/05b8a93e2777679d40af62cc5ffce933216e6a85/sign_url.py
method = 'GET'
gcs_filename = urllib.quote(bucket_object)
content_md5, content_type = None, None
# expiration : number of seconds since epoch
expiration_dt = datetime.utcnow() + timedelta(seconds=expires_after_seconds)
expiration = int(time.mktime(expiration_dt.timetuple()))
# Generate the string to sign.
signature_string = '\n'.join([
method,
content_md5 or '',
content_type or '',
str(expiration),
gcs_filename])
signature_bytes = app_identity.sign_blob(signature_string)[1]
google_access_id = app_identity.get_service_account_name()
# Set the right query parameters. we use a gae service account for the id
query_params = {'GoogleAccessId': google_access_id,
'Expires': str(expiration),
'Signature': base64.b64encode(signature_bytes)}
# Return the built URL.
result = '{endpoint}{resource}?{querystring}'.format(endpoint=GCS_API_ACCESS_ENDPOINT,
resource=gcs_filename,
querystring=urllib.urlencode(query_params))
return result
FILE_DATA = "This is file contents."
MIME = "text/plain"
class TestGCS(unittest.TestCase):
def setUp(self):
self.testbed = testbed.Testbed()
self.testbed.activate()
self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=0)
self.testbed.init_datastore_v3_stub(consistency_policy=self.policy)
self.testbed.init_app_identity_stub()
self.testbed.init_memcache_stub()
self.testbed.init_urlfetch_stub()
self.testbed.init_blobstore_stub()
ndb.get_context().clear_cache()
def tearDown(self):
self.testbed.deactivate()
def test_gcs_works(self):
with cloudstorage.open('/mybucket/test.txt', 'w', content_type=MIME) as f:
f.write(FILE_DATA)
with cloudstorage.open('/mybucket/test.txt', 'r') as f:
data = f.read()
print(data)
self.assertEqual(data, FILE_DATA)
def test_signurl(self):
url = sign_url('/mybucket/test.txt')
# FIXME: Not yet working as we have no idea on how to access local GCS during the test.
result = urllib2.urlopen(url)
self.assertEqual(200, result.code)
self.assertEqual(FILE_DATA, result.read())
You can test GCS and service_accounts in your SDK, but you do not have a local appengine GCS service when you use a signed url.
But you can test your local app with service accounts and google cloud services.
Service accounts make it very easy to authorize appengine requests to other Google APIs and services.
To use a service account in the appengine SDK, you have to add two undocumented options when you run the development server:
--appidentity_email_address=<SERVICE_ACCOUNT_EMAIL_ADDRESS>
--appidentity_private_key_path=<PEM_KEY_PATH>
More info in this request for documentation issue
You can create or find the service account in the developers console permissions section of your appengine cloud project.
And you can create and download a p12 key for the service account.
Use OpenSSL to convert this p12 key in a RSA pem key.
I used this OpenSSL installer for Windows.
To create the pem key file in Windows use:
openssl pkcs12 -in <P12_KEY_PATH> -nocerts -nodes -passin pass:notasecret | openssl rsa -out <PEM_KEY_PATH>
Now you can use your cloud app service accounts in the development server and use app_identity to sign and authorize requests.
I'm trying to use the App Engine datastore in my application outside of the bounds of one of the request handler pages (like main.py or the other files you can specify in app.yaml). Is this possible? When I try to run it, it says my database model does not have a method "put" associated with it, but it has no problems when I run it from a script connected to CGI and request handling (like main.py or any other .py files I declare similarly in app.yaml). Do you know what I can do to add CGI to those pages without adding request handling? Thanks!
Edit my code:
users.py file:
from google.appengine.ext import db, webapp
class User(db.Model):
email = db.EmailProperty()
password = db.StringProperty()
main.py file
from user import *
class CreateHandler(webapp.RequestHandler):
def get(self):
u = User()
u.email = "email#email.com"
u.password = "mypass"
u.put()
It gives me this error:
File "........./main.py", line 75, in get
u.put()
AttributeError: User instance has no attribute 'put'
Yes, you can access the datastore from other scripts. You don't have to add request handling to them, that can stay in your main script. Eg., you can do something like this:
app.yaml:
- url: /.*
script: main.py
main.py:
from SectionHandlers import * # This imports classes from SectionHandlers.py
application = webapp.WSGIApplication([
("/section1/.*", Section1Handler), # Map requests to handlers
("/section2/.*", Section2Handler),
], debug=True)
SectionHandlers.py:
from google.appengine.ext import db, webapp
class Section1Handler(BlogHandler):
def get(self):
# Code using 'db' here