Flask and connecting to a database confusion - database

I am currently creating a web application using Flask. My main issue at the moment understanding the concept of connecting to a database as there are many resources online which are confusing me in terms of establishing a solid connection to a database. The Syntax to SQL is not a problem as I have knowledge of that.
I am choosing SQLAlchemy with a dialect of SQLite instead of MySQL, PostgresSQL and etc.
My first question is: is choosing a dialect while using SQLAlchemy necessary? Can we not use SQLAlchemy as it is?
Second Question: I have seen many examples and tutorials online using "phpMyAdmin" or something similar to have a visual and interactive way to deal with their database (relations) in their localhost browser. Is this necessary to set-up before creating any type of database connection for any type of project?
Second Question (extension): To set up pypMyAdmin, there are tutorials such as "https://www.youtube.com/watch?v=hVHFPzjp064&t=238s" indicating to activate apache, activate PHP, and download MySQL to use a workbench. As stated in the second question, are these steps mandatory - as many tutorials don't seem to show how to set this up.
Third Question: Due to my project slowly growing, I am using the 'separation of concerns' concept. My file tree is the following:
After researching, I believe I should include database related code with the __init__.py file? Plus, of course updating the config file with the necessary configurations? What I don't understand is, the syntax used to connect to a database. The following code will show my code in both files stated above:
__init__.py
# This class will ultimately bring our entire application together.
from flask import Flask
from config import Config
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
# Creating Flask app.
app = Flask(__name__)
# Creating a database object which represents the database.
# Created a migration object which represents the migration engine.
db = SQLAlchemy(app)
migrate = Migrate(app, db)
# TODO Explain reasons for using this method:
# Using method to determine Flask environment from the following link:
# https://www.youtube.com/watch?v=GW_2O9CrnSU&t=366s
if app.config["ENV"] == "production":
app.config.from_object("config.ProductionConfig")
elif app.config["ENV"] == "testing":
app.config.from_object("config.TestingConfig")
else:
app.config.from_object("config.DevelopmentConfig")
# Importing views file to avoid circular import.
from app import views
from app import admin_views
from app import routes, models
config.py
# This class contains important information regarding the conifgurations for this application.
# It is good practice to keep configurations of the application in a seperate file. This enforces the
# practice of 'seperation of concerns'.
# There is a main class "Config" which has subclasses as illustrated below. The configuration settings
# are defined as class variables within the 'Config' class. As the application grows, we can create subclasses.
import os
basedir = os.path.abspath(os.path.dirname(__file__))
# The SECRET_KEY is important as it...
class Config(object):
DEBUG = False
TESTING = False
SECRET_KEY = '\xb6"\xc5\xce\xc2D\xd1*\x0c\x06\x83 \xbc\xdbM\x97\xe2\xf4OZ\xdc\x16Jv'
# The SQLAlchemy extension is connecting the location of the database from the URI variable.
# The fallback value if the value is not defined is given below as the URL.
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') or \
'sqlite:///' + os.path.join(basedir, 'app.db')
# The 'modifications' config option is set to false as it prevents a signal from appearing whenever
# there is a change made within the database.
SQLALCHEMY_TRACK_MODIFICATIONS = False
class ProductionConfig(Config):
pass
class DevelopmentConfig(Config):
DEBUG = True
class TestingConfig(Config):
TESTING = True
I apologise if my questions seem all over the place. The more I research, the more confused I am becoming with being able to successfully connect to a database.
I would appreciate if someone could answer my concerns in an 'easy to understand' way.

You can avoid (or forestall) some amount of confusion by starting from
flask-sqlalchemy, which provides a convenience layer over SQLAlchemy.
It will arrange for SQLALCHEMY_DATABASE_URI to be turned into an SQLAlchemy "engine" for the database specified in the URI. SQLAlchemy does all of the heavy lifting. There's no need to do the create_engine() yourself when using flask-sqlalchemy.
Adding that following the organizational scheme from chapter 15 of the Flask Mega Tutorial will carry you quite far.

Related

How to deploy the hugging face model via sagemaker pipeline

Below is the code to get the model from Hugging Face Hub and deploy the same model via sagemaker.
from sagemaker.huggingface import HuggingFaceModel
import sagemaker
role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
'HF_MODEL_ID':'siebert/sentiment-roberta-large-english',
'HF_TASK':'text-classification'
}
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
transformers_version='4.17.0',
pytorch_version='1.10.2',
py_version='py38',
env=hub,
role=role,
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1, # number of instances
instance_type='ml.g4dn.xlarge' # ec2 instance type
)
How can I deploy this model via sagemaker pipeline .
How can I include this code in sagemaker pipeline.
Prerequisites
SageMaker pipelines offer many different components with lots of functionality. Your question is quite general and needs to be contextualized to a specific problem.
You have to start with putting up a pipeline first.
See the complete guide "Defining a pipeline".
Quick response
My answer is to follow this official AWS guide that answers your question exactly:
SageMaker Pipelines: train a Hugging Face model, deploy it with a Lambda step
General explanation
Basically, you need to build your pipeline architecture with the components you need and register the trained model within the Model Registry.
Next, you have two paths you can follow:
Trigger a lambda that automatically deploys the registered model (as the guide does).
Out of the context of the pipeline, do the automatic deployment by retrieving the ARN of the registered model on the Model Registry. You can get it from register_step.properties.ModelPackageArn or in an external script using boto3 (e.g. using list_model_packages)

How can you access Azure Web App AppSettings from a React app at run time

I have a React app created with create-react-app which I have deployed to an Azure Web App. There is no back-end: the site is purely 'static'.
In the production environment, there are a number of keys to API services and other secrets that I need to keep secure, but which the client app needs to be able to read.
In React there's a mechanism for accessing environment-specific information using .env files, such as .env.production, but this is not suitable for keeping secrets, as environment variables mentioned in the code are substituted with the actual value from the .env file during the build process, and are consequently visible to anyone looking at the JavaScript in their browser.
Setting the values of the AppSettings can be done on the Azure Portal (or via suitable scripting in the CI/CD pipeline), but how can I read the AppSettings values at runtime?
A number of StackOverflow questions have been asked about this, but none of the answers address the fundamental question properly, or seem to miss the point. For example, here and here.
First off, this topic is puzzling lot of developers around the globe and it should be addressed properly in Aspnet Core. One viable option would be to set up Server-Side Rendering, but there are some of us who wouldn't benefit from it. Also, there are no proper examples for doing it for ReactJS + Redux in Aspnet Core world.
My solution is to go for a InMemoryFileProvider, see https://github.com/dazinator/Dazinator.AspNet.Extensions.FileProviders for details about the NuGet package.
To start using one, in my Configure() I'll do a:
configuredSpaFileProvider = CreateFileProvider(env);
app.UseSpaStaticFiles(new StaticFileOptions
{
FileProvider = configuredSpaFileProvider
});
My CreateFileProvider() contains an instantiation of InMemoryFileProvider-class, which can be populated with additional fake "files". Pre-existing static file on filesystem will be served with a PhysicalFileProvider.
To allow access to index.html, CSS, JavaScript and all possible static files, do what a DefaultSpaStaticFileProvider would do:
var defaultPath = "build";
if (env.IsDevelopment())
defaultPath = "public";
var absoluteRootPath = System.IO.Path.Combine(
env.ContentRootPath,
$"{ReactJSdirectory}/{defaultPath}");
var physicalFileProvider = new PhysicalFileProvider(absoluteRootPath);
Then create a provider and virtual file like this:
var inMemoryProvider = new InMemoryFileProvider();
inMemoryProvider.Directory.AddFile("/", new StringFileInfo(configJsContent, "app.config.js"));
Finally glue them together, prioritizing on the virtual files:
return new CompositeFileProvider(inMemoryProvider, physicalFileProvider);
Now that a dynamic JavaScript-file exists, it needs to be accessed on client-code. There are options for this, but I'll simply do a:
<script src="/app.config.js" async defer></script>
in my index.html. The trick is to construct a string with suitable JavaScript-parseable content setting up variables and storing them into window-object and accessing them later on a client.
Now my 12 factor application is fully compliant with third factor "Store config in the environment".

Play Siena failing to connect to MySQL on GAE

I am using play framework 1.2.7, gae module 1.6.0 and siena module 2.0.7 (also tested 2.0.6). This is a simple project that should run in play deployed on App Engine and connect to a MySQL database in Google Cloud SQL. My project runs fine locally but fails to connect to the database in production. Looking at the logs it looks like it is using the postgresql driver instead of the mysql one.
Application.conf
# db=mem
db.url=jdbc:google:mysql://PROJECT_ID:sienatest/sienatest
db.driver=com.mysql.jdbc.GoogleDriver
db.user=root
db.pass=root
This is the crash stack trace
play.Logger niceThrowable: Cannot connected to the database : null
java.lang.NullPointerException
at com.google.appengine.runtime.Request.process-a3b6145d1dbbd04d(Request.java)
at java.util.Hashtable.put(Hashtable.java:432)
at java.util.Properties.setProperty(Properties.java:161)
at org.postgresql.Driver.loadDefaultProperties(Driver.java:121)
at org.postgresql.Driver.access$000(Driver.java:47)
at org.postgresql.Driver$1.run(Driver.java:88)
at java.security.AccessController.doPrivileged(AccessController.java:63)
at org.postgresql.Driver.getDefaultProperties(Driver.java:85)
at org.postgresql.Driver.connect(Driver.java:231)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at play.modules.siena.GoogleSqlDBPlugin.onApplicationStart(GoogleSqlDBPlugin.java:103)
at play.plugins.PluginCollection.onApplicationStart(PluginCollection.java:525)
at play.Play.start(Play.java:533)
at play.Play.init(Play.java:305)
What is going on here? I am specifying the correct driver and url schema and it's using postgresql driver. Google Cloud SQL API access is enabled, the app is allowed to connect to the mysql instance, I am not using db=mem, ... I am stuck and can't figure out how to move forward! :-((
UPDATE: I thought I found the solution, but that was not the case. If I keep the %prod. prefix and create a war normally (or just don't define any DB properties), then the application will use Google DataStore instead of the Cloud SQL. If I create the war file adding --%prod at the end (or just delete the %prod. prefix in the application.conf), then it will keep failing to connect to the database showing the same initial error.
Any ideas please?
After being stuck for so long on this I just found the solution in no time after posting the question. Quite stupid actually.
The production environment properties in the application.conf file must be preceded by %prod. so the database config should read
%prod.db.url=jdbc:google:mysql://PROJECT_ID:sienatest/sienatest
%prod.db.driver=com.mysql.jdbc.GoogleDriver
%prod.db.user=root
%prod.db.pass=root
And everything runs fine.
EDIT: This is NOT the solution. The problem went away, but the app is using the DataStore instead of the Cloud SQL
At the end I ended doing a slight modification in play siena module source code and recompiling it.
In case anyone is interested, you will need to remove/comment/catch exception in this code around line 97 in GoogleSqlDBPlugin class:
// Try the connection
Connection fake = null;
try {
if (p.getProperty("db.user") == null) {
fake = DriverManager.getConnection(p.getProperty("db.url"));
} else {
fake = DriverManager.getConnection(p.getProperty("db.url"), p.getProperty("db.user"), p.getProperty("db.pass"));
}
} finally {
if (fake != null) {
fake.close();
}
}
For some reason the connection fails when initiated with DriverManager.getConnection() but it works when initiated with basicDatasource.getConnection(); which apparently is the way used by the module in the rest of the code. So if you delete the above block, and recompile the module everything will work as expected. If you are compiling with JDK 7, you will also need to implement public Logger getParentLogger() throws SQLFeatureNotSupportedException in the ProxyDriver inner class at the end of GoogleSqlDBPlugin file.
Strangely, I digged into the DriverManager.getConnection() and it looked like some postgresql driver is registered somehow, because otherwise I can't see why DriverManager.getConnection() would call to org.postgresql.Driver.connect().

What's a namespace used for in the App Engine datastore?

In the development admin console, when I look at my data, it says "Select different namespace".
What are namespaces for and how should I use them?
Namespaces allow you to implement segregation of data for multi-tenant applications. The official documentation links to some sample projects to give you an idea how it might be used.
Namespaces is used in google app engine to create Multitenant Applications. In Multitenent applications single instance of the application runs on a server, serving multiple client organizations (tenants). With this, an application can be designed to virtually partition its data and configuration (business logic), and each client organization works with a customized virtual application instance..you can easily partition data across tenants simply by specifying a unique namespace string for each tenant.
Other Uses of namespace:
Compartmentalizing user information
Separating admin data from application data
Creating separate datastore instances for testing and production
Running multiple apps on a single app engine instance
For More information visit the below links:
http://www.javacodegeeks.com/2011/12/multitenancy-in-google-appengine-gae.html
https://developers.google.com/appengine/docs/java/multitenancy/
http://java.dzone.com/articles/multitenancy-google-appengine
http://www.sitepoint.com/multitenancy-and-google-app-engine-gae-java/
Looking, towards this question is not that much good reviewed and answered so trying to give this one.
When using namespaces, we can have a best practice of key and value separation there on a given namespace. Following is the best example of giving the namespace information thoroughly.
from google.appengine.api import namespace_manager
from google.appengine.ext import db
from google.appengine.ext import webapp
class Counter(db.Model):
"""Model for containing a count."""
count = db.IntegerProperty()
def update_counter(name):
"""Increment the named counter by 1."""
def _update_counter(name):
counter = Counter.get_by_key_name(name)
if counter is None:
counter = Counter(key_name=name);
counter.count = 1
else:
counter.count = counter.count + 1
counter.put()
# Update counter in a transaction.
db.run_in_transaction(_update_counter, name)
class SomeRequest(webapp.RequestHandler):
"""Perform synchronous requests to update counter."""
def get(self):
update_counter('SomeRequest')
# try/finally pattern to temporarily set the namespace.
# Save the current namespace.
namespace = namespace_manager.get_namespace()
try:
namespace_manager.set_namespace('-global-')
update_counter('SomeRequest')
finally:
# Restore the saved namespace.
namespace_manager.set_namespace(namespace)
self.response.out.write('<html><body><p>Updated counters')
self.response.out.write('</p></body></html>')

How to run the CherryPy web server in the Google App Engine

The CherryPy web server can supposedly be deployed in the Google App Engine.
Who has done it, and what was the experience like?
What special effort was required (configuration, etc.)?
Would you recommend it to others?
The article is a good example but its slightly out of date now as the patch is no longer required, the latest version of Cherrypy should run without it, I've gotten the sample below running in the development environment.
I've included cherrypy inside a zip file as the google app engine has a limit of one thousand files per application, it also makes it easier to deploy.
I'm also using the cherrypy dispatch handler to route the request.
import sys
sys.path.insert(0, 'cherrypy.zip')
import cherrypy
import wsgiref.handlers
class Root:
exposed = True
def GET(self):
return "give a basic description of the service"
d = cherrypy.dispatch.MethodDispatcher()
conf = {'/':
{
'request.dispatch': d
}
}
app = cherrypy.tree.mount(Root(), "/",conf)
wsgiref.handlers.CGIHandler().run(app)
So far I've not come across any particular issues but I have read some people have had issues with sessions.
See boodebr.org article (missing, but here on the Wayback machine) It works for me.
If you are looking for an example, look for the condition that accepts ServerMode.GAE in ServerInterface.auto in this example.
There is a good article on how to do this over here now here. I haven't actually tried this yet, I stuck with django on App Engine, but it seems to be a solid example.

Resources