I use peewee related with an exsits table:
import peewee
from playhouse.postgres_ext import *
class Rules(peewee.Model):
channels = JSONField(null=True)
remark = peewee.CharField(max_length=500, null=True)
class Meta:
database = db
db_table = 'biz_rule'
schema = 'opr'
example: in my table there exists a record in column channels:
["A012102","C012102","D012102","E012102"]
I want to judge whether "A012102" is in the list,how to write the code?
If you're using PostgreSQL 9.4+, you can use the jsonb data type using the corresponding postgres_ext.BinaryJSONField peewee field type. It has contains_any() and contains_all() methods that correspond to the PostgreSQL ?| and ?& operators (see the PostgreSQL JSON docs). So I think it'd be something like this:
from playhouse.postgres_ext import BinaryJSONField
class Rules(peewee.Model):
channels = BinaryJSONField(null=True)
...
query = Rules.select().where(Rules.channels.contains_all('A012102'))
Related
I have tables in my database, that are not managed by Symfony; there are no entities for these tables. They are tables from another application, I import them and use Symfony to generate statistics from the data in the tables.
How do I access this?
Can i use doctrine and a regular repository for this?
I just want to read data, not update.
Right now I'm using straight mysqli_connect and mysqli_query, but that just doesn't feel right using Symfony 5.
You should just be able to query with sql. The following example comes straight from the docs:
// src/Repository/ProductRepository.php
// ...
class ProductRepository extends ServiceEntityRepository
{
public function findAllGreaterThanPrice(int $price): array
{
$conn = $this->getEntityManager()->getConnection();
$sql = '
SELECT * FROM product p
WHERE p.price > :price
ORDER BY p.price ASC
';
$stmt = $conn->prepare($sql);
$stmt->execute(['price' => $price]);
// returns an array of arrays (i.e. a raw data set)
return $stmt->fetchAllAssociative();
}
}
https://symfony.com/doc/current/doctrine.html#querying-with-sql
I'm trying to incorporate a AWS GLUE Data Catalog to my Data Lake I'm building out. I'm using a few different databases and would like to add COMMENTS to columns in a few of these tables. These databases include Redshift and MySql. I usually add the comments to the column by doing something along the lines of
COMMENT ON COLUMN table.column_name IS 'This is the comment';
Now i know that Glue has a comment field that shows in the GUI. Is there a way to sync the comment field in Glue with the comments I add to the columns in a DB?
In order to update some meta information about a table that has been defined in AWS Glue Data Catalog, you would need to use a combination of get_table() and update_table() methods with boto3 for example .
Here is the most naive approach to do that:
import boto3
from pprint import pprint
glue_client = boto3.client('glue')
database_name = "__SOME_DATABASE__"
table_name = "__SOME_TABLE__"
response = glue_client.get_table(
DatabaseName=database_name,
Name=table_name
)
original_table = response['Table']
Here original_table adheres response syntax defined by get_table(). However, we need to remove some fields from it so it would pass validation when we use update_table(). List of allowed keys could be obtained by passing original_table directly to update_table() without any chagnes
allowed_keys = [
"Name",
"Description",
"Owner",
"LastAccessTime",
"LastAnalyzedTime",
"Retention",
"StorageDescriptor",
"PartitionKeys",
"ViewOriginalText",
"ViewExpandedText",
"TableType",
"Parameters"
]
updated_table = dict()
for key in allowed_keys:
if key in original_table:
updated_table[key] = original_table[key]
For simplicity sake, we will change comment of the very first column from the table
new_comment = "Foo Bar"
updated_table['StorageDescriptor']['Columns'][0]['Comment'] = new_comment
response = glue_client.update_table(
DatabaseName=database_name,
TableInput=updated_table
)
pprint(response)
Obviously, if you want to add a comment to a specific column you would need to extend this to
new_comment = "Targeted Foo Bar"
target_column_name = "__SOME_COLUMN_NAME__"
for col in updated_table['StorageDescriptor']['Columns']:
if col['Name'] == target_column_name:
col['Comment'] = new_comment
response = glue_client.update_table(
DatabaseName=database_name,
TableInput=updated_table
)
pprint(response)
I'm using Peewee for working with database. I have a User tables with 3 fields: username, password and last_login. When a user login to the system i want to update last_login. I've use following lines of code:
from peewee import *
import datetime
class User(Model):
username = CharField(unique=True)
password = CharField()
last_login = DateTimeField(default=datetime.datetime.now())
class Meta:
database = MySQLDatabase('mydb', user='root', charset='123456')
u=User(username="user1", last_login=datetime.datetime.now())
u.save()
Although i haven't specified any value for password, it is overwritten after u.save() is called. How should i force peewee to only update last_login field?
Replace u.save() with:
u.save(only=[User.last_login])
As the API's documentation says:
only (list) – A list of fields to persist – when supplied, only the given fields will be persisted.
So you should specify a list of fields you want to be changed.
You can use the only argument when calling save(). http://docs.peewee-orm.com/en/latest/peewee/api.html#Model.save
When a user login to the system i want to update last_login. I've use following lines of code:
If you want to do this, you should do an atomic update, however:
User.update({User.last_login: datetime.datetime.now()}).where(User.username == 'whatever').execute()
The following code will demonstrate how to create, get and update a record in the database:
now = datetime.datetime.now()
# create a user
u = User.create(username="user1", password="bla", last_login=now)
# now `u` has your user, you can do: print u.username, u.password, u.last_login
# get an existing user from the db
u = User.get(User.username == "user1")
print u.username, u.password, u.last_login
sleep(1)
now = datetime.datetime.now()
# update an existing user
u = User.update(password="blabla", last_login=now).where(User.username == "user1")
u.execute()
If you want to save only modified fields, you may use the method below:
class User(Model):
username = CharField(unique=True)
password = CharField()
last_login = DateTimeField(default=datetime.datetime.now())
class Meta:
database = MySQLDatabase('mydb', user='root', charset='123456')
# This method saves only modefied fields
only_save_dirty = True
u=User(username="user1", last_login=datetime.datetime.now())
u.save()
I am trying to populate a table with data and am using Django's get_or_create method. Whenever I do this it will enter records into the database but at a certain record it will throw the above error. My queryset function is
r, created = Response.objects.get_or_create(
auth_user=auth_user,
name=surv_name,
organization=org_id,
category=category,
question=question,
present_order=present_order,
reference=reference,
quest_id=quest_id,
survey_id=survey_id
)
My response table is
class Response(models.Model):
auth_user = models.ForeignKey('AuthUser')
survey = models.ForeignKey('Survey')
name = models.CharField(max_length=50)
organization = models.ForeignKey('Organization')
tf_question_key = models.CharField(max_length=50)
category = models.CharField(max_length=25, blank=True, null=True)
question = models.CharField(max_length=2048)
quest_id = models.CharField(max_length=25)
present_order = models.IntegerField()
reference = models.CharField(max_length=20)
answer = models.CharField(max_length=2048)
remediation = models.CharField(max_length=2048, blank=True, null=True)
dt_started = models.DateTimeField(db_column='DT_Started',
auto_now_add=True) # Field name made lowercase.
dt_completed = models.DateTimeField(db_column='DT_COMPLETED',
auto_now_add=True) # Field name made lowercase.
class Meta:
managed = False
db_table = 'response'
and the traceback where the error is located is
organization <Organization: Individual Offices>
r <Response: Response object>
user_id 2
question ('Does your written policy include the follow-up process for significant outstanding checks, including, but not limited to, checks to recording clerk, checks to tax collector, hazard insurance checks, underwriter checks or checks for mortgage payoffs and any other high risk items? ( 2.03 k )')
present_order 21
survey_id 1
reference '2.03 (k)'
quest_id 27
created True
category 'Pillar II'
surv_name 'Compliance Benchmark'
org_id 1
auth_user <AuthUser: AuthUser object>
I can add records to the table by using
r = Response(
auth_user=auth_user,
name=surv_name,
organization=organization,
category=category,
question=question,
present_order=present_order,
reference=reference,
quest_id=quest_id,
survey_id=survey_id
)
r.save()
but I need to use the get_or_create method to avoid duplicating records. I am not sure why I can add records with the .save() method but not with get_or_create and also why with get_or_create it will add records up to a certain one and then fail. The only thing that is changing is the question, quest_id, present_order, and reference.
I am using python 3.4, django 1.8.4 and SQL Server 2014
Any insight would be greatly appreciated.
I ran into the same issue and turned on logging on sql server to see what was occurring. It looks like long text fields are being converted to ntext. This is then being compared to the nvarchar field causing the error.
The error is occurring during the SELECT within the get_or_create function. Instead of using get_or_create, query for your model with startswith. Using startswith performs a LIKE check which will work. I also added a length check on the field to ensure the fields will match instead of finding other rows with the same starting value.
from django.core.exceptions import ObjectDoesNotExist
from django.db.models.functions import Length
attrs = {
auth_user=auth_user,
name=surv_name,
organization=org_id,
category=category,
present_order=present_order,
reference=reference,
quest_id=quest_id,
survey_id=survey_id,
}
try:
r = Response.objects.annotate(
text_len=Length('question')
).get(
text_len__exact=len(question),
question__startswith=question,
**attrs
)
except ObjectDoesNotExist:
r = Response.objects.create(
question=question,
**attrs
)
I've got a simple User model, defined like so:
# models.py
from datetime import datetime
from myapp import db
class User(db.Model):
id = db.Column(db.Integer(), primary_key=True)
email = db.Column(db.String(100), unique=True)
password = db.Column(db.String(100))
date_updated = db.Column(db.DateTime())
def __init__(self, email, password, date_updated=None):
self.email = email
self.password = password
self.date_updated = datetime.utcnow()
When I create a new User object, my date_updated field gets set to the current time. What I'd like to do is make it so that whenever I save changes to my User object my date_updated field is set to the current time automatically.
I've scoured the documentation, but for the life of me I can't seem to find any references to this. I'm very new to SQLAlchemy, so I really have no prior experience to draw from.
Would love some feedback, thank you.
Just add server_default or default argument to the column fields:
created_on = db.Column(db.DateTime, server_default=db.func.now())
updated_on = db.Column(db.DateTime, server_default=db.func.now(), server_onupdate=db.func.now())
I prefer the {created,updated}_on column names. ;)
SQLAlchemy docs about column insert/update defaults.
[Edit]: Updated code to use server_default arguments in the code.
[Edit 2]: Replaced onupdate with server_onupdate arguments.
date_created = db.Column(db.DateTime, default=db.func.current_timestamp())
date_modified = db.Column(db.DateTime, default=db.func.current_timestamp(),
onupdate=db.func.current_timestamp())