Django: executing UPDATE query always returns rowcount 0 - sql-server

I'm new to programming and I'm not sure, whether the problem is in me or in the Django code. I call link method from my view and update field MatchId on Record model. Database is SQL Server 2017.
My view:
class RecordViewSet(viewsets.ModelViewSet):
"""
API for everything that has to do with Records.
Additionally we provide an extra `link` action.
"""
queryset = Record.objects.all().order_by("Id")
serializer_class = RecordSerializer
permission_classes = [permissions.IsAuthenticated]
#action(methods=["post"], detail=False)
def link(self, request, *args, **kwargs):
idToMatch = request.POST.getlist("Id")
recordsToMatch = Record.objects.filter(Id__in=idToMatch)
lastMatchId = Record.objects.latest("MatchId").MatchId
matchedSuccesfully = recordsToMatch.update(MatchId=lastMatchId + 1)
if matchedSuccesfully > 1:
return Response(data=matchedSuccesfully, status=status.HTTP_200_OK)
else:
return Response(data=matchedSuccesfully, status=status.HTTP_404_NOT_FOUND)
For some reason matchedSuccessfully always returns zero. Relevant Django code:
def execute_sql(self, result_type):
"""
Execute the specified update. Return the number of rows affected by
the primary update query. The "primary update query" is the first
non-empty query that is executed. Row counts for any subsequent,
related queries are not available.
"""
cursor = super().execute_sql(result_type)
try:
rows = cursor.rowcount if cursor else 0
is_empty = cursor is None
finally:
if cursor:
cursor.close()
for query in self.query.get_related_updates():
aux_rows = query.get_compiler(self.using).execute_sql(result_type)
if is_empty and aux_rows:
rows = aux_rows
is_empty = False
return rows
I rewrote execute_sql as follows:
def execute_sql(self, result_type):
"""
Execute the specified update. Return the number of rows affected by
the primary update query. The "primary update query" is the first
non-empty query that is executed. Row counts for any subsequent,
related queries are not available.
"""
cursor = super().execute_sql(result_type)
try:
if cursor:
cursor.execute("select ##rowcount")
rows = cursor.fetchall()[0][0]
else:
rows = 0
is_empty = cursor is None
finally:
if cursor:
cursor.close()
for query in self.query.get_related_updates():
aux_rows = query.get_compiler(self.using).execute_sql(result_type)
if is_empty and aux_rows:
rows = aux_rows
is_empty = False
return rows
and now it works, but I'm unsure if there is a more elegant way to resolve this since now I have to ship this exact code everywhere. Source code at:
https://github.com/django/django/blob/main/django/db/models/sql/compiler.py

I've faced the same issue and came to the same point in django's depths.
In my case — the problem was in trigger configured for UPDATE.
It should have return ##ROWCOUNT as a result, but in my case it didn't.
Btw, the thing I did (due to restriction on editing triggers) — overrided save method in base model for such models to force_update=True:
class BaseModel(models.Model):
def save(self, force_insert=False, force_update=False, using=None, update_fields=None):
if self._state.adding:
super().save(force_insert=force_insert, force_update=force_update, using=using, update_fields=update_fields)
else:
try:
super().save(force_insert=force_insert, force_update=True, using=using, update_fields=update_fields)
except DatabaseError as e:
if str(e) == 'Forced update did not affect any rows.':
pass
else:
raise e
class Meta:
managed = False
abstract = True

Related

Return no row - SQL Server

There is a condition if not satisfied then I just want to return no rows as my application will pick no row and will show no record msg on front end. Is there any other professional way?
For now I am using following query to return no row.
select 0
where 1 = 0
Even when you return no rows, you are still returning a schema. And most applications expect the same schema to be returned regardless of the number of rows. Even when 0 rows are returned.
If you can change the SQL in #SqlStr that you are executing with sp_executesql, I would insert into a temporary table in that query and then return the results of selecting from that temporary table:
Select * from #myTempTable where <conditionRequiredForResults>
You said you want to return no rows on your frontend and return nothing. Neither message nor row.
You just need to use if statement to avoid returning anything to your frontend
require_once 'db_connection.php';
$sql = "SELECT * FROM table_name WHERE <your filtering Condition>";
$result = mysqli_query($db_connection, $sql);
if (mysqli_num_rows($result) > 0) {
// If row to be fetched is more than 0
// Return something Example:
echo "Rows detected";
} else {
// If there is no row to be fetched
// Return nothing
}
And this should return nothing if no row fetched from database

Google App Engine Datastore - query with StructuredProperty in projection and filter

I have a couple ndb models looks like these:
class Product(ndb.Model):
manufacturer = ndb.StringProperty()
category = ndb.StringProperty()
price = ndb.FloatProperty()
class Customer(ndb.Model):
customerId = ndb.StringProperty()
name = ndb.StringProperty()
products = ndb.StructuredProperty(Product, repeated=True)
And I'd like to query based on 'manufacturer' and 'category' of the product he/she owns. So this query works as expected.
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"],
category=data_json["product"]["category"]))
results= query.fetch()
However, I cannot get the "projection" to work along with this query. The following query simply returned nothing.
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"],
category=data_json["product"]["category"]))
results= query.fetch(projection=[Customer.products.price])
But if I use the projection without the filter, the projection part works fine. The following query will return all entities but only the 'price' property
results= Customer.query().fetch(projection=[Customer.products.price])
Any thoughts? Thanks.
BTW, my queries were developed based on this article.
https://cloud.google.com/appengine/docs/standard/python/ndb/queries#filtering_structured_properties
The correct way of combining AND and OR operations in the ndb library is documented in NDB Client Library's documentation.
With the query below, you are performing an AND operation in the filter, so instead of this one, you should use the one I propose below, using ndb.AND().
# Your query
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"], category=data_json["product"]["category"]))
# Query using ndb.AND
query = Customer.query(ndb.AND(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"]), Customer.products == Product(category=data_json["product"]["category"])))
Also, it turns out that if you perform the filtering in multiple steps, the query also works:
# Your request
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"], category=data_json["product"]["category"]))
results = query.fetch(projection=[Customer.products.price])
# Request performing filter in multiple steps
query = Customer.query(Customer.products == Product(category=data_json["product"]["category"]))
query1 = query.filter(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"]))
results = query1.fetch(projection=[Customer.products.price])
You can use either of the proposed alternatives, although I would suggest using ndb.AND() as it minimizes the code and is also the best way to combine AND operations.
UPDATE with some code:
app.yaml
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.app
main.py
import webapp2
from google.appengine.ext import ndb
# Datastore Models
class Product(ndb.Model):
manufacturer = ndb.StringProperty()
category = ndb.StringProperty()
price = ndb.FloatProperty()
class Customer(ndb.Model):
customerId = ndb.StringProperty()
name = ndb.StringProperty()
products = ndb.StructuredProperty(Product, repeated=True)
# Create entities for testing purposes
class CreateEntities(webapp2.RequestHandler):
def get(self):
prod1 = Product(manufacturer="Google", category="GCP", price=105.55)
prod2 = Product(manufacturer="Google", category="GCP", price=123.45)
prod3 = Product(manufacturer="Google", category="Drive", price=10.38)
prod1.put()
prod2.put()
prod3.put()
cust1 = Customer(customerId="Customer1", name="Someone", products=[prod1,prod2,prod3])
cust2 = Customer(customerId="Customer2", name="Someone else", products=[prod1])
cust3 = Customer(customerId="Customer3", name="Noone", products=[prod3])
cust1.put()
cust2.put()
cust3.put()
# Response text
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Done creating entities')
class GetEntities(webapp2.RequestHandler):
def get(self):
# This will not work
#query = Customer.query(Customer.products == Product(category="GCP", manufacturer="Google"))
#results = query.fetch(projection=[Customer.products.price])
# Alternative 1 - WORKS
#query = Customer.query(Customer.products == Product(category="GCP"))
#query1 = query.filter(Customer.products == Product(manufacturer="Google"))
#results = query1.fetch(projection=[Customer.products.price])
# Alternative 2 - WORKS
query = Customer.query(ndb.AND(Customer.products == Product(manufacturer="Google"), Customer.products == Product(category="GCP")))
results = query.fetch(projection=[Customer.products.price])
self.response.out.write('<html><body>')
for result in results:
self.response.out.write("%s<br><br>" % result)
self.response.out.write('</body></html>')
app = webapp2.WSGIApplication([
('/createEntities', CreateEntities),
('/getEntities', GetEntities),
], debug=True)

How to get paginated results (async query) from bigquery python client? [duplicate]

I am writing Python code with the BigQuery Client API, and attempting to use the async query code (written everywhere as a code sample), and it is failing at the fetch_data() method call. Python errors out with the error:
ValueError: too many values to unpack
So, the 3 return values (rows, total_count, page_token) seem to be the incorrect number of return values. But, I cannot find any documentation about what this method is supposed to return -- besides the numerous code examples that only show these 3 return results.
Here is a snippet of code that shows what I'm doing (not including the initialization of the 'client' variable or the imported libraries, which happen earlier in my code).
#---> Set up and start the async query job
job_id = str(uuid.uuid4())
job = client.run_async_query(job_id, query)
job.destination = temp_tbl
job.write_disposition = 'WRITE_TRUNCATE'
job.begin()
print 'job started...'
#---> Monitor the job for completion
retry_count = 360
while retry_count > 0 and job.state != 'DONE':
print 'waiting for job to complete...'
retry_count -= 1
time.sleep(1)
job.reload()
if job.state == 'DONE':
print 'job DONE.'
page_token = None
total_count = None
rownum = 0
job_results = job.results()
while True:
# ---- Next line of code errors out...
rows, total_count, page_token = job_results.fetch_data( max_results=10, page_token=page_token )
for row in rows:
rownum += 1
print "Row number %d" % rownum
if page_token is None:
print 'end of batch.'
break
What are the specific return results I should expect from the job_results.fetch_data(...) method call on an async query job?
Looks like you are right! The code no longer return these 3 parameters.
As you can see in this commit from the public repository, fetch_data now returns an instance of the HTTPIterator class (guess I didn't realize this before as I have a docker image with an older version of the bigquery client installed where it does return the 3 values).
The only way that I found to return the results was doing something like this:
iterator = job_results.fetch_data()
data = []
for page in iterator._page_iter(False):
data.extend([page.next() for i in range(page.num_items)])
Notice that now we don't have to manage pageTokens anymore, it's been automated for the most part.
[EDIT]:
I just realized you can get results by doing:
results = list(job_results.fetch_data())
Got to admit it's way easier now then it was before!

GAE-NDB: how prevent projection changed the results

I used ndb projection but it did change the results, how the results are not affected by projection?
class T(ndb.Model):
name = ndb.StringProperty()
name2 = ndb.StringProperty(repeated=True)
#classmethod
def test(cls):
for i in range(0, 10):
t = T(name=str(i))
if i%2 == 0:
t.name2=["zzz"]
t.put()
qr = T.query()
qo = ndb.QueryOptions(projection=['name', 'name2'])
items, cursor, more = qr.fetch_page(20, options=qo)
print len(items)
qo = ndb.QueryOptions(projection=['name'])
items, cursor, more = qr.fetch_page(20, options=qo)
print len(items)
The result is 5, 10
How to make result is 10, 10 ?
Thanks
An empty list-property (repeated=True) won't get indexed and as it's the index that projection queries use to return results, entities without values for the property won't be returned.
Your test case is susceptible to the eventual-consistency that Tim's comment mentions, but it isn't the only issue.

Between query equivalent on App Engine datastore?

I have a model containing ranges of IP addresses, similar to this:
class Country(db.Model):
begin_ipnum = db.IntegerProperty()
end_ipnum = db.IntegerProperty()
On a SQL database, I would be able to find rows which contained an IP in a certain range like this:
SELECT * FROM Country WHERE ipnum BETWEEN begin_ipnum AND end_ipnum
or this:
SELECT * FROM Country WHERE begin_ipnum < ipnum AND end_ipnum > ipnum
Sadly, GQL only allows inequality filters on one property, and doesn't support the BETWEEN syntax. How can I work around this and construct a query equivalent to these on App Engine?
Also, can a ListProperty be 'live' or does it have to be computed when the record is created?
question updated with a first stab at a solution:
So based on David's answer below and articles such as these:
http://appengine-cookbook.appspot.com/recipe/custom-model-properties-are-cute/
I'm trying to add a custom field to my model like so:
class IpRangeProperty(db.Property):
def __init__(self, begin=None, end=None, **kwargs):
if not isinstance(begin, db.IntegerProperty) or not isinstance(end, db.IntegerProperty):
raise TypeError('Begin and End must be Integers.')
self.begin = begin
self.end = end
super(IpRangeProperty, self).__init__(self.begin, self.end, **kwargs)
def get_value_for_datastore(self, model_instance):
begin = self.begin.get_value_for_datastore(model_instance)
end = self.end.get_value_for_datastore(model_instance)
if begin is not None and end is not None:
return range(begin, end)
class Country(db.Model):
begin_ipnum = db.IntegerProperty()
end_ipnum = db.IntegerProperty()
ip_range = IpRangeProperty(begin=begin_ipnum, end=end_ipnum)
The thinking is that after i add the custom property i can just import my dataset as is and then run queries on based on the ListProperty like so:
q = Country.gql('WHERE ip_range = :1', my_num_ipaddress)
When i try to insert new Country objects this fails though, complaning about not being able to create the name:
...
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/db/__init__.py", line 619, in _attr_name
return '_' + self.name
TypeError: cannot concatenate 'str' and 'IntegerProperty' objects
I tried defining an attr_name method for the new property or just setting self.name but that does not seem to help. Hopelessly stuck or heading in the right direction?
Short answer: Between queries aren't really supported at the moment. However, if you know a priori that your range is going to be relatively small, then you can fake it: just store a list on the entity with every number in the range. Then you can use a simple equality filter to get entities whose ranges contain a particular value. Obviously this won't work if your range is large. But here's how it would work:
class M(db.Model):
r = db.ListProperty(int)
# create an instance of M which has a range from `begin` to `end` (inclusive)
M(r=range(begin, end+1)).put()
# query to find instances of M which contain a value `v`
q = M.gql('WHERE r = :1', v)
The better solution (eventually - for now the following only works on the development server due to a bug (see issue 798). In theory, you can work around the limitations you mentioned and perform a range query by taking advantage of how db.ListProperty is queried. The idea is to store both the start and end of your range in a list (in your case, integers representing IP addresses). Then to get entities whose ranges contain some value v (i.e., between the two values in your list), you simply perform a query with two inequality filters on the list - one to ensure that v is at least as big as the smallest element in the list, and one to ensure that v is at least as small as the biggest element in the list.
Here's a simple example of how to implement this technique:
class M(db.Model):
r = db.ListProperty(int)
# create an instance of M which has a rnage from `begin` to `end` (inclusive)
M(r=[begin, end]).put()
# query to find instances of M which contain a value `v`
q = M.gql('WHERE r >= :1 AND r <= :1', v)
My solution doesn't follow the pattern you have requested, but I think it would work well on app engine. I'm using a list of strings of CIDR ranges to define the IP blocks instead of specific begin and end numbers.
from google.appengine.ext import db
class Country(db.Model):
subnets = db.StringListProperty()
country_code = db.StringProperty()
c = Country()
c.subnets = ['1.2.3.0/24', '1.2.0.0/16', '1.3.4.0/24']
c.country_code = 'US'
c.put()
c = Country()
c.subnets = ['2.2.3.0/24', '2.2.0.0/16', '2.3.4.0/24']
c.country_code = 'CA'
c.put()
# Search for 1.2.4.5 starting with most specific block and then expanding until found
result = Country.all().filter('subnets =', '1.2.4.5/32').fetch(1)
result = Country.all().filter('subnets =', '1.2.4.4/31').fetch(1)
result = Country.all().filter('subnets =', '1.2.4.4/30').fetch(1)
result = Country.all().filter('subnets =', '1.2.4.0/29').fetch(1)
# ... repeat until found
# optimize by starting with the largest routing prefix actually found in your data (probably not 32)

Resources