I am trying to use HTSQL for one of my Django projects. For that I followed the procedure given HERE for furnishing HTSQL/Django requirements. Then I cloned the HTSQL repository for trying the example/demo in it from HERE. The default db used in the demo example is sqlite3.
I have tried this demo on both Django v 1.4 and Django v 1.3.1(had to make some tweaks in settings.py for Django v 1.3.1). As instructed in the HTSQL Django-gateway Blog, I wrote the following code in the django project shell:
>>> from htsql_django import produce
>>> query = "/polls_poll{question, total:=sum(polls_choice.votes)}"
>>> for row in produce(query):
>>> print "%s: %s" % (row.question, row.total)
It throws following error:
TransactionManagementError: This code isn't under transaction management
The whole error trace can be viewed at pastebin
I have also tried this on my own new project but same error.
When you use HTSQL from a Django shell, you have to open a transaction explicitly:
>>> from django.db import transaction
>>> from htsql_django import produce
>>> with transaction.commit_on_success():
... query = "/polls_poll{question, total:=sum(polls_choice.votes)}"
... for row in produce(query):
... print "%s: %s" % (row.question, row.total)
I'm sorry the documentation isn't clear about that. We may change it in a future release.
Related
I am trying to run a simple python 2 server code with AppEngine and Datastore. When I run dev_appserver.py app.yaml, the program immediately exits (without an error) after the following outputs:
/home/username/google-cloud-sdk/lib/third_party/google/auth/crypt/_cryptography_rsa.py:22: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
import cryptography.exceptions
INFO 2022-12-20 11:59:41,931 devappserver2.py:239] Using Cloud Datastore Emulator.
We are gradually rolling out the emulator as the default datastore implementation of dev_appserver.
If broken, you can temporarily disable it by --support_datastore_emulator=False
Read the documentation: https://cloud.google.com/appengine/docs/standard/python/tools/migrate-cloud-datastore-emulator
INFO 2022-12-20 11:59:41,936 devappserver2.py:316] Skipping SDK update check.
INFO 2022-12-20 11:59:42,332 datastore_emulator.py:156] Starting Cloud Datastore emulator at: http://localhost:22325
INFO 2022-12-20 11:59:42,981 datastore_emulator.py:162] Cloud Datastore emulator responded after 0.648865 seconds
INFO 2022-12-20 11:59:42,982 <string>:384] Starting API server at: http://localhost:38915
Ideally, it should have continued by runnning the server on port 8000. Also, it works with option --support_datastore_emulator=False.
This is the code:
import webapp2
import datetime
from google.appengine.ext import db, deferred, ndb
import uuid
from base64 import b64decode, b64encode
import logging
class Email(ndb.Model):
email = ndb.StringProperty()
class DB(webapp2.RequestHandler):
def post(self):
try:
mail = Email()
mail.email = 'Test'
mail.put()
except Exception as e:
print(e)
self.response.headers['Content-Type'] = 'application/json; charset=utf-8'
return self.response.out.write(e)
def get(self):
try:
e1 = Email.query()
logging.critical('count is: %s' % e1.count)
e1k = e1.get(keys_only=True)
logging.critical('count 2 is: %s' % e1k.count)
e1 = e1.get()
key = unicode(e1.key.urlsafe())
logging.critical('This is a critical message: %s' % key)
logging.critical('This is a critical message: %s' % e1k)
e2 = ndb.Key(urlsafe=key).get()
self.response.headers['Content-Type'] = 'application/json; charset=utf-8'
return self.response.out.write(str(e2.email))
except Exception as e:
print(e)
self.response.headers['Content-Type'] = 'application/json; charset=utf-8'
return self.response.out.write(e)
app = webapp2.WSGIApplication([
('/', DB)
], debug=True)
How can I find the reason this is not working?
Edit: I figured that the dev server works and writes to a datastore even with support_datastore_emulator=False option. I am confused by this option. I also don't know where the database is stored currently.
It should be count() and not count i.e.
logging.critical('count is: %s' % e1.count())
A get returns only 1 record and so it doesn't make sense to do a count after calling a get. Besides, the count operation is a method of the query instance not the results. This means the following code is incorrect
e1k = e1.get(keys_only=True)
logging.critical('count 2 is: %s' % e1k.count)
You should replace it with
elk = e1.fetch(keys_only=True) # fetch gives an array
logging.critical('count 2 is: %s' % len(e1k))
When you first run your App, it will execute the GET part of your code and because this is the first time your App is being run, you have no record in Datastore. This means e1 = e1.get() will return None and key = unicode(e1.key.urlsafe()) will lead to an error.
You have to modify your code to first check you have a value for e1 or e2 before you attempt to use the keys.
I ran your code with dev_appserver.py and it displayed these errors for me in the logs. But I ran it with an older version of gcloud SDK (Google Cloud SDK 367.0.0). I don't know why yours exited without displaying any errors. Maybe it's due to the version...??
Separately - Don't know why you're importing db. Google moved on to ndb long ago and you don't use db in your code
The default datastore (for the older generation runtimes like Python 2) is in .config (hidden folder) > gcloud > emulators > datastore
You can also specify your own location by using the flag --datastore_path. See documentation
I've decided to experiment with apache flink a bit. I decided to use scala console (or more precisely http://ammonite.io/) to read some stuff from csv file and print it locally... just to debug end experiments.
import $ivy.`org.apache.flink:flink-csv:1.10.0`
import $ivy.`org.apache.flink::flink-scala:1.10.0`
import org.apache.flink.api.scala._
import org.apache.flink.api.scala.extensions._
val env = ExecutionEnvironment.createLocalEnvironment()
val lines = env.readCsvFile[(String, String, String)]("/home/slovic/Dokumenty/test.csv")
lines.collect()
//java.lang.NullPointerException: Cannot find compatible factory for specified execution.target (=local)
//org.apache.flink.util.Preconditions.checkNotNull(Preconditions.java:104)
//org.apache.flink.api.java.ExecutionEnvironment.executeAsync(ExecutionEnvironment.java:937)
//org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:860)
//org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:844)
//org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:495)
//org.apache.flink.api.scala.DataSet.collect(DataSet.scala:739)
//ammonite.$sess.cmd24$.<init>(cmd24.sc:1)
//ammonite.$sess.cmd24$.<clinit>(cmd24.sc)
What I need to do to run this code locally? (tested with scala 2.11 & 2.12)
EDIT: SOLLUTION BY Piyush_Rana
We need additional import:
import $ivy.`org.apache.flink::flink-streaming-scala:1.10.0` //Piyush_Rana's advice. !!!FIX!!!
I also got the same error and figured out that was missing one dependency -
val flinkVersion = "1.10.0"
"org.apache.flink" %% "flink-streaming-scala" % flinkVersion,
or in ammonite repl:
import $ivy.`org.apache.flink::flink-streaming-scala:1.10.0`
You didnt execute the flink program .
Try to add execute command at the end.
env.execute("unique name")
Since the forum at akeneo.com is locked down, I posted my question here.
When I try to add Produkts to a Product-Model via mass-edit, I get the following error message:
No JobInstance found with code "add_to_existing_product_model"
[2018-06-19 19:39:31] request.INFO: Matched route "pim_enrich_mass_edit_rest_launch". {"route":"pim_enrich_mass_edit_rest_launch","route_parameters":{"_controller":"pim_enrich.controller.rest.mass_edit:launchAction","_route":"pim_enrich_mass_edit_rest_launch"},"request_uri":"http://pim.eu-trading.eu/rest/mass_edit/","method":"POST"} []
[2018-06-19 19:39:32] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Translation\Exception\NotFoundResourceException: "No JobInstance found with code "add_to_existing_product_model"" at ./vendor/akeneo/pim-community-dev/src/Pim/Bundle/EnrichBundle/MassEditAction/OperationJobLauncher.php line 59 {"exception":"[object] (Symfony\\Component\\Translation\\Exception\\NotFoundResourceException(code: 0): No JobInstance found with code \"add_to_existing_product_model\" at ./vendor/akeneo/pim-community-dev/src/Pim/Bundle/EnrichBundle/MassEditAction/OperationJobLauncher.php:59)"} []
I get this error with the latest version of Akeneo 2 (v2.2.5). The product model was created manually, the products to be associated with the model came through the api.
This error looks like a missing job in the database. Did you run all the doctrine migrations?
To do so you need to launch this command:
bin/console doctrine:migrations:migrate --env=prod
If you already launched the migrations and they failed, you can install a clean 2.2.5 PIM elsewhere and dump the job instance table to be able to add the missing jobs. Here is the list of the jobs to add or update in 2.2:
- add_association
- move_to_category
- add_to_category
- remove_from_category
- add_to_existing_product_model
- compute_family_variant_structure_changes
- compute_completeness_of_products_family
- add_attribute_value
- delete_products_and_product_models
So I had a lot of problems with datetimes in Django on the App Engine, but then I started tracing the error and seems like there's more serious problems. Here's what I'm seeing.
Here's my database:
mysql> select * from polls_question;
+----+---------------+----------------------------+
| id | question_text | pub_date |
+----+---------------+----------------------------+
| 1 | test | 2016-02-08 15:24:44.000000 |
+----+---------------+----------------------------+
1 row in set (0.16 sec)
And here's the code trying to read this data:
import MySQLdb
import os
import webapp2
class IndexPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
env = os.getenv('SERVER_SOFTWARE')
if (env and env.startswith('Google App Engine/')):
# Connecting from App Engine
db = MySQLdb.connect(
unix_socket='/cloudsql/<removed>:<removed>',
user='root',db='gaetest')
else:
# Connecting from an external network.
# Make sure your network is whitelisted
db = MySQLdb.connect(
host='<removed>',
port=3306,
user='root', passwd='<removed>',db='<removed>')
cursor = db.cursor()
cursor.execute('SELECT * FROM polls_question')
for r in cursor.fetchall():
self.response.write('%s\n' % str(r))
db.close()
app = webapp2.WSGIApplication([
('/', IndexPage),
])
On my computer it gives me:
(1L, 'test', datetime.datetime(2016, 2, 8, 15, 24, 44))
and when I access the remote url I get:
(1L, 'test', None)
Not sure what more I can do, this example is as simplified as it can get. Does anyone have any clue what's going on? Too bad Google is typically impossible to get hold off.
So after having spent way too much time on this I found the problem, the database was created using django's migrate function, which lead to it creating a datetime(6) (high precision), these don't work when accessing the site remotely. Normal datetimes work. So changing those makes it work.
The cause of the problem was when I used a newer mysql-python lib locally which supported datetime(6).
It was a bug with App Engine library MySQLdb 1.2.4b4, which at time of writing is also "latest". Switching the app.yaml dependency for MySQLdb from "latest" or "1.2.4b4" to "1.2.5" or "1.2.4" should fix the issue.
When I query Google AppEngine's datastore using PHP(through Quercus) and the low-level data-access API for an entity, I get an error that the entity doesn't exist, even though I've put it in the datastore previously.
The specific error is "com.caucho.quercus.QuercusException: com.google.appengine.api.datastore.DatastoreService.get: No entity was found matching the key: Test(value1)"
Here's the relevant code -
<?php
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.EntityNotFoundException;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.datastore.PreparedQuery;
import com.google.appengine.api.datastore.Query;
$testkey = KeyFactory::createKey("Test", "value1");
$ent = new Entity($testkey);
$ent->setProperty("field1", "value2");
$ent->setProperty("field2", "value3");
$dataService = DatastoreServiceFactory::getDatastoreService();
$dataService->put($ent);
echo "Data entered";
try
{
$ent = $dataService->get($testkey);
echo "Data queried - the results are \n";
echo "Field1 has value ".$ent->getProperty("field1")."\n";
echo "Field2 has value ".$ent->getProperty("field2")."\n";
}
catch(EntityNotFoundException $e)
{
echo("<br/>Entity test not found.");
echo("<br/>Stack Trace is:\n");
echo($e);
}
And here's the detailed stack-trace - link.
This same code runs fine in Java (of course after changing the syntax). I wonder what's wrong.
Thanks.
I have found the solution to my problem. It was caused by missing dependencies and I solved it by using the prepackaged PHP Wordpress application available here.
One thing is to be noted. The package overlooked a minor issue in that all files other than the src/ directory need to be in a war/ directory which stays alongside the src/ directory (this as per appengine conventions as mentioned on its documentation). So I organized the files thus myself, put the above PHP file in the war/ directory, and it's working fine on the appengine.