The documentation about datastore config objects confuses me:
"A configuration object can be used any number of times. You must create a separate configuration object for each datastore call that uses it."
(from AppEngine doc)
So can I do something like this:
config = db.create_config(deadline=5)
db.put(someModels, config=config)
db.delete(someKeys, config=config)
Or do I have to do something like this:
config = db.create_config(deadline=5)
db.put(someModels, config=config)
config = db.create_config(deadline=5)
db.delete(someKeys, config=config)
?
Thanks
That is a left-over from when config options were changed by creating a RPC. Each RPC could be used only once. The new datastore Configuration objects can be used multiple times; parameters are now read from them and passed on.
For reference, when settings were passed by creating RPC objects the docs read:
An RPC object can only be used once. You must create a separate RPC object for each datastore call that uses it.
Related
Apache camel components page have fields under option and query parameters but no clear path position for parameters , from samples I was able to make out they go along options .
I would like to know the difference between options and query parameters.
When an application with Apache Camel starts, it registers the routes in the Camel Context, and once the context is started, components present in from () and to () cannot be modified, for example:
String param = "a = xxxx & y = bbb";
...
to ("http4: //api.xxx.yy?" + stop)
...
It will only be evaluated at startup, so even if the value of the string in the param variable changes, the route will always use a=xxxx&y=bbb as the default since it has already been initialized in the context of Camel (you can see Camel recording the routes in the logs during application startup).
The options can undergo changes not only in construction, depending on the design of the component in question, but can also be exposed for configuration via starters using application.yml or application.properties or via java as in the example below:
application.properties
camel.component.http.http-configuration=br.com.pack.impl.MyHttpConfiguration
In java
HttpConfiguration config = new HttpConfiguration();
config.setProxyAuthMethod("Digest");
config.setProxyAuthUsername("myUser");
config.setProxyAuthPassword("myPassword");
HttpComponent http = context.getComponent("http", HttpComponent.class);
http.setHttpConfiguration(config);
from("direct:start")
.to("http4://www.google.com/search");
I hope it helped to clarify a little more
Options are used to configure Component and Query Parameters are used while creating endpoints.
I don't see a solution to this using the available api documentation.
It is also not available on the web console.
Is it possible to get the file url using the Watson Discovery Service?
If you need to store the original source/file URL, you can include it as a field within your documents in the Discovery service, then you will be able to query that field back out when needed.
I also struggled with this request but ultimately got it working using Python bindings into Watson Discovery. The online documentation and API reference is very poor; here's what I used to get it working:
(Assume you have a Watson Discovery service and have a created collection):
# Programmatic upload and retrieval of documents and metadata with Watson Discovery
from watson_developer_cloud import DiscoveryV1
import os
import json
discovery = DiscoveryV1(
version='2017-11-07',
iam_apikey='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
url='https://gateway-syd.watsonplatform.net/discovery/api'
)
environments = discovery.list_environments().get_result()
print(json.dumps(environments, indent=2))
This gives you your environment ID. Now append to your code:
collections = discovery.list_collections('{environment-id}').get_result()
print(json.dumps(collections, indent=2))
This will show you the collection ID for uploading documents into programmatically. You should have a document to upload (in my case, an MS Word document), and its accompanying URL from your own source document system. I'll use a trivial fictitious example.
NOTE: the documentation DOES NOT tell you to append , 'rb' to the end of the open statement, but it is required when uploading a Word document, as in my example below. Raw text / HTML documents can be uploaded without the 'rb' parameter.
url = {"source_url":"http://mysite/dis030.docx"}
with open(os.path.join(os.getcwd(), '{path to your document folder with trailing / }', 'dis030.docx'), 'rb') as fileinfo:
add_doc = discovery.add_document('{environment-id}', '{collections-id}', metadata=json.dumps(url), file=fileinfo).get_result()
print(json.dumps(add_doc, indent=2))
print(add_doc["document_id"])
Note the setting up of the metadata as a JSON dictionary, and then encoding it using json.dumps within the parameters. So far I've only wanted to store the original source URL but you could extend this with other parameters as your own use case requires.
This call to Discovery gives you the document ID.
You can now query the collection and extract the metadata using something like a Discovery query:
my_query = discovery.query('{environment-id}', '{collection-id}', natural_language_query="chlorine safety")
print(json.dumps(my_query.result["results"][0]["metadata"], indent=2))
Note - I'm extracting just the stored metadata here from within the overall returned results - if you instead just had:
print(my_query) you'll get the full response from Discovery ... but ... there's a lot to go through to identify just your own custom metadata.
In Fusion's Javascript Indexing Stage, we can import Java classes and run them in the javascript such as this :
var imports = new JavaImporter(java.lang.String);
with (imports) {
var name = new String("foo"); ...
}
If we have customized complex Java classes, how to include the compile jar with Fusion so that the class can be imported in Javascript Indexing Stages for use?
And where can we store configuration values for the Javascript Indexing Stage to look up and how to retrieve them?
I'm thinking of something like this:
var imports = new JavaImporter(mycompany.com.custompkg.SomeParser);
with (imports) {
var some_config = ResourceManager.GetString("key");
var sp = new SomeParser(some_config); ...
}
Regards,
Kelvin
Starting in Fusion 4.x The API and Connectors started using a common location for jars i.e. apps/libs . This is a reasonable place to put custom jars but the services must be told about the new jars as well. That's done in two places
/jetty/connectors-classic/webapps/connectors-extra-classpath.txt
./jetty/api/webapps/api-extra-classpath.txt
Also, index documents can get processed by the api service so even if the jar is only used for indexing, register with both classpaths. Finally, bounce the services.
Put the Java class file, as a jar file, in $FUSION_HOME/apps/jetty/api/webapps/api/WEB-INF/lib/.
I used this to access my custom class.
var SomeParser = Java.type('mycompany.com.custompkg.SomeParser');
I'm migrating from the original file- based datastore to the sqlite version.
I have a command line script which initialises the stub as follows:
from google.appengine.api import apiproxy_stub_map
from google.appengine.datastore.datastore_sqlite_stub import DatastoreSqliteStub
apiproxy_stub_map.apiproxy=apiproxy_stub_map.APIProxyStubMap()
apiproxy_stub_map.apiproxy.RegisterStub("datastore_v3", DatastoreSqliteStub("myapp", Datastore, "/"))
Querying the datastore raises NeedIndexError; however -
the relevant index definitions are staring me in the face in index.yaml
there was no problem accessing the old file- based datastore [using DatastoreFileStub]
Am I somehow failing to initialise the datastore with index.yaml ?
The constructor arguments DatastoreSqliteStub takes are:
app_id,
datastore_file,
require_indexes=False,
verbose=False,
service_name='datastore_v3',
trusted=False,
consistency_policy=None
By providing those named arguments, you're specifying the app ID (correctly), the datastore file, which you've specified is some object called Datastore, and whether or not to require indexes (which you've set to '/', which evaluates to True). Instead, just specify the first and third arguments.
I am trying to use post_import_function while uploading data using bulkloader.yaml. As per this link, Using post_import_function in App Engine bulkuploader yaml , I am using type google.appengine.api.datastore.Entity for entity operations. As in the link, this is a subclass of 'dict'. However I am not sure how do I apply methods to this entity.
My code looks like (I am using Geomodel):
def post_import_restaurants(input_dict, instance, bulkload_state_copy):
lat = instance['lat']
lng = instance['lng']
latlng = "%s,%s" % (lat,lng)
instance['location'] = db.GeoPt(latlng)
instance.update_location()
return instance
instance.update_location(), is where I am having issues. And I am not sure how to write this statement.
You can't add methods to an Entity. Just inline the code, or write it as a separate function that you pass the entity to.