Have a question regarding the Snowflake External Table. As we now the External Tables still a preview feature and still not a General Available. Hence have guys used External table in your production environment and you see any issue ? Snowflake do not recommend to use Preview feature in Production. Any thoughts?
External table feature is publicly available feature as per documentation
https://docs.snowflake.com/en/user-guide/tables-external-intro.html
Related
I am using eclipse link jpa 2.0. I am using hana database. So I created views and generated java entity files using eclipse. The problem is, after starting Java Web application in Tomcat server 7, if any table data is modified, then view is not returning updated values. Even if I am running view with native query it is giving old values only.. please let me know what changes need to do in configuration level or entity creating level. ( I even added #Cacheable(false) also).
Assuming that when you say view you mean database views, perhaps the following existing answered question might help (though the link talks about Oracle instead):
Materialized View - Oracle / Data is not updating
You might be using some sort of materialized view made in the database that can be configured to refresh during certain events. In this case, the problem does not lie in Eclipselink's caching mechanisms, but in your database instead (as you mentioned that even native queries returned stale data).
I got connectors from https://cloud.google.com/hadoop/datastore-connector
But I'm trying to add the datastore-connector (and bigquery-connector too) as a dependency in the pom... I don't know if it this is possible. I could not find the right artifact and groupId.
Is there some maven repository that contain the datastore-connector?
Furthermore, I am looking for the source of datastore-connector, but I didn't find it. By the notes in the CHANGES.txt, it seems to be coming from:
https://github.com/GoogleCloudPlatform/bigdata-interop
The source should be in the package com.google.cloud.hadoop.io.datastore (src/main/***/com/google/cloud/hadoop/io/datastore/) but it's not there.
In fact, the source of bigquery-connector appears to be on GitHub along with its pom, but is the source of datastore-connector available?
What David says in the other answer is correct. To elaborate more, the connector under the hood uses the Protocol Buffers SDK, and uses, for example, the QuerySplitter to define splits. In the near future, we will be posting more information to gcp-hadoop-announce with further guidance regarding the future of the Datastore connector for Hadoop.
You may want to familiarize yourself with other Datastore features that may suit your purposes better, including Datastore backup to GCS, and this codelab walking through an AppEngine-friendly approach to extracting data from Datastore and loading it into BigQuery for analysis. You may notice at the top of that page an announcement of trusted-tester availability for direct backend loading of Datastore backups into BigQuery.
The datastore-connector source is not available, nor is there a maven repo with the artifact. Your best option is to create a local raven repo in your source tree as described in this helpful article.
Amazon provides a batch of documents describing the format of the feeds we can send via MWS, however, we also need to know what to expect in their responses, what status codes may be reported or what is the structure of XML when errors reported, etc...
Where can I get the information?
The MWS XML schemata are documented within the Selling on Amazon Guide to XML linked from the Developer Guides section in the Amazon Marketplace Web Service (Amazon MWS) Documentation.
I'm omitting a direct link to the PDF, as this might change once in a while. For the same reason the XSD files you are looking for are not publicly linked by Amazon as well, rather you'll find the links to the most current schema documents within the respective sections of the Selling on Amazon Guide to XML.
You might also be interested in the Amazon MWS Developer Guide, the Feeds API Reference and the guide for the Amazon MWS Scratchpad, which are all available there as well.
Good luck!
I know this is a rather old question but I just wanted to look at the actual XML schema files myself today.
There is an XML Documentation PDF hosted on images-na.ssl-images-amazon.com which I assume will stay there for a while. This PDF contains links to the core schema files amzn-envelope.xsd, amzn-header.xsd, and amzn-base.xsd and some other API schemas like Product.xsd which all appear to be relative to https://images-na.ssl-images-amazon.com/images/G/01/rainier/help/xsd/release_1_9/.
The PDF explicitly states that
The XSD samples shown on the Help pages may not reflect the latest XSDs. We recommend
using the provided XSD links to obtain the latest [ve]rsions.
However, the official MWS Feeds API documentation also links to some XSDs but these are relative to https://images-na.ssl-images-amazon.com/images/G/01/rainier/help/xsd/release_4_1/ now, e.g. Price.xsd. Schema references also seem to be relative to this path. For example, Price.xsd includes amzn-base.xsd via <xsd:include schemaLocation="amzn-base.xsd"/> and sure enough there it is.
Unfortunately, I have no idea whether release_4_1 is the latest release of the schemas but the link from the MWS API documentation is a good indicator to me.
Another way to get the XSD's which I think is the most "official" way is to go to your Seller Central and navigate to Help > XML & data exchange > Reference > XSDs.
There you can download all the XSD's available to your account.
Hope it helps!
It seems that this XSD files are outdated.
Just checked the official sellercentral help page for the XSD files https://sellercentral-europe.amazon.com/gp/help/G1611
For the OrderReport there is still release_4_1 referenced.
Some time ago amazon has added a new field to OrderReport for EU markets. The new field is IsSoldByAB.
I am using the xsd files since many years for automatic code generation. And this fails from time to time because of new fields like this. This field is not descriped in one of this:
release_1_9 ($Revision: #7 $, $Date: 2006/05/23 $)
release_4_1 ($Revision: #10 $, $Date: 2007/09/06 $)
XSD files and I am not able to find a version that include this field.
Since some years I extend the XSD file on my own to generate my code. IsSoldByAB is just a boolean field as IsPrime or IsBusinessOrder. So this was an easy task but not "official"...
I am trying to making a tool which can help in maintaining data base version(like maintaining source code version). The technology which I am thinking to use is spring-hibernate so that the tool can be web based and it can be used by multiple project . The idea is that any database change can only be triggered with the help of this tool,so that the database version information can be maintained and the database can be made consistent .Operations like commit,roll back,branching,merging should be possible. Can you suggest me that how should I approach to this problem?
I have found an opensource tool called LiquidBase which has already provided some sort of solution in maintaining database version. Here is a short preview on what this tool can do. But this tool has some limitations like it does not handle stored procedures and triggers and it works on the basis of an XML file . But I think I can integrate this tool with my requirement and I can speed up development. If you have any other tool in knowledge which can be better than this then please let me know.
If possible tell me that how the tool should be organized so that different project can easily maintain their database version. What all problem the tool should try to address and what minimum support should at least be there in this tool? What should be the UI so that user should be easily able to use it.?
There are at least two Grails plugins that emulate the database migration functionality of Rails:
Autobase
Liquibase
Is there a consensus about which of these is best, or is there another plugin that does database migration better than either of the above?
There is now a standard Grails database migration plugin available. According to this blog post at least the liquibase plugin will therefore not be maintained past the liquibase 1.9 release anymore.
The new database migration plugin has built-in functionality to execute changelogs on startup and supports the definition of changes in Groovy DSL, so it's probably what you are looking for.
I use Autobase (which is built on top of Liquibase) as it (last time I checked) allows you to automatically check/apply your migrations when the app starts. With the Liquibase plugin I have to do this myself in servlet init code. This allows you to set your datasource to dbCreate = none and let Autobase handle getting the DB into shape.
It does mean you need to write a migration each time you add a property to a domain class, but I think this is a good thing as it makes you think about what the underlying field should actually be instead of just letting Hibernate take a guess at it.
I think some of the Autobase plugin (e.g. the groovy dsl) is being migrated back to the Liquibase plugin, but you'd need to check up on that.
The only downside to Autobase is the lack of good documentation. There is some but it's not complete. Luckily, the dsl is the same as the xml Liquibase tags so you can work most of it out.
I use liquibase, I'm not sure that Robert is still actively maintaining Autobase and the xml that liquibase provides is actually pretty DSL-like. I think it also gives a little bit of separation to your database commands and doesn't make it ingrained into the start-up process (some people might prefer the reverse).
At least as of Grails2.0, the database migration plugin is the defacto way to handle non-trivial database changes. The plugin is built on Liquibase, and is authored by the Springsource folks - always a mark of quality. I wrote an introduction to the database migration plugin which might be of use to someone reading this.
I have heard that Autobase is still maintained, but consider that the Grails Database Migration Plugin is written by the core team, and likely going to be the officially supported one.
In other words, encourage you can wait on Grails 1.4 --> roadmap before choosing either of the plugins above.
YEs i also see the migration pluging. This is helpful...
http://grails-plugins.github.io/grails-database-migration/