Same changelog on Postgres and MS SQL Server - sql-server

I am trying to make a changelog work on two different databases: MS SQL Server and PostgreSQL. The changelog works fine on SQL Server, but the case of the database and fields make it break on PostgreSQL. I have tried not using quotes around the values (which throws an error) and used the objectQuotingStategy="QUOTE_ALL_OBJECTS", both which didn't work.
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd">
<changeSet id="create-PushApplication" author="me">
<createTable tableName="PushApplication">
<column name="ApplicationKey" type="NUMERIC(19,0)" autoIncrement="true" remarks="Unique id of the application">
<constraints primaryKey="true" primaryKeyName="PushApplication_PK" nullable="false" />
</column>
<column name="ApplicationId" type="VARCHAR(100)" remarks="Id of the application in the Push Server">
<constraints unique="true" uniqueConstraintName="PushApplication_U1" nullable="false" />
</column>
<column name="MasterSecret" type="VARCHAR(100)" remarks="Password for the application in the Push Server">
<constraints nullable="false" />
</column>
<column name="Name" type="VARCHAR(100)" remarks="Name of the application">
<constraints nullable="false" />
</column>
<column name="Description" type="VARCHAR(200)" remarks="Description of the application" />
<column name="DateCreated" type="DATETIME" remarks="Date the application was created" />
</createTable>
</changeSet>
<changeSet id="create-PushVariant" author="me">
<createTable tableName="PushVariant">
<column name="VariantKey" type="NUMERIC(19,0)" autoIncrement="true" remarks="Unique id of the variant">
<constraints primaryKey="true" primaryKeyName="PushVariant_PK" nullable="false" />
</column>
<column name="VariantId" type="VARCHAR(100)" remarks="Id of the variant in the Push Server">
<constraints unique="true" uniqueConstraintName="PushVariant_UK1" nullable="false" />
</column>
<column name="Secret" type="VARCHAR(100)" remarks="Password for the variant in the Push Server">
<constraints nullable="false" />
</column>
<column name="Name" type="VARCHAR(100)" remarks="Name of the variant">
<constraints nullable="false" />
</column>
<column name="Description" type="VARCHAR(200)" remarks="Description of the variant" />
<column name="ApplicationKey" type="NUMERIC(19,0)" remarks="Id of the application the variant belogns to">
<constraints nullable="false" foreignKeyName="PushVariant_FK1" references="PushApplication(ApplicationKey)" />
</column>
</createTable>
</changeSet>
</databaseChangeLog>
ERROR: relation "pushapplication" does not exist
The table "PushApplication" is created in PostgreSQL but when it tries to create "PushVariant", this error is thrown for the foreign key.
This will work if I change all the database names and column names to lower case. However, this then makes the SQL Server have the incorrect case.
GOAL
The goal is to have "PushVariant" in SQL Server and "pushvariant" in PostgreSQL.
Is there a way that the case in the changelog can remain in SQL Server but have it lower case in PostgreSQL?

Supporting multiple DBMS with a common source code is always about compromises.
Even though there is a SQL standard that clearly defines how non-quoted identifiers have to be stored (all UPPERCASE), the two DBMS you are targeting both ignore this. Postgres stores un-quoted identifiers in lowercase, SQL Server is "case-preserving" (although not always case in-sensitive depending on the collation of the database).
In my personal experience when doing cross DBMS work, (always!) using un-quoted lower case identifiers with underscores is the least problematic way. And if you throw Oracle into the mix at some point in time you wind up all uppercase names.
Having said that: I think Liquibase's quoting strategy actually has a bug. According to the documentation, objectQuotingStrategy="QUOTE_ONLY_RESERVED_WORDS" should only quote reserved words, so in your example nothing should be quoted. But Liquibase still quotes any name that is using mixed case - this is still the case with the current 3.4.1 version.
I think the best thing was if Liquibase supported the choice objectQuotingStrategy="NEVER" (which it doesn't)
Another option would be to overwrite the default implementation on how the "need" for quoting is detected for Postgres and then use that implementation when running against the Postgres database.
The implementation is actually quite short and would only need to check for non-standard names (starting with a number, containing spaces or other illegal characters) and then quote only those.
I just tried this with the following implementation:
public class NonQuotingPostgresDatabase
extends PostgresDatabase {
#Override
public String correctObjectName(String objectName, Class<? extends DatabaseObject> objectType) {
return quoteIfNecessary(objectName);
}
#Override
public String escapeObjectName(String objectName, Class<? extends DatabaseObject> objectType) {
return quoteIfNecessary(objectName);
}
private String quoteIfNecessary(String objectName) {
if (requiresQuoting(objectName)) {
return "\"" + objectName + "\"";
}
return objectName;
}
protected boolean requiresQuoting(String identifier) {
if (identifier == null) {
return false;
}
if (isQuoted(identifier)) {
return false;
}
return (identifier.contains("-") || startsWithNumeric(identifier) || isReservedWord(identifier));
}
protected boolean isQuoted(String identifier) {
if (identifier == null) {
return false;
}
return (identifier.startsWith("\"") && identifier.endsWith("\""));
}
}
That class essentially leaves quoted identifiers alone and quotes only identifiers that really require it. The big difference to the built-in class is that it doesn't check for mixed case identifiers.
If this is put into a .jar file and the put into the lib subfolder of the Liquibase distribution it will be picked up automatically. All you need to do is add the parameter --databaseClass=NonQuotingPostgresDatabase when running Liquibase against Postgres

Related

I can't change range and/or default values in DevExpress Dashboard

I have a page where users register and can make their personal diet plan etc.
I am trying to make some graphs with built-in VS dashboards.
I am using VS 2015 and SQL Server Management Studio 15.0.18206.0 if that matters.
My problems are the following:
Automatic (SUM)
Dashboard values default to SUM function, which I am not able to change, as you see in the first picture, but I kind of managed to solve this problem with the help of SQL (SELECT DISTINCT, SELECT TOP(1) etc.)
Automatic (SUM)
Default year count
When I am trying to make a dashboard to show track of user's vitals and activity, which is the core of my statistics, time defaults to YEAR and the result end up like the second picture.
Default year count
There is nothing I can do to change these and I just made irrelevant graphs to fill the page, but it's a shame because I planned to make a lot of graphs with the table you see in the third picture.
User History table
If it helps, the XML code generated from the last dashboard is this:
<?xml version="1.0" encoding="utf-8"?>
<Dashboard>
<Title Text="YearDashBoard" />
<DataSources>
<SqlDataSource ComponentName="DataSource1">
<Name>UserHistory1</Name>
<Connection Name="foodConnectionString" FromAppConfig="true" />
<Query Type="SelectQuery" Name="UserHistory1" Distinct="true">
<Tables>
<Table Name="UserHistory" />
</Tables>
<Columns>
<Column Table="UserHistory" Name="CaloriesDate" />
<Column Table="UserHistory" Name="DailyCalories" />
<Column Table="UserHistory" Name="UserID" />
</Columns>
<Filter>[UserHistory.UserID] = 17</Filter>
</Query>
<ConnectionOptions CloseConnection="true" CommandTimeout="0" />
</SqlDataSource>
</DataSources>
<Items>
<Chart ComponentName="chartDashboardItem1" Name="Chart 1" DataSource="DataSource1" DataMember="UserHistory1">
<DataItems>
<Measure DataMember="DailyCalories" DefaultId="DataItem0" />
<Dimension DataMember="CaloriesDate" DefaultId="DataItem1" />
</DataItems>
<Arguments>
<Argument DefaultId="DataItem1" />
</Arguments>
<Panes>
<Pane Name="Pane 1">
<Series>
<Simple>
<Value DefaultId="DataItem0" />
</Simple>
</Series>
</Pane>
</Panes>
</Chart>
</Items>
<LayoutTree>
<LayoutGroup Weight="100">
<LayoutItem DashboardItem="chartDashboardItem1" Weight="100" />
</LayoutGroup>
</LayoutTree>
</Dashboard>
Finally, with the help of DevExpress technical team, I found out that the whole time there was an invisible scroll bar, but you had to mouse over it (!) while being on the "BINDING" tab of the menu.

custom table doesnt exist in Moodle

I am building a block moodle plugin. For the plugin I have created three tables:
'block_learning_strategizer':contains just one field-id
'ls_basic': containing 3 fields-id,lp_name,description
'ls_path_details': containing 9 fields.
The definition is done through install.xml(under blocks/learning_strategizer/db)
The XML is as below:
<?xml version="1.0" encoding="UTF-8" ?>
<XMLDB PATH="blocks/learning_strategizer/db" VERSION="20120122" COMMENT="XMLDB file for Moodle blocks/learning_strategizer"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="../../../lib/xmldb/xmldb.xsd"
>
<TABLES>
<TABLE NAME="block_learning_strategizer" COMMENT="Default for block_learning_strategizer" NEXT="ls_basic">
<FIELDS>
<FIELD NAME="id" TYPE="int" LENGTH="10" NOTNULL="true" SEQUENCE="true"/>
</FIELDS>
<KEYS>
<KEY NAME="primary" TYPE="primary" FIELDS="id"/>
</KEYS>
</TABLE>
<TABLE NAME="ls_basic" COMMENT="Table contains name and description of learning paths" NEXT="ls_path_details">
<FIELDS>
<FIELD NAME="id" TYPE="int" LENGTH="10" NOTNULL="true" SEQUENCE="true"/>
<FIELD NAME="lp_name" TYPE="char" LENGTH="255" NOTNULL="true" SEQUENCE="false" PREVIOUS="id" NEXT="description"/>
<FIELD NAME="description" TYPE="text" NOTNULL="false" SEQUENCE="false" PREVIOUS="lp_name"/>
</FIELDS>
<KEYS>
<KEY NAME="primary" TYPE="primary" FIELDS="id"/>
</KEYS>
</TABLE>
<TABLE NAME="ls_path_details" COMMENT="Table contains details of created learning paths" PREVIOUS="ls_basic">
<FIELDS>
<FIELD NAME="id" SEQUENCE="true" TYPE="int" NOTNULL="true" LENGTH="10" NEXT="lp_id"/>
<FIELD NAME="lp_id" SEQUENCE="false" TYPE="int" LENGTH="10" NOTNULL="true" NEXT="id" PREVIOUS="course" />
<FIELD NAME="course" SEQUENCE="false" TYPE="int" LENGTH="10" NOTNULL="true" NEXT="section" PREVIOUS="lp_id"/>
<FIELD NAME="section" SEQUENCE="false" TYPE="int" NOTNULL="true" LENGTH="10" NEXT="req" PREVIOUS="course"/>
<FIELD NAME="req" SEQUENCE="false" TYPE="int" NOTNULL="true" LENGTH="2" NEXT="inc" PREVIOUS="section"/>
<FIELD NAME="inc" SEQUENCE="false" TYPE="int" NOTNULL="true" LENGTH="2" NEXT="modid" PREVIOUS="req"/>
<FIELD NAME="modid" SEQUENCE="false" TYPE="int" NOTNULL="true" LENGTH="3" NEXT="seqno" PREVIOUS="inc"/>
<FIELD NAME="seqno" SEQUENCE="false" TYPE="int" NOTNULL="true" LENGTH="10" NEXT="filename" PREVIOUS="modid"/>
<FIELD NAME="filename" SEQUENCE="false" TYPE="text" NOTNULL="true" LENGTH="255" PREVIOUS="seqno"/>
</FIELDS>
<KEYS>
<KEY NAME="primary" TYPE="primary" FIELDS="id" NEXT="lp_id"/>
<KEY NAME="lp_id" TYPE="foreign" FIELDS="lp_id" REFTABLE="ls_basic" REFFIELDS="id" PREVIOUS="primary" />
</KEYS>
</TABLE>
</TABLES>
</XMLDB>
However when I am trying to insert records I am getting an error :"Table "ls_basic" does not exist"
I checked XMLDB editor from Site administration and I could see the tables have been made.
I havent included an upgrade.php but as far as I know that file is optional.
It would be really helpful if someone could point out why do i get this error?
When a plugin is first installed, Moodle parses the install.xml file and uses this to create the database tables required for your plugin.
After the first installation of the plugin, Moodle does not look at the install.xml file again. Instead it relies on checking the upgrade.php file, at the point where your plugin's version number (in version.php) changes, in order to find out how to transform the previous database structure to match the new structure.
If your plugin is still under local development, you can get Moodle to re-parse the install.xml file by using the 'uninstall' feature in the 'Plugins' part of the 'Site administration' area. This will remove all the data for the plugin, then, if the code for the plugin still exists on the server, will immediately offer to re-install the plugin (which will create all the tables in install.xml).
If your plugin is already in use, or you do not want to lose any existing data, then you will need to use the XMLDB editor to generate the relevant lines of code to go in your upgrade.php file (and increase your plugin's version number to match).
See https://docs.moodle.org/dev/Upgrade_API for more details.
I would also suggest that this is a good time to fix your database tables to match the Moodle coding guidelines:
Variable names should not have _ characters in them - this applies to database field names as well (although _ in database table names are fine).
Plugin database tables should all start with the name of the plugin ('block_learning_stategizer' in this case) - if you end up using Travis CI to automatically check your plugins, then it will complain about your database table names.
Plugin names are strongly discouraged from having _ in them (other than between the plugin type and the rest of the name) - there have been a number of bugs over the years caused by Moodle core getting stuck on names that break this rule. It may be a good idea to rename your plugin 'block_learningstrategizer' now, before you hit any problems.

Example Databases

I am new to oracle. I wanted a huge sample database ( with a million tuples ) . I couldn't find any using google.
I'am using oracle 10g..
You guys know anywhere from where i can download?
Thank you ..
I don't know of any "ready-made" sample database of that size
As far as I can see, you have two options:
Use PolePosition to create a sample database. It's originally a benchmark frameworks but comes with it's own database schema and the necessary tools to generate a large database (you can define which size)
Use a test data generator like Benerator to completely create your test data from scratch. It seems a bit intimidating at first, but it's a really powerful tool. It also has generators to create meaningful names, zip codes and so on. So you'll get test data that "looks" real and doesn't contain gibberish.
The following benerator script generates a million rows for the table items and for each row in items it generates 10 rows in item_details (so you wind up with 1 million and 10 million rows)
<?xml version="1.0" encoding="iso-8859-1"?>
<setup xmlns="http://databene.org/benerator/0.7.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://databene.org/benerator/0.7.0 http://databene.org/benerator-0.7.0.xsd">
<import defaults="true"/>
<import platforms="csv"/>
<generate type="items" count="1000000">
<consumer class="org.databene.platform.csv.CSVEntityExporter">
<property name="uri" value="items.csv"/>
<property name="separator" value="|"/>
<property name="encoding" value="ISO-8859-1"/>
</consumer>
<id name="item_id" type="big_integer" generator="IncrementalIdGenerator"/>
<attribute name="item_name" type="string" pattern="[A-Z][a-z ]{6,25}"/>
<generate type="item_details" count="10">
<consumer class="org.databene.platform.csv.CSVEntityExporter">
<property name="uri" value="item_details.csv"/>
<property name="separator" value="|"/>
<property name="encoding" value="ISO-8859-1"/>
</consumer>
<id name="item_detail_id" type="big_integer" generator="IncrementalIdGenerator"/>
<attribute name="item_id" script="items.item_id"/>
<attribute name="sort_sequence" type="int" />
</generate>
</generate>
</setup>
If you want more "realistic" names, have a look a the following script which generates products with valid EAN Codes and some "normal" looking manufacturer names:
<?xml version="1.0" encoding="iso-8859-1"?>
<setup xmlns="http://databene.org/benerator/0.7.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://databene.org/benerator/0.7.0 http://databene.org/benerator-0.7.0.xsd">
<import platforms="csv"/>
<import domains="product"/>
<import domains="organization" />
<setting name="product_count" value="100000"/>
<generate type="product" count="{product_count}">
<consumer class="CSVEntityExporter">
<property name="uri" value="products.csv" />
<property name="separator" value=","/>
</consumer>
<id name="id" type="long"/>
<attribute name="ean_code" unique="true" generator="EANGenerator"/>
<attribute name="product_code" unique="true" pattern="[A-Z]{3}[0-9]{6}"/>
<variable name="cust" generator="CompanyNameGenerator" dataset="DE" locale="de_DE"/>
<attribute name="manufacturer_name" source="cust.shortName"/>
</generate>
</setup>
Once you have created the data files, you can use SQL*Loader to import them into the database.
This approach has the advantage that you have full control over the tables in your test database and you can tailor them to whatever you are trying to do with it.

Symfony Propel Many-To-Many Add

I have e functional Symfony Project. I need to add a Many-To-Many relationships in the project.
I have jobs table and cities table. How can I add this relantionships using Generator (I dont want to alter existing data)?
I will use this relationship in Jobs forms (add,edit,list) to assign multiple Cities to a Job.
You will need a cross reference table sitting between your Jobs and Cities:
<table name="job_city" isCrossRef="true">
<column name="job_id" type="INTEGER" primaryKey="true" />
<column name="city_id" type="INTEGER" primaryKey="true" />
<foreign-key foreignTable="job">
<reference local="job_id" foreign="id" />
</foreign-key>
<foreign-key foreignTable="city">
<reference local="city_id" foreign="id" />
</foreign-key>
</table>
After regenerating you will have a new set of classes: JobCity, JobCityQuery, and JobCityPeer to use. You can add a new job to a city like so:
$job = new Job();
// add job info ...
$job->save();
$city = CityQuery::create()->findOneByName("Austin");
$city->addJob($job);
$city->getJobs(); // returns PropelObjectCollection of Job objects

NHibernate YesNo on SQL-Server CE No mapping exists from DbType AnsiStringFixedLength

Trying to use nhibernate on sql-server compact edition for unit testing and having some trouble. At the moment I just have one entity which is:
<class name="Audit" table="eolaudit_llk">
<id name="ID" column="eolauditlk_ky">
<generator class="identity"></generator>
</id>
<property name="Name" column="eolauditlk_nm" />
<property name="Description" column="eolauditlk_dn" />
<property name="Active" column="active_fg" type="YesNo" />
<property name="UpdateUser" column="update_user_id" />
<property name="UpdateDateTime" column="update_dm"/>
I'm pretty new to nhibernate. I'm using the following code to build the session:
public static ISessionFactory CrashAndBurnCompactSQLSessionFactory(string ConnectionString)
{
return Fluently.Configure()
.Database(MsSqlCeConfiguration.Standard
.ConnectionString(ConnectionString)
.ShowSql()
.UseOuterJoin()
.QuerySubstitutions("true 1, false 0, yes 'Y', no 'N'")
.Dialect("NHibernate.Dialect.MsSqlCeDialect")
)
.Mappings(m =>
{
m.HbmMappings
.AddFromAssemblyOf<Audit>();
})
.ExposeConfiguration(BuildSchema)
.BuildSessionFactory();
}
It works fine with SQL-Server, but when I use the SQL-Server CE session from above I get the following error:
failed: System.ArgumentException: No mapping exists from DbType AnsiStringFixedLength to a known SqlDbType. at System.Data.SqlServerCe.SqlCeType.FromDbType(DbType type)
at System.Data.SqlServerCe.SqlCeParameter.set_DbType(DbType value)
This is a head-melter - I don't want to do my testing on sql-server, and can't change the db. It's definitley related that the handling of that YesNo type which maps from a 'Y' or'N' char(1) db field to a boolean in the object.
As I said, the thing works fine in SQL-Server.
SQL Server CE doesn't support DbType.AnsiStringFixedLength (see here). Unfortunately YesNo derives from this DbType.
You could write your own YesNo custom type, or use type Char (it maps to DbType.StringFixedLength, which is supported by both versions of SQL Server). Neither option is particularly appealing. I know you said this isn't an option, but...if you are targeting SQL Server, it might be worth re-evaluating your strategy of testing on CE. Why not SQL Server Express, for example?

Resources