Example Databases - database

I am new to oracle. I wanted a huge sample database ( with a million tuples ) . I couldn't find any using google.
I'am using oracle 10g..
You guys know anywhere from where i can download?
Thank you ..

I don't know of any "ready-made" sample database of that size
As far as I can see, you have two options:
Use PolePosition to create a sample database. It's originally a benchmark frameworks but comes with it's own database schema and the necessary tools to generate a large database (you can define which size)
Use a test data generator like Benerator to completely create your test data from scratch. It seems a bit intimidating at first, but it's a really powerful tool. It also has generators to create meaningful names, zip codes and so on. So you'll get test data that "looks" real and doesn't contain gibberish.
The following benerator script generates a million rows for the table items and for each row in items it generates 10 rows in item_details (so you wind up with 1 million and 10 million rows)
<?xml version="1.0" encoding="iso-8859-1"?>
<setup xmlns="http://databene.org/benerator/0.7.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://databene.org/benerator/0.7.0 http://databene.org/benerator-0.7.0.xsd">
<import defaults="true"/>
<import platforms="csv"/>
<generate type="items" count="1000000">
<consumer class="org.databene.platform.csv.CSVEntityExporter">
<property name="uri" value="items.csv"/>
<property name="separator" value="|"/>
<property name="encoding" value="ISO-8859-1"/>
</consumer>
<id name="item_id" type="big_integer" generator="IncrementalIdGenerator"/>
<attribute name="item_name" type="string" pattern="[A-Z][a-z ]{6,25}"/>
<generate type="item_details" count="10">
<consumer class="org.databene.platform.csv.CSVEntityExporter">
<property name="uri" value="item_details.csv"/>
<property name="separator" value="|"/>
<property name="encoding" value="ISO-8859-1"/>
</consumer>
<id name="item_detail_id" type="big_integer" generator="IncrementalIdGenerator"/>
<attribute name="item_id" script="items.item_id"/>
<attribute name="sort_sequence" type="int" />
</generate>
</generate>
</setup>
If you want more "realistic" names, have a look a the following script which generates products with valid EAN Codes and some "normal" looking manufacturer names:
<?xml version="1.0" encoding="iso-8859-1"?>
<setup xmlns="http://databene.org/benerator/0.7.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://databene.org/benerator/0.7.0 http://databene.org/benerator-0.7.0.xsd">
<import platforms="csv"/>
<import domains="product"/>
<import domains="organization" />
<setting name="product_count" value="100000"/>
<generate type="product" count="{product_count}">
<consumer class="CSVEntityExporter">
<property name="uri" value="products.csv" />
<property name="separator" value=","/>
</consumer>
<id name="id" type="long"/>
<attribute name="ean_code" unique="true" generator="EANGenerator"/>
<attribute name="product_code" unique="true" pattern="[A-Z]{3}[0-9]{6}"/>
<variable name="cust" generator="CompanyNameGenerator" dataset="DE" locale="de_DE"/>
<attribute name="manufacturer_name" source="cust.shortName"/>
</generate>
</setup>
Once you have created the data files, you can use SQL*Loader to import them into the database.
This approach has the advantage that you have full control over the tables in your test database and you can tailor them to whatever you are trying to do with it.

Related

Composite index in datastore-index.xml not created

Here is my datastore-index file:
<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes autoGenerate="true">
<datastore-index kind="Stock" ancestor="false" source="manual">
<property name="code" direction="asc" />
<property name="endTime" direction="asc" />
</datastore-index>
</datastore-indexes>
The index entry was created exactly as suggested by App Engine in error log.
I put the index file in folder webapp/WEB-INF.
Despite all my efforts, no index were necer displayed on the Datastore index page and when I query I get the same error message: no matching index found.
Could you help me?

WSo2 Esb filtering messages to an output file Part 2

Ok, this is a continuation to my original question. (WSo2 Esb filtering messages to an output file)
After a couple more days of research into the iterate and aggregate mediator I have reached a progress wall and would be very grateful for any advice how to resolve my blocking issue. The task at hand is simple, read and xml file, filter on certain records, and only produce an output xml file with just those records that meet the criteria. After previous discussion and research this tasks solution is as follows: use a iterate mediator to access each xml record and use a filter mediator to decide what I want to keep. Then use a aggregator mediator to write all these records to 1 file. Without the aggregator my configuration is producing 1 record per file.
Here is my proxy at this state:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="RenFileFilterProxy"
transports="vfs"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<log level="full"/>
<clone>
<target sequence="RenIqtFilterSequence"/>
</clone>
</inSequence>
<outSequence>
<log level="custom">
<property name="sequence" value="In the outSequence"/>
</log>
<log level="full"/>
<aggregate id="QuizType">
<completeCondition>
<messageCount min="-1" max="-1"/>
</completeCondition>
<onComplete xmlns:z="RowsetSchema" expression="//z:row">
<call-template target="FileWriteTemplate">
<with-param name="targetFileName" value="TEST_FILE"/>
<with-param name="addressUri" value="vfs:file:///var/process/ren/rrout"/>
</call-template>
</onComplete>
</aggregate>
</outSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///var/process/ren/extractedfiles</parameter>
<parameter name="transport.vfs.FileURI">file:///var/process/ren/extractedfiles</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///var/process/ren/failure</parameter>
<parameter name="transport.vfs.FileNamePattern">test.xml</parameter>
<parameter name="transport.vfs.ContentType">application/xml</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<description/>
</proxy>
Here is the RenIqtFilterSequence:
<sequence xmlns="http://ws.apache.org/ns/synapse" name="RenIqtFilterSequence">
<log level="custom">
<property name="sequence" value="RenIqtFilterSequence"></property>
</log>
<iterate xmlns:ns2="http://org.apache.synapse/xsd" xmlns:ns="http://org.apache.synapse/xsd" xmlns:z="RowsetSchema" expression="//z:row" id="QuizType">
<target>
<sequence>
<log>
<property name="iteratesequence" value="Iterating through the the records"></property>
</log>
<filter xmlns:rs="urn:schemas-microsoft-com:rowset" xpath="//z:row/#name='RP'">
<then>
<log level="custom">
<property name="sequence" value="Condition Write"></property>
</log>
<log level="full"></log>
<log level="custom">
<property name="sequence" value="Getting ready to aggregate"></property>
</log>
</then>
<else>
<log level="custom">
<property name="sequence" value="Condition Drop"></property>
</log>
<drop></drop>
</else>
</filter>
</sequence>
</target>
</iterate>
</sequence>
The issue I am facing now is I cannot get to the outSequence to perform my aggregator logic. The fact is I do not need to call a backend service for this process from the inSequence so I am struggling to find a clean way to get to my outSequence. Any Recomendations? I tried to include the aggregator mediator in my inSequence but it did not work, no output is produced. I figured the aggregator needs to be in the outSequence to capture all the records I filtered and then I can write them all out to 1 file.
I believe I need some configuration here:
<filter xmlns:rs="urn:schemas-microsoft-com:rowset" xpath="//z:row/#name='RP'">
<then>
<log level="custom">
<property name="sequence" value="Condition Write"></property>
</log>
<log level="full"></log>
<log level="custom">
<property name="sequence" value="Getting ready to aggregate"></property>
</log>
===============> Need to transfer to OutSequence somehow here!!!!<============
</then>
<else>
I have tried several approaches based on articles and blogs online but I just cannot put the final pieces in place. Any advice would be much appreciated. Thank you for your time reading this.

create a new file from payload in wso2

I am new to wso2 and working on a few POCs where i have to create a file at some location , i have looked into all vfs examples where there is always a file processed and written to a new location.
What i want to achieve is write a new file to a directory by the content i receive in a sequence.
For my requirement process i have exposed a REST service and it calls this sequence.
The sequence configuration is as follows.
<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="FileWriteSequence">
<clone>
<target>
<sequence>
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<property name="transport.vfs.ReplyFileName" value="myOutputFile.txt" scope="transport" type="STRING"/>
<send>
<endpoint name="FileEpr">
<address uri="vfs:file://D:/Tools"/>
</endpoint>
</send>
</sequence>
</target>
</clone>
This sequence creates a file from the latest message from the REST resource but the file name is always the project name.
Whatever i try it doesnt change.
I have tried giving other proxy parameters shown below as property above my sequence as well .Instead of paramters i passed them above the property <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
Namely :-
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file://D:/Tools</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file://D:/backup</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
Still no progress.
Can anyone help me here?
Regards,
Rahul.
In case anyone wants to know i have made a workaround for this i called a proxy service from my rest resource and it worked .
In my resource i called endpoint like this
<send>
<endpoint key="FileProxyEndPt"/>
</send>
Then i created a proxy service as follows
<proxy xmlns="http://ws.apache.org/ns/synapse" name="FileWriteProxy"
transports="http https vfs" startOnLoad="true" trace="enable">
<target>
<inSequence>
<clone>
<target sequence="FileWriteSequence" />
</clone>
</inSequence>
<outSequence />
<faultSequence />
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file://D:/Tools</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file://D:/backup</parameter>
<parameter name="transport.vfs.ContentType">text/plain; charset=ISO-8859-1</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
This way it worked it but never worked exactly in a resource.So had to take a longer route , hope somebody provides a solution which can be implemented directly.

Configuring 2 database in mybatis + springs

I have an existing setup of mybatis + springs for working with Oracle DB. I have a set of java mapper interfaces, a set of corresponding mapper xmls (each having reference to their corresponding java mapper). I need to setup support for MSSQL also, but finding it hard to do that. I have created a separate set of xmls (specific to queries of MSSQL) in com/mycomp/mob/db/mappers/mssql.
Below is the extract of my applicationConext.xml (Here DBDataSource is an internal class which reads a config files to get all DB details).
<bean name="dataSource" class="com.mycomp.mob.core.db.DBDataSource">
<constructor-arg index="0" name="dbAlias" value="mob" />
<constructor-arg index="1" name="cfgSection" value="primary" />
</bean>
<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
<property name="basePackage" value="com.mycomp.mob.db.mappers.oracle" />
</bean>
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="typeAliasesPackage" value="com.mycomp.mob.db.mappers.oracle" />
<property name="configLocation" value="classpath:mybatis-config.xml" />
</bean>
<bean id="sqlSession" class="org.mybatis.spring.SqlSessionTemplate">
<constructor-arg index="0" ref="sqlSessionFactory" />
</bean>
Below is part of mybatis-config.xml (here ${} params are replaced using the earlier DBDatasource)
<configuration>
<typeAliases>
<package name="com.mycomp.mob.db.model" />
</typeAliases>
<environments default="development">
<environment id="development">
<transactionManager type="JDBC" />
<dataSource type="POOLED">
<property name="driver" value="${driver}" />
<property name="url" value="${url}" />
<property name="username" value="${username}" />
<property name="password" value="${password}" />
</dataSource>
</environment>
</environments>
<mappers>
<mapper resource="com/mycomp/mob/db/mappers/oracle/tenant.xml" />
<mapper resource="com/mycomp/mob/db/mappers/oracle/functionenty.xml" />
</mappers>
</configuration>
com.mycomp.mob.db.model contains POJO for tenant and functionentry.
Usage is as below :
ITenantMapper mappper = sqlSessionFactory.openSession().getMapper(ItenantMapper.class)
Tenant t = mapper.getTenant();
Now at a time only one DB (which is configured as primary in DB config file) will be used. So how to make sure that XML corresponding to that particular DB is invoked by the java mapper interface.
Also need to know can I configure same java mapper class name inside the mapper xmls of mssql ?
Is the flow used correct or I need to change the flow for supporting multiple databases.
Create two datasources and two SessionFactories in spring one for Oracle another for MSSQL.

Ant: Find the path of a file in a directory

I want to find the path of a file in a directory (similar to unix 'find' command or the 'which' command, but I need it to work platform-independent) and save it as a property.
Tried to use the whichresource ant task, but it doesn't do the trick (I think it's only good for looking inside jar files).
I would prefer if it would be pure ant and not to write my own task or use a 3rd-party extension.
Notice that there might be several instances of a file by that name in the path - I want it to only return the first instance (or at least I want to be able to choose only one).
Any suggestions?
One possibility is to use the first resource selector. For example to find a file called a.jar somewhere under directory jars:
<first id="first">
<fileset dir="jars" includes="**/a.jar" />
</first>
<echo message="${toString:first}" />
If there are no matching files nothing will be echoed, otherwise you'll get the path to the first match.
Here is an example which selects the first matching file. The logic is as follows:
find all matches using a fileset.
using pathconvert, store the result in a property, separating each matching file with line separator.
use a head filter to match the first matching file.
The functionality is encapsulated in a macrodef for reusability.
<project default="test">
<target name="test">
<find dir="test" name="*" property="match.1"/>
<echo message="found: ${match.1}"/>
<find dir="test" name="*.html" property="match.2"/>
<echo message="found: ${match.2}"/>
</target>
<macrodef name="find">
<attribute name="dir"/>
<attribute name="name"/>
<attribute name="property"/>
<sequential>
<pathconvert property="#{property}.matches" pathsep="${line.separator}">
<fileset dir="#{dir}">
<include name="#{name}"/>
</fileset>
</pathconvert>
<loadresource property="#{property}">
<string value="${#{property}.matches}"/>
<filterchain>
<headfilter lines="1"/>
</filterchain>
</loadresource>
</sequential>
</macrodef>
</project>
I created a macro based on martin-clayton's answer.
sample project with macro and a property file that is read from the found file
<?xml version="1.0" encoding="utf-8"?>
<project name="test properties file read" default="info">
<macrodef name="searchfile">
<attribute name="file" />
<attribute name="path" default="custom,." />
<attribute name="name" />
<sequential>
<first id="#{name}">
<multirootfileset basedirs="#{path}" includes="#{file}" erroronmissingdir="false" />
</first>
<property name="#{name}" value="${toString:#{name}}" />
</sequential>
</macrodef>
<searchfile name="custom.properties.file" file="config.properties" />
<property file="${custom.properties.file}" />
<target name="info" >
<echo>
origin ${config.origin}
</echo>
</target>

Resources