how to index files over ftp ,
the FTP repo contain all my documents in different format, i am able to do this task for system folder but it doesn't work with ftp
i have this configuration via (DIH)
<dataConfig>
<dataSource type="BinFileDataSource" />
<dataSource type="BinURLDataSource" name="binSource" baseUrl="ftp://localhost:21/" onError="skip" user="solr_ftp" password="solr_ftp_pass" />
<document>
<!-- baseDir: path to the folder that containt the files (pdf | doc | docx | ...) -->
<entity name="files" dataSource="binSource" baseDir="ftp://localhost" rootEntity="false" processor="FileListEntityProcessor" fileName=".*\.(doc)|(pdf)|(docx)|(txt)|(rtf)|(html)|(htm)" onError="skip" recursive="true">
<field column="fileAbsolutePath" name="filePath" />
<field column="resourceName" name="resourceName" />
<field column="fileSize" name="size" />
<field column="fileLastModified" name="lastModified" />
<!-- tika -->
<entity name="documentImport" processor="TikaEntityProcessor" url="${files.fileAbsolutePath}" format="text">
<field column="title" name="title" meta="true"/>
<field column="subject" name="subject" meta="true"/>
<field column="description" name="description" meta="true"/>
<field column="comments" name="comments" meta="true"/>
<field column="Author" name="author" meta="true"/>
<field column="Keywords" name="keywords" meta="true"/>
<field column="category" name="category" meta="true"/>
<field column="xmpTPg:NPages" name="Page-Count" meta="true"/>
<field column="text" name="content"/>
</entity>
</entity>
</document>
</dataConfig>
Error:
failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir' value: ftp://localhost is not a directory Processing Document # 1
I want to have in generated by Propel objects function e.g. getUsersPrivileges(). I have two tables Users and Privileges described as;
<table name="users" phpName="User">
<column name="user_id" type="integer" required="true" autoIncrement="true" primaryKey="true" />
<column name="login" type="longvarchar" required="true" />
<column name="password" type="longvarchar" required="true" />
<column name="email" type="longvarchar" required="true" />
<column name="language" type="longvarchar" required="true" defaultValue="en" />
<column name="salt1" type="longvarchar" required="true" />
<column name="salt2" type="longvarchar" required="true" />
<column name="date_created" type="timestamp" required="true" />
<column name="archived" type="integer" required="true" defaultValue="0" />
</table>
<table name="privileges" phpName="Privilege">
<column name="privilege_id" type="integer" required="true" autoIncrement="true" primaryKey="true" />
<column name="name" type="longvarchar" required="true" />
<column name="description" type="longvarchar" required="true" />
</table>
<table name="users_privileges" phpName="UserPrivilege" isCrossRef="true">
<column name="user_privilege_id" type="integer" required="true" autoIncrement="true" unique="true" primaryKey="true" />
<column name="user_id" type="integer" />
<column name="privilege_id" type="integer" />
<column name="archived" type="integer" required="true" defaultValue="0" />
<foreign-key foreignTable="users">
<reference local="user_id" foreign="user_id"/>
</foreign-key>
<foreign-key foreignTable="privileges">
<reference local="privilege_id" foreign="privilege_id"/>
</foreign-key>
</table>
I want to have a user_privilege_id as primary key in SQL schema, but Propel requires to generated this method that columns user_id and privilege_id as primary key and foreign key to users.user_id and privilege_id to privileges.privilege_id. It is good to have any additional column in this table.
How to tell Propel to join these tables through these columns and no primary key?
It doesn't work. PrimaryKeys are the only way Propel knows which relations from left to right should created. If you add additional PKs it will create different relation methods. If you have a completely different PK Propel is unable to check which outgoing relations from the cross-table needs to be cross linked from user to privileges and vice-versa. If you really need a ID, you can add this id column as normal with a unique constrain and auto-increment.
Or alternative, you don't use isCrossRef and write your relation methods from user to privileges and vice-versa on your own.
This is my xml
DECLARE #XMLValues XML
SET #XMLValues = '<?xml version="1.0" encoding="UTF-8"?>
<DOCUMENTS name="NYSPIT">
<DOCUMENT ID="140208512T200911101">
<REPEATS>
<REPEAT NAME="EXCEPTIONS">
<ROW>
<FIELD VALUE="09_NYC-3A_2" NAME="PageType"/>
<FIELD VALUE="" NAME="KeyWord"/>
<FIELD VALUE="020852009111001.002" NAME="ImageName"/>
<FIELD VALUE="2" NAME="PageNo"/>
<FIELD VALUE="" NAME="Qualifier"/>
</ROW>
</REPEAT>
</REPEATS>
</DOCUMENT>
<DOCUMENT ID="140208512T200911102">
<REPEATS>
<REPEAT NAME="EXCEPTIONS">
<ROW>
<FIELD VALUE="09_NYC-3A_2" NAME="PageType"/>
<FIELD VALUE="" NAME="KeyWord"/>
<FIELD VALUE="020852009111001.002" NAME="ImageName"/>
<FIELD VALUE="2" NAME="PageNo"/>
<FIELD VALUE="" NAME="Qualifier"/>
</ROW>
</REPEAT>
</REPEATS>
</DOCUMENT>
</DOCUMENTS>
and i need to retrieve the XML node for ID - 140208512T200911101 alone. i cant able to get the information using various methods, still didnt get the correct one.
my desired result should be like this :
<DOCUMENT ID="140208512T200911101">
<REPEATS>
<REPEAT NAME="EXCEPTIONS">
<ROW>
<FIELD VALUE="09_NYC-3A_2" NAME="PageType"/>
<FIELD VALUE="" NAME="KeyWord"/>
<FIELD VALUE="020852009111001.002" NAME="ImageName"/>
<FIELD VALUE="2" NAME="PageNo"/>
<FIELD VALUE="" NAME="Qualifier"/>
</ROW>
</REPEAT>
</REPEATS>
</DOCUMENT>
Please help on this...
Thanks for your support and it is working fine, for getting the #ID value dynamically from a variable we need to user like this :
DECLARE #DCN Varchar(50)
SET #DCN = '140208512T200911101'
select #XMLValues.query('/DOCUMENTS/DOCUMENT[#ID = sql:variable("#DCN")]')
select #XMLValues.query('/DOCUMENTS/DOCUMENT[#ID = "140208512T200911101"]')
Result
<DOCUMENT ID="140208512T200911101">
<REPEATS>
<REPEAT NAME="EXCEPTIONS">
<ROW>
<FIELD VALUE="09_NYC-3A_2" NAME="PageType" />
<FIELD VALUE="" NAME="KeyWord" />
<FIELD VALUE="020852009111001.002" NAME="ImageName" />
<FIELD VALUE="2" NAME="PageNo" />
<FIELD VALUE="" NAME="Qualifier" />
</ROW>
</REPEAT>
</REPEATS>
</DOCUMENT>
Getting started with NHibernate
How can I generate identity fields in nHibernate using Hilo algorithm?
use class="hilo":
<generator class="hilo">
example:
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" namespace="NHibernate__MyClass" assembly="NHibernate__MyClass">
<class name="MyClass" table="MyClass">
<id name="Id" type="int" column="ID">
<generator class="hilo">
</id>
<property name="Name">
<column name="Name" not-null="true" />
</property>
<property name="Value">
<column name="Value" not-null="true" />
</property>
</class>
</hibernate-mapping>
I simplified:
<id name="Id">
<column name="ID" sql-type="int" not-null="true"/>
<generator class="hilo" />
</id>
to:
<id name="Id" type="int" column="ID">
<generator class="hilo">
</id>
You could have a syntax error of some sort that is confusing NHibernate.
If you could provide more detail about the code that is executing before the failure or anything else you might think is important, that could speed the rate at which your problem is resolved.
I haven't watched the screencasts yet. But Summer of nHibernate should help you.
I am sorry - I am not answering your original question.
I've been racking my head trying to get Nhibernate to work with a byte
array as version mapping to an sql timestamp. I'd implemented an
IUserVersionType but Nhibernate was creating varbinary in the database
rather than timestamp. Inspired by a blog post by Ayende recently on
concurrency, I changed my mapping to specify the sql-type to timestamp
which worked perfectly. However I now face a rather curious problem
wherein Nhibernate does an insert, gets the new version and then
immediately tries to do an update and attempts to set the version
column, which being an sql timestamp fails.
This is my mapping:
<?xml version="1.0" encoding="utf-8"?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2"
assembly="Core.Domain, Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" namespace="Core.Domain.Entities"
default-lazy="false">
<class name="Contact" table="Contacts" xmlns="urn:nhibernate-
mapping-2.2" optimistic-lock="version" dynamic-insert="true" dynamic-
update="true">
<id name="Id" type="Int32" column="Id">
<generator class="identity" />
</id>
<version name="Version" type="BinaryBlob" generated="always"
unsaved-value="null">
<column name="Version" sql-type="timestamp" not-null="false" />
</version>
<property name="Title" type="String">
<column name="Title" length="5" />
</property>
<property name="FirstName" type="String">
<column name="FirstName" not-null="true" length="50" />
</property>
<property name="MiddleName" type="String">
<column name="MiddleName" length="50" />
</property>
<property name="LastName" type="String">
<column name="LastName" not-null="true" length="50" />
</property>
<property name="Suffix" type="String">
<column name="Suffix" length="5" />
</property>
<property name="Email" type="String">
<column name="Email" length="50" />
</property>
<bag name="PhoneNumbers" inverse="true" cascade="all-delete-
orphan">
<key foreign-key="FK_Contacts_PhoneNumbers_ContactId" on-
delete="cascade" column="ContactId" />
<one-to-many class="Core.Domain.Entities.PhoneNumber,
Core.Domain, Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" />
</bag>
<property name="DateCreated" type="DateTime">
<column name="DateCreated" />
</property>
<property name="DateModified" type="DateTime">
<column name="DateModified" />
</property>
<property name="LastModifiedBy" type="String">
<column name="LastModifiedBy" />
</property>
</class>
</hibernate-mapping>
<?xml version="1.0" encoding="utf-8"?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2"
assembly="Core.Domain, Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" namespace="Core.Domain.Entities"
default-lazy="false">
<class name="Customer" table="Customers" xmlns="urn:nhibernate-
mapping-2.2" optimistic-lock="version" dynamic-insert="true" dynamic-
update="true">
<id name="Id" type="Int32" column="Id">
<generator class="identity" />
</id>
<version name="Version" type="BinaryBlob" generated="always"
unsaved-value="null">
<column name="Version" sql-type="timestamp" not-null="false" />
</version>
<property name="AccountNumber" access="nosetter.pascalcase-
underscore" type="String">
<column name="AccountNumber" unique="true" length="25" />
</property>
<!-- other mappings... -->
<property name="DateCreated" type="DateTime">
<column name="DateCreated" />
</property>
<property name="DateModified" type="DateTime">
<column name="DateModified" />
</property>
<property name="LastModifiedBy" type="String">
<column name="LastModifiedBy" />
</property>
<joined-subclass name="Core.Domain.Entities.Individual,
Core.Domain, Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" table="Individuals">
<key column="CustomerId" />
<many-to-one fetch="join" lazy="false" not-null="true"
cascade="all" unique="true" not-found="exception" name="Contact"
column="ContactID" />
<bag name="Addresses" table="Addresses_Individuals">
<key column="AddressId" foreign-
key="FK_Addresses_Individuals_Addresses_AddressId" />
<many-to-many column="IndividualId"
class="Core.Domain.Entities.Address, Core.Domain,
Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" foreign-
key="FK_Addresses_Individuals_Individuals_IndividualId" />
</bag>
</joined-subclass>
<joined-subclass name="Core.Domain.Entities.Store, Core.Domain,
Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" table="Stores">
<key column="CustomerId" />
<many-to-one unique="true" cascade="save-update" fetch="join"
not-null="true" not-found="exception" name="Address"
column="AddressId" />
<many-to-one lazy="proxy" not-null="true" cascade="all" not-
found="exception" name="Client" column="ClientId" />
<property name="StoreName" type="String">
<column name="StoreName" not-null="true" length="50" />
</property>
<bag name="Contacts" table="Contacts_Stores">
<key column="ContactId" foreign-
key="FK_Contacts_Stores_Contacts_ContactId" />
<many-to-many column="StoreId"
class="Core.Domain.Entities.Contact, Core.Domain,
Version=0.1.3397.31993, Culture=neutral,
PublicKeyToken=94dc7dc697cfcfc0" foreign-
key="FK_Contacts_Stores_Stores_StoreId" />
</bag>
</joined-subclass>
</class>
</hibernate-mapping>
Calling Session.Save on an Individual with associated Contact results
in the following error:
NHibernate: INSERT INTO Addresses (Line1, PostalCode, Country,
DateCreated, DateModified, LastModifiedBy) VALUES (#p0, #p1, #p2, #p3,
#p4, #p5); select SCOPE_IDENTITY(); #p0 = 'Order Address Line 1', #p1
= 'CV31 6BW', #p2 = 'United Kingdom', #p3 = '20/04/2009 19:45:32', #p4
= '20/04/2009 19:45:32', #p5 = ''
NHibernate: SELECT address_.Version as Version22_ FROM Addresses
address_ WHERE address_.Id=#p0; #p0 = '1'
NHibernate: INSERT INTO Contacts (FirstName, LastName, DateCreated,
DateModified, LastModifiedBy) VALUES (#p0, #p1, #p2, #p3, #p4); select
SCOPE_IDENTITY(); #p0 = 'Joe', #p1 = 'Bloggs', #p2 = '20/04/2009
19:45:34', #p3 = '20/04/2009 19:45:34', #p4 = ''
NHibernate: SELECT contact_.Version as Version33_ FROM Contacts
contact_ WHERE contact_.Id=#p0; #p0 = '1'
NHibernate: INSERT INTO Customers (AccountNumber, DateCreated,
DateModified, LastModifiedBy) VALUES (#p0, #p1, #p2, #p3); select
SCOPE_IDENTITY(); #p0 = '', #p1 = '20/04/2009 19:45:34', #p2 =
'20/04/2009 19:45:34', #p3 = ''
NHibernate: INSERT INTO Individuals (ContactID, CustomerId) VALUES
(#p0, #p1); #p0 = '1', #p1 = '1'
NHibernate: SELECT individual_1_.Version as Version2_ FROM Individuals
individual_ inner join Customers individual_1_ on
individual_.CustomerId=individual_1_.Id WHERE
individual_.CustomerId=#p0; #p0 = '1'
NHibernate: UPDATE Contacts SET Version = #p0 WHERE Id = #p1 AND
Version = #p2; #p0 = 'System.Byte[]', #p1 = '1', #p2 = 'System.Byte[]'
System.Data.SqlClient.SqlException: Cannot update a timestamp column.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception,
Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException
exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning
(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior,
SqlCommand cmdHandler, SqlDataReader dataStream,
BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject
stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader
ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds
(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean
returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior
cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String
method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery
(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at NHibernate.AdoNet.AbstractBatcher.ExecuteNonQuery(IDbCommand cmd)
in c:\CSharp\NH\nhibernate\src\NHibernate\AdoNet\AbstractBatcher.cs:
line 203
at NHibernate.Persister.Entity.AbstractEntityPersister.Update(Object
id, Object[] fields, Object[] oldFields, Object rowId, Boolean[]
includeProperty, Int32 j, Object oldVersion, Object obj,
SqlCommandInfo sql, ISessionImplementor session) in c:\CSharp\NH
\nhibernate\src\NHibernate\Persister\Entity
\AbstractEntityPersister.cs: line 2713
NHibernate.Exceptions.GenericADOException: could not update:
[Core.Domain.Entities.Contact#1][SQL: UPDATE Contacts SET Version =
#p0 WHERE Id = #p1 AND Version = #p2]
Any ideas why NHibernate is attempting to update the version column
for Contact, even though it didn't for the Address?
I have found that using dynamic-insert="true" on the class along with causes this issue. I use the following mapping successfully:
...
<class name="Contact" table="Contact">
<id name="ID" column="ID" type="int">
<generator class="identity" />
</id>
<version name="Version" generated="always" unsaved-value="null" type="BinaryBlob"/>
...
The Address doesn't have a version column I assume.
I wonder where you have the sql-type from. Why not this way?
<version name="Version" type="Timestamp" generated="always" unsaved-value="null">
<column name="Version" not-null="false" />
</version>
You need of course a DateTime in the entity.
http://ayende.com/Blog/archive/2009/04/15/nhibernate-mapping-concurrency.aspx