I have installed the snowflake plugin in my Pentaho version 9.2. I have specific flows with me that I need to update. Those flows are working fine then suddenly they start showing errors. enter. find this Error is in the table input step.
Table input.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from
2021-06-02 06.36.08 by buildguy) : Unexpected error
2022/06/23 20:35:38 - Table input.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2021-06-02 06.36.08 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:
2022/06/23 20:35:38 - Table input.0 - Couldn't get row from result set
2022/06/23 20:35:38 - Table input.0 - at java.lang.Thread.run (null:-1)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.trans.step.RunThread.run (RunThread.java:62)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.trans.steps.tableinput.TableInput.processRow (TableInput.java:143)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.trans.steps.tableinput.TableInput.doQuery (TableInput.java:265)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow (Database.java:2712)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow (Database.java:2734)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow (Database.java:2756)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.DatabaseMeta.getValueFromResultSet (DatabaseMeta.java:3030)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.BaseDatabaseMeta.getValueFromResultSet (BaseDatabaseMeta.java:2138)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.row.value.ValueMetaBase.getValueFromResultSet (ValueMetaBase.java:5309)
2022/06/23 20:35:38 - Table input.0 - at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getTimestamp (SnowflakeBaseResultSet.java:302)
2022/06/23 20:35:38 - Table input.0 - at net.snowflake.client.jdbc.SnowflakeResultSetV1.getTimestamp (SnowflakeResultSetV1.java:236)
2022/06/23 20:35:38 - Table input.0 - at net.snowflake.client.core.SFBaseResultSet.getTimestamp (SFBaseResultSet.java:522)
2022/06/23 20:35:38 - Table input.0 -
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow(Database.java:2764)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow(Database.java:2734)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow(Database.java:2712)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.trans.steps.tableinput.TableInput.doQuery(TableInput.java:265)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.trans.steps.tableinput.TableInput.processRow(TableInput.java:143)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2022/06/23 20:35:38 - Table input.0 - at java.lang.Thread.run(Unknown Source)
2022/06/23 20:35:38 - Table input.0 - Caused by: java.lang.NullPointerException
2022/06/23 20:35:38 - Table input.0 - at net.snowflake.client.core.SFBaseResultSet.getTimestamp(SFBaseResultSet.java:522)
2022/06/23 20:35:38 - Table input.0 - at net.snowflake.client.jdbc.SnowflakeResultSetV1.getTimestamp(SnowflakeResultSetV1.java:236)
2022/06/23 20:35:38 - Table input.0 - at net.snowflake.client.jdbc.SnowflakeBaseResultSet.getTimestamp(SnowflakeBaseResultSet.java:302)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.row.value.ValueMetaBase.getValueFromResultSet(ValueMetaBase.java:5309)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.BaseDatabaseMeta.getValueFromResultSet(BaseDatabaseMeta.java:2138)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.DatabaseMeta.getValueFromResultSet(DatabaseMeta.java:3030)
2022/06/23 20:35:38 - Table input.0 - at org.pentaho.di.core.database.Database.getRow(Database.java:2756)
2022/06/23 20:35:38 - Table input.0 - ... 6 more
also in database connection, the connection type is automatically selected to snowflake database from snowflake. I thought that it can be the cause. Now my all-flows table input step is showing an error.
kindly guide me.
thanks
shivam
Related
I need to use a simple Pentaho Spoon transformation to copy a table from MySQL database to OpenLink Virtuoso RDB (Virtuoso Open Source).
The table on MySQL database contains a column typed LONG VARCHAR.
This data type exists in OpenLinkVirtuoso.
The problem is that Pentaho Spoon seems not understand this data type.
I tried this SQL script directly on Openlink Virtuoso RDB:
CREATE TABLE DBTEST.GSDRB_SUB_CSN
(
SUBCOUNTER INTEGER
, REF_CATALOGO INTEGER
, CSNC VARCHAR(5)
, CSNL VARCHAR(10)
, CSNS VARCHAR(255)
, CSNO VARCHAR(10)
, CSNN LONG VARCHAR
)
And it works fine. So the LONG VARCHAR data type actually exists and is understood.
The same is run on Pentaho Kettle (inside Create Table action), but raises this error:
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - ERROR (version 9.3.0.0-428, build 9.3.0.0-428 from 2022-04-12 04.56.25 by buildguy) : An error occurred executing this job entry :
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - Couldn't execute SQL: CREATE TABLE GSDRB_SUB_CSN
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - (
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - SUBCOUNTER INTEGER
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - , REF_CATALOGO INTEGER
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - , CSNC VARCHAR(5)
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - , CSNL VARCHAR(10)
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - , CSNS VARCHAR(255)
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - , CSNO VARCHAR(10)
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - , CSNN LONG VARCHAR
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - )
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] -
2023/01/18 11:33:35 - Create table [GSDRB_SUB_CSN] - SQ074: Line 9: syntax error
I tried also with BLOB but i have the same issue.
Do you have any suggestions?
I'm trying to drop my foreign key but I got the error:
The ALTER TABLE statement conflicted with the FOREIGN KEY constraint "FK_TABLE_ITEM_LIST_DETAILS_TABLE_ITEM1_ITEM_ID". The conflict occurred in database "TABLE", table "dbo.TABLE_ITEM", column 'ID'
I tried to run add-migration and update-database.
How to fix this error?
I have a very large table with ~500 mil records. This table already has index on 3 fields
CREATE TABLE temp (id bigserial primary key, min_timestamp timestamp with time zone, max_timestamp timestamp with time zone, orgname varchar);
min_timestamp index_1
max_timestamp index_2
orgname index_3
I am trying to create concurrent multi column index_4 on (orgname, min_timestamp, max_timestamp).
When the CREATE CIC index_4 executes it also executes automatically create index of index_2 which already exists.
question:
Why is index_2 getting executed which is already existing?
index_2 is blocked by index_4 and results in deadlock after several hours. How to over come this?
Environment: AWS RDS Postgres ver: 9.6
Any pointers would be of great help.
I am moving from Teradata to Oracle and I have a question regarding Primary Keys and Indexes in the DDLs.
Here are several scenarios and what I am doing:
1) In Teradata there is:
Primary Key("X","Y")
then in Exadata I use:
constraint "PK" PRIMARY KEY ("X","Y")
USING INDEX TABLESPACE DB_NAME" ENABLE;
2) In Teradata there is no primary key but there is:
PRIMARY INDEX ("X")
then in Exadata I use:
CREATE INDEX "DB_NAME"."INDEX_NAME" ON "DB_NAME"."TABLE_NAME" ("X")
3) In Teradata there is no primary key but there is:
UNIQUE PRIMARY INDEX ("X")
then in Exadata I use
constraint "PK" PRIMARY KEY ("X","Y")
USING INDEX TABLESPACE DB_NAME" ENABLE;
4) In Teradata there is no primary key but there is:
INDEX ("X")
then in Exadata I use Create Index
As you can see, It's not entirely clear when I need to use just the Constraint Primary Key and when I need to use create index. Any help would be much appreciated.
Is it better to have a single primary key, or use composite primary keys (usually, they are combination of one primary key and foriegn keys). I have examples below:
Composite Primary Key example:
AMeta
--- AMetaId - Primary Key
--- AMetaText
BMeta
--- BMetaId - Primary Key
--- AMetaID - Foreign Key to Table AMeta
--- BMetaText
A
--- AId - Primary Key
--- AMetaId - Primary Key and Foreign Key to Table AMeta
--- AText
B
--- BId - Primary Key
--- BMetaId - Primary Key Foreign Key to Table BMeta
--- AId - Primary Key and Foreign Key to Table A
--- BText
Single Primary Key example:
AMeta
--- AMetaId - Primary Key
--- AMetaText
BMeta
--- BMetaId - Primary Key
--- AMetaId - Foreign Key to Table AMeta
--- BMetaText
A
--- AId - Primary Key
--- AMetaId - Foreign Key to Table AMeta
--- AText
B
--- BId - Primary Key
--- BMetaId - Foreign Key to Table BMeta
--- AId - Foreign Key to Table A
--- BText
Which is the better Database Design?
I genereally tend to use single-column primary keys almost exclusively - either there is a good natural key available (pretty rarely), or then I add a surrogate INT IDENTITY key to the table.
My main reasons are:
the primary key needs to be unique, non-null at all times
the primary key is the clustering key in SQL Server by default, which adds stable (non-changing), narrow (4 bytes max.) and preferably ever-increasing to the list of criteria
with a single column primary key, all my foreign keys are also easy to use single columns - I don't like having to join two tables on 3, 5 or even more columns just to get the join to work, which is exactly what happens if you have a compound primary key
The first scheme makes no sense, because it implies (and allows) that there can be multiple rows in Table B with the same BId value but with different values of AId, and there is no meaning associated with column Bid. Is it a FK to somewhere else? If so, to what? If not, what is generating it ? What does it mean?
The second scheme, on the other hand, is logically consistent, but implies that rows in Table B can be associated with two different rows in Table AMeta,
through Table B using the FK Column
BMetaId and from there to Table
AMeta using TableB.AMetaId, and
Through table BMeta using column
BMetaId to Table BMeta and from
there to AMeta using BMEta.AMetaId
Is this really an accurate representation of your business domain model?