Need some help with pglogical replication. Following is my replication setup:
On the Publisher side, node created successfully.
I applied row filtering on a column for some tables. Something like:
SELECT pglogical.replication_set_add_table(set_name:= 'default', relation := 'table_1', row_filter:= ' country=''USA'' ');
SELECT pglogical.replication_set_add_table(set_name:= 'default', relation := 'table_2', row_filter:= ' country=''USA'' ');
SELECT pglogical.replication_set_add_table(set_name:= 'default', relation := 'table_3', row_filter:= ' country=''USA'' ');
On the Subscriber side, node created successfully. When I create the subscription to the publisher, the pglogical.show_subscription_status() stays in initializing status for sometime and then changes to down
I checked the subscriber DB logs and I see the following error:
LOG: statement: COPY "public"."table_1"("id","column_1","column_2",...........,"some_column") FROM stdin
ERROR: null value in column "some_column" violates not-null constraint
I check the Publisher and there are no NULL values in some_column in table_1. I also checked the specific row in table_1 which error'ed out and the some_column's value is 'f'. The some_column is a Boolean NOT NULL column.
Any suggestions on how to resolve this issue?
Thanks.
Related
In our tool we use triggers on materialized views in order to create log-entries (and do some other things) when a transaction is commited.
The code works good in Oracle 12. In Oracle 19 the old values in that trigger (":old") seems to be lost.
Investigations:
This seems to be the case in the combination of materialized views/triggers. If we set the same trigger on a table the logs are generated correctly (but we do not get the transaction-awareness which is required).
I have created a MWE and added comments to the DBMS_OUTPUT-Lines which describe what we see in oracle 12 and Oracle 18/19:
/*Create Test-Table*/
CREATE TABLE MAT_VIEW_TEST (
PK number(10,0) PRIMARY KEY ,
NAME NVARCHAR2(50)
);
/*insert some values*/
insert into MAT_VIEW_TEST values (1, 'Herbert');
insert into MAT_VIEW_TEST values (2, 'Hubert');
commit;
/*Create mateterialized view (log) in order to set trigger on it*/
CREATE MATERIALIZED VIEW LOG ON MAT_VIEW_TEST WITH PRIMARY KEY, ROWID including new values;
CREATE MATERIALIZED VIEW MV_MAT_VIEW_TEST
refresh fast on commit
AS select * from MAT_VIEW_TEST;
/*Create trigger to log old and new value*/
CREATE OR REPLACE TRIGGER MAT_VIEW_TRIGGER
BEFORE INSERT OR UPDATE
ON MV_MAT_VIEW_TEST
FOR EACH ROW
DECLARE
old_pk number(10,0);
new_pk number(10,0);
old_name NVARCHAR2(50);
new_name NVARCHAR2(50);
BEGIN
old_pk := :old.pk;
old_name := :old.name;
new_pk := :new.pk;
new_name := :new.name;
DBMS_OUTPUT.PUT_LINE('TEST BEGIN');
DBMS_OUTPUT.PUT_LINE('old p ' || old_pk); /*old is set in oracle 12, but not in oracle18/19*/
DBMS_OUTPUT.PUT_LINE('old n ' || old_name); /*old is set in oracle 12, but not in oracle18/19*/
DBMS_OUTPUT.PUT_LINE('new p ' || new_pk); /*new is set correctly*/
DBMS_OUTPUT.PUT_LINE('new n ' || new_name); /*new is set correctly*/
DBMS_OUTPUT.PUT_LINE('TEST END');
END;
/
/*test the log*/
update MAT_VIEW_TEST set name = 'Test' where pk = 1;
commit;
Any ideas what was changed in Oracle or what we could do to get the old values in our trigger?
I don't have a 12c to rerun your tests, but I did on a 21c, and with the trigger you show, the old values are never shown, neither on insert (normal) nor on update( which is what you're complaining about). When I changed the trigger to be 'on insert or update or delete', and reran an update, I can see the old values. So, the refresh process is converting your UPDATE to DELETE/INSERT, hence the old values when it is deleting the old row.
I'm trying to creating a table in a SQL Server database using FireDAC. However, instead of using the index name I provide, FireDAC uses a bad index name, raising an exception and the table does not get created. Am I doing something wrong? If not, is there a work-around?
Note that I'm using the valid database schema name cnf for TableName. I specifically need to create the table in a schema.
Simplest test case:
var
Connection: TFDConnection;
Table: TFDTable;
begin
Connection := TFDConnection.Create(nil);
Table := TFDTable.Create(nil);
try
Connection.Params.Add ('DriverID=MSSQL');
Connection.Params.Add ('OSAuthent=No');
Connection.Params.Add ('User_Name=sa');
Connection.Params.Add ('Password=XXXXXX');
Connection.Params.Add ('Server=DAVE-DELL\MSSQLSERVER2016');
Connection.Params.Add ('Database=PROJECT_DB');
Connection.Params.Add ('MARS=No');
Connection.Open;
Table.Connection := Connection;
Table.TableName := 'cnf.TestTable';
Table.FieldDefs.Add ('TableID', ftAutoInc, 0, true);
Table.FieldDefs.Add ('Field1', ftInteger, 0, true);
Table.FieldDefs.Add ('Field2', ftstring, 100, true);
Table.IndexDefs.Add ('PK_XYZ', 'TableID', [ixPrimary]); // should use this index name!
Table.CreateTable (true);
finally
Table.Free;
Connection.Free;
end;
end;
An exception is raised:
[FireDAC][Phys][ODBC][Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near '.'.
Running SQL Server Profiler shows me that FireDAC is trying to create the index using the following SQL code:
ALTER TABLE temp.TestTable ADD CONSTRAINT [cnf].[PK_TestTable] PRIMARY KEY (TableID)
And, of course, [cnf].[PK_TestTable] is not a valid index name in T-SQL, which is the crux of the problem.
If I remove the line Table.IndexDefs.Add, the table is created properly, but without the index.
If I replace that line with the following, it gives the same problem:
with Table.IndexDefs.AddIndexDef do begin
Name := 'PK_XYZ';
Options := [ixPrimary];
Fields := 'TableID';
end;
If I replace setting the table name with the following, it gives the same problem:
Table.TableName := 'TestTable';
Table.SchemaName := 'cnf';
Why is it using it's own (wrong) index name, instead of the name I gave it? (i.e. PK_XYZ)
Embarcadero® Delphi 10.1 Berlin Version 24.0.25048.9432
SQL Server 2016 (SP2-CU4) - 13.0.5233.0 (X64)
Am I doing something wrong?
Why is it using it's own (wrong) index name, instead of the name I gave it?
You seem to be doing everything just right. The issue is with the generated SQL command as you have tracked that down. SQL Server doesn't allow schema name in constraint name when adding a constraint using ALTER TABLE. Constraints created this way automatically become part of schema of the related table, however you should later use schema name when referring to the constraint:
SELECT OBJECT_ID('cnf.PK_XYZ')
Now where do the things go wrong? FireDAC uses TFDPhysCommandGenerator and its ancestors to generate SQL commands for specific DBMS. Your call to CreateTable method results in call to TFDPhysCommandGenerator.GetCreatePrimaryKey, which is responsible for generating SQL for primary key. It also contains this code:
sTab := GetFrom;
FConnMeta.DecodeObjName(sTab, rName, nil, [doUnquote]);
rName.FObject := 'PK_' + rName.FObject;
Result := 'ALTER TABLE ' + sTab + ' ADD CONSTRAINT ' +
FConnMeta.EncodeObjName(rName, nil, [eoQuote, eoNormalize]) + ' PRIMARY KEY (';
What this code does is that it takes your fully qualified table name (sTab) splits it (DecodeObjName) into parts (rName) prepends 'PK_' to table name and joins the parts (EncodeObjName) back to fully qualified name, which is then used as the constraint name for your primary key. Now we can clearly see that command generator ignores your index name and generates erroneous T-SQL. This can either be a bug or just a not supported feature. EMBT has to make decision on that. I'd recommend reporting it as a bug.
Is there a work-around?
Yes, you can either hook problematic method or you can override it in your own derived class. Implementation none of these is trivial and due to legal issues I'm not going to extend it here, because I would have to duplicate the original FireDAC code.
As for the syntax error adding these lines to 'TFDPhysCommandGenerator.GetCreatePrimaryKey' implementation after DecodeObjName would fix the issue:
rName.FCatalog := '';
rName.FSchema := '';
rName.FBaseObject := '';
rName.FLink := '';
Fixing constraint name is going to be more cumbersome than that, because the method only receives index column names as argument and has no obvious access to original IndexDefs where you could just use index name as primary key constraint name. Gaining access to index name from there would also allow you to get rid of decoding/encoding table name into index name. This process, however, could be essential for other DMBS's than SQL Server.
PS: If only half of all the questions were written in this manner ... Thank you for this wonderful question.
The scenario is the following -
OrderTable with Columns "OrderId" and "OrderType"
OrderRelationTable with Columns "OrderId" and "CustId"
OrderProcessTable with Columns "OrderId", "OrderType", "CustId", and "ProcessFlag"
The flow goes like this-
Application1 creates the record in OrderTable -> Then pass the record to Application2 by using MQ protocol, Application 2 in this case insert/create the record passed in the OrderRelationTable -> Then a trigger is called in Oracle DB to create the record in OrderProcessTable
Problem
Sometimes the record is not inserted into table 3 OrderProcessTable. Not sure if it is cause by timing or there is something that is not correct with the trigger?
Application1 Code
boolean updated = false;
/** JDBC prepare statement execution insert into OrderTable in Java**/
int rowCount = ps.executeUpdate();
if(rowCount>0){
updated=true;
}
log.log("updated flag:"+updated);
/** I am able to see the log shows the flag is true, and recored inserted into OrderTable **/
Application2 Code
This doesn't really matter much assuming that it is some Java JDBC code that does the insert into OrderRelationTable and it is successful.
The Trigger
Assuming the syntax is correct.
CREATE OR REPLACE TRIGGER INSERTINTOOrderProcessTable
AFTER INSERT ON OrderRelationTable
FOR EACH ROW DECLEAR
v_order_type := null;
BEGIN
SELECT OrderType INTO v_order_type FROM OrderTable
WHERE OrderId = :new.OrderId
AND OrderType IS NOT NULL
AND rownum=1;
IF v_order_type IS NOT NULL THEN
INSERT INTO OrderProcessTable VALUES (:new.OrderId, v_order_type, :new.CustId, 'N');
END IF;
END;
Questions -
After the Application 1 Code is executed is guaranteed DB will have the OrderTable record avaliable for SELECT statement? (Assume that updated flag is true)
Is there a timing issue with the app code and trigger? for example when trigger calls the SELECT statement from OrderTable? (of course the order id is matching in the OrderRelationTable and OrderTable)
Basically right now my problem is that sometimes (rarely) the record is not inserted into OrderProcessTable via the trigger even though it should (Order Type is not null)? Any idea?
There's no timing issue, as far as I can tell.
As of trigger code: what is the purpose of and rownum = 1 condition? I'm not saying that it is wrong, I'm just asking. Do you expect several rows to be returned by that query? If so, is that a legal situation? Wouldn't you rather handle it with the WHEN TOO_MANY_ROWS exception handler (i.e. instead of using the ROWNUM condition)?
What happens if SELECT returns nothing? It raises then NO_DATA_FOUND exception and trigger fails and certainly doesn't insert anything. Is it propagated so that someone (human being) or something (error logging procedure) sees / catches it so that you'd know that something went wrong?
And, of course, the fact that V_ORDER_TYPE remains NULL which causes INSERT to fail (as P. Salmon already suggested).
I have a problem to resolve cache updates when delta includes fields that have UNIQUE constraint on the database. I have a database with the following DDL schema (SQLite in memory can be used to reproduce):
create table FOO
(
ID integer primary key,
DESC char(2) UNIQUE
);
The initial database table contains one record with ID = 1 and DESC = R1
Acessing this table with a TFDQuery (select * from FOO), if the following steps are performed, the generated delta will be correctly applied with ApplyUpdates:
Update record ID = 1 to DESC = R2
Append a new record ID = 2 with DESC = R1
Delta includes the following:
R2
R1
No error will be generated on ApplyUpdates, because the first operation on delta will be an update. The second will be an insert. As record 1 now is R2, the insertion can be done because there are no violation of the unique contraint on this transaction.
Now, performing the following steps, will generate the exactly same delta (look at the FDQuery.Delta property), but a UNIQUE constraint violation will be generated.
Append a new temporary record ID = 2 with DESC = TT
Update the first record ID = 1 to DESC = R2
Update the temporary record 2 - TT to DESC = R1
Delta includes the following:
R2
R1
Note that FireDAC generates the same delta on both scenarios, this can be viewed through the FDquery's Delta property.
This steps cand be used to reproduce the error:
File > New VCL Forms Application; Drop a FDConnection and FDQuery on form; Set FDConnection to use SQLite driver (using in memory database); Drop two buttons on form, one to reproduce the correctly behavior, and another to reproduce the error, as follows:
Button OK:
procedure TFrmMain.btnOkClick(Sender: TObject);
begin
// create the default database with a FOO table
con.Open();
con.ExecSQL('create table FOO' + '(ID integer primary key, DESC char(2) UNIQUE)');
// insert a default record
con.ExecSQL('insert into FOO values (1,''R1'')');
qry.CachedUpdates := true;
qry.Open('select * from FOO');
// update the first record to T2
qry.First();
qry.Edit();
qry.Fields[1].AsString := 'R2';
qry.Post();
// append the second record to T1
qry.Append();
qry.Fields[0].AsInteger := 2;
qry.Fields[1].AsString := 'R1';
qry.Post();
// apply will not generate a unique constraint violation
qry.ApplyUpdates();
end;
Button Error:
// create the default database with a FOO table
con.Open();
con.ExecSQL('create table FOO' + '(ID integer primary key, DESC char(2) UNIQUE)');
// insert a default record
con.ExecSQL('insert into FOO values (1,''R1'')');
qry.CachedUpdates := true;
qry.Open('select * from FOO');
// append a temporary record (TT)
qry.Append();
qry.Fields[0].AsInteger := 2;
qry.Fields[1].AsString := 'TT';
qry.Post();
// update R1 to R2
qry.First();
qry.Edit();
qry.Fields[1].AsString := 'R2';
qry.Post();
qry.Next();
// update TT to R1
qry.Edit();
qry.Fields[1].AsString := 'R1';
qry.Post();
// apply will generate a unique contraint violation
qry.ApplyUpdates();
Update Since writing the original version of this answer, I've done some more investigation and am beginning to think that either there is a problem with ApplyUpdates, etc, in FireDAC's support for Sqlite (in Seattle, at least), or we are not using the FD components correctly. It would need FireDAC's author (who is a contributor here) to say which it is.
Leaving aside the ApplyUpdates business for a moment, there are a number of other problems with your code, namely your dataset navigation makes assumptions about the ordering on the rows in qry and the numbering of its Fields.
The test case I have used is to start (before execution of the application) with the Foo table containing the single row
(1, 'R1')
Then, I execute the following Delphi code, at the same time as monitoring the contents of Foo using an external application (the Sqlite Manager plug-in for FireFox). The code executes without an error being reported in the application, but notice that it does not call ApplyUpdates.
Con.Open();
Con.StartTransaction;
qry.Open('select * from FOO');
qry.InsertRecord([2, 'TT']);
assert(qry.Locate('ID', 1, []));
qry.Edit;
qry.FieldByName('DESC').AsString := 'R2';
qry.Post;
assert(qry.Locate('ID', 2, []));
qry.Edit;
qry.FieldByName('DESC').AsString := 'R1';
qry.Post;
Con.Commit;
qry.Close;
Con.Close;
The added row (ID = 2) is not visible to the external application until after Con.Close has executed, which I find puzzling. Once Con.Close has been called, the external application shows Foo as containing
(1, 'R2')
(2, 'R1')
However, I have been unable to avoid the constraint violation error if I call ApplyUpdates, regardless of any other changes I make to the code, including adding a call to ApplyUpdates after the first Post.
So, it seems to me that either the operation of ApplyUpdates is flawed or it is not being used correctly.
I mentioned FireDAC's author. His name is Dmitry Arefiev and he has answered a lot of FD qs on SO, though I haven't noticed him here in the past couple of months or so. You might try catching his attention by posting in EMBA's FireDAC NG forum, https://forums.embarcadero.com/forum.jspa?forumID=502.
I have to sync changes done in MSSQL with a remote MySQL database. The changes to be synced are adding invoices and users to the system. The remote server is not expected to be always reachable so I'm trying to set up a kind of log table for storing changes done in MSSQL.
Here is a fully working trigger for that:
CREATE TRIGGER [dbo].[dokument_insert]
ON [dbo].[dokument]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [bcg_ekodu].[dbo].[sync_stack] (event,sql, table_name, import_priority)
SELECT
'INSERT',
'INSERT INTO bills SET
date = "'+CONVERT(VARCHAR(19),dok_kuup,120)+'",
total = "'+CAST(kokkusum AS nvarchar)+'",
number = "'+RTRIM(dok_nr)+'",
created = "'+CONVERT(VARCHAR(19),savetime,120)+'",
rounded = "'+CAST(ymardus AS nvarchar)+'",
currency = "'+CAST(valuuta AS nvarchar)+'",
due_date = "'+CONVERT(VARCHAR(19),tasupaev,120)+'",
pk_joosep = "'+CAST(dok_kood AS nvarchar)+'",
joosep_hankija = "'+CAST(hankija AS nvarchar)+'";
UPDATE
bills, users, companies
SET
bills.user_id = users.id,
bills.imported = NOW()
WHERE
bills.imported IS NULL
AND companies.id = users.company_id
AND companies.pk_joosep = 10
AND bills.user_id = users.pk_joosep',
'bills',
'200'
FROM inserted
END
It inserts a row into 'sync_stack' table every time a row is inserted to 'dokument' table. The 'sql' column will contain an SQL to create the same kind of row in another (MySQL) database.
But this trigger is not working:
CREATE TRIGGER [dbo].[klient_insert]
ON [dbo].[klient]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [bcg_ekodu].[dbo].[sync_stack] (event,sql, table_name, import_priority)
SELECT
'INSERT',
'INSERT INTO users SET
username =10'+CAST(kl_kood as nvarchar)+',
password = NULL,
name ="'+LTRIM(RTRIM(kl_nimi))+'",
email ="'+CAST(LTRIM(RTRIM(kl_email)) as nvarchar)+'",
reference_no ="'+CAST(LTRIM(RTRIM(kl_viide)) as nvarchar)+'",
phone ="'+CAST(LTRIM(RTRIM(kl_tel1)) as nvarchar)+'",
logins ="'+CAST(0 as nvarchar)+'",
last_login = NULL,
created ="'+CONVERT(VARCHAR(19),savetime,120)+'",
updated = NULL,
deleted ="0",
address ="'+CAST(LTRIM(RTRIM(kl_aadr1)) as nvarchar)+'",
pk_joosep ="'+CAST(kl_kood as nvarchar)+'"',
'users',
'210'
FROM inserted
END
While the execution of the above SQL to create that trigger completes just fine, when I try to insert some rows to the 'triggered' table, I get the following error:
No row was updated.
The data in row 175 was not committed.
Error Source: .Net SqlClient Data Provider.
Error Message: Cannot insert the value NULL into column 'sql', table 'mydb.dbo.sync_stack'; column does not allow nulls. INSERT fails.
The statement has been terminated.
Correct the errors and retry or press ESC to cancel the change(s).
If I delete this trigger, this error does not occur.
If I insert just plain text for 'sql' column, it works as expected.
If I use any field from the inserted row, even just a text field, it fails again.
If I allow NULL values in 'sql' column, inserting rows succeeds but I get a NULL value in 'sql' column.
How to make the second trigger work as expected, too?
I suspect that at least one of the values from inserted that you are concatenating into your SQL statement is NULL. You can circumvent this by using COALESCE, e.g.
username =10'+COALESCE(CAST(kl_kood as nvarchar), '')+',
Of course you shouldn't be declaring nvarchar without specifying a length, right?
Bad habits to kick : declaring VARCHAR without (length)
Concatenating any value to NULL is NULL:
select 'test' + NULL
Results in null, you should use something like that for your columns:
select isnull(column, '')
This would result in an empty string.