I have a table in which the id column gets its value from a sequence.
I am interacting with database using OCI.
After inserting a record to the table, I need to get the value of the sequence which was used when inserting the record (into to my process memory), which effectively is the most recent sequence value generated for the current session. According to this answer, it should be possible.
I found USER_SEQUENCES table which may be suitable for my purpose, but I am not sure whether I can use it because it has sequence data about current user. What I need is sequence data of current session. I have multiple concurrent database sessions which use same user id and password.
you can make like this :
insert into testtable (ID, ...)
values (schema.seq_testtable.nextval, ...)
returning ID into currID;
you can read here about "returning" clause :
http://psoug.org/snippet/UPDATE-with-RETURNING-clause_604.htm
You can use sequence_name.currval . Oracle documentation says:
Using Sequence Numbers with CURRVAL
To use or refer to the current sequence value of your session,
reference seq_name.CURRVAL. CURRVAL can only be used if
seq_name.NEXTVAL has been referenced in the current user session (in
the current or a previous transaction). CURRVAL can be referenced as
many times as necessary, including multiple times within the same
statement. The next sequence number is not generated until NEXTVAL is
referenced.
You should specify which development environment are you using. If you use JAVA, when you have problem, because JDBC assumes what DML statements have ONLY input bind parameters. The common trick is to wrap an INSERT statement into anonymous block and make it callable statement.
begin
insert into t(id, x) valuse(t_seq.next_val, :x) returning in into :id;
end;
Such a callable statement will have one IN and one OUT parameter. Also JDBC 3.0 introduced it's own way ( getGeneratedKeys() method), but I'm not sure how it works with Oracle.
In case of OCI it works out-of-the-box. If I recall correctly in OCI you simply call OCIBindByName and you specify, that variable from returning clause is of type (INOUT). Then after execution you will find ID returned from a statement in that bind variable. It even works with bulk inserts, i.e you pass vector(s) of values to be inserted into the DB, and you get a vector of IDs generated.
I can not quickly find any OCI example. In case of OTL (OCI wrapper lib) it's here here. I think any OCI wrapper lib supports it. Also look at OCILIB, it's source code is easier to read.
SEQ_NAME.CURRVAL as per #Stefan said, but in that case you have 1st time you must have NEXTVAL for the same sequence in current session, in case of having CACHE in sequence you can use #EntGriff option of return inserted ID to variable.
Related
I need to generate the progressive number of a invoices, avoiding gaps in the sequence:
At beginning I thought it was quite easy as
SELECT MAX(Docnumber)+1 as NewDocNumber
from InvoicesHeader
but since it takes some time to build the "insert into InvoiceHeader" query and another request could arrive, assigning to both Invoices the same NewDocNumber
I'm now thinking to avoid to generate the DocNumber in advanced and changed query to:
INSERT INTO InvoicesHeader (InvoiceID,..., DocNumber,...)
SELECT #InvoiceID,..., MAX(Docnumber)+1,... FROM InvoicesHeader
but although (it should) solve some problems, it is still thread unsafe and not suitable for race conditions:
adding TABLOCK or UPDLOCK, in this way:
BEGIN TRANSACTION TR1
INSERT INTO InvoicesHeader WITH (TABLOCK)
(InvoiceID,..., DocNumber,...)
SELECT #InvoiceID,..., MAX(Docnumber)+1,... FROM InvoicesHeader
COMMIT TRANSACTION TR1
Will solve the issue?
Or better to use ISOLATION LEVEL, NEXT VALUE FOR or other solution?
You already having thread safe generation of sequence generation in SQL Server. Read about Create Sequence. It is available starting from SQL Server 2012. It is better to use as sequence is generated outside the transaction scope.
Sequence numbers are generated outside the scope of the current
transaction. They are consumed whether the transaction using the
sequence number is committed or rolled back.
You can get next value from the sequence. We have been using sequences for generating order numbers and we have not found issues, when multiple order nubmers are generated in parallel.
SELECT NEXT VALUE FOR DocumentSequenceNumber;
Updated, based on comments, if you have four different documenttypes, I would suggest you to first generate sequence and then concatenate with a specific document type. It will be easier for you to understand. At the end of the year, you can restart the sequence using ALTER SEQUENCE
RESTART [ WITH ] The next value that will be returned by
the sequence object. If provided, the RESTART WITH value must be an
integer that is less than or equal to the maximum and greater than or
equal to the minimum value of the sequence object. If the WITH value
is omitted, the sequence numbering restarts based on the original
CREATE SEQUENCE options.
The question in short:
Can I call a stored procedure that has an output parameter in a data flow?
In long:
I have many tables to extract, transform, and load from one db to another one.
Almost all of the tables require one transformation which is fixing the country codes (from 3 letters to two). So my idea is as follows:
for each row: call the stored procedure, pass the wrong country code, replace the wrong code with the correct one (the output of the stored procedure)
There are at least two solutions for this:
Look-Up component: configuring it in advance mode and make sure the last sentence of the SProc is the Select statement that returns the good country code (e.g. SELECT #good_country_code)
Using an OLEDB Command
The latter (OLEDB Command) is actually quite simple, you need to configure it with:
EXEC ? = dbo.StoredProc #param1 = ?, #param2 = ?
As a consequence a #RETURN_VALUE will appear on the Available Destination Columns which you can then map to an existing column in the pipeline. Remember to create a new pipeline field/column (e.g. Good_Country_Code) using a Derived Column component before the OLEDB component and you'll have the chance to have both values, or replace the wrong one using another Derived Column component after OLEDB Command.
No, natively there isn't a component that is going to handle that. You can accomplish it with a Script Component but you don't want to.
What you're describing is a Lookup. The Data Flow Task has a Lookup Component but you'll be better served, especially for a finite list of values like Countries to push your query into the component.
SELECT T.Country3, T.Country2 FROM dbo.Table T;
Then you drag your SourceCountry column and match to Country3. Check Country2 and for all the rows that match, you'll get the 2 letter abbreviation.
A big disadvantage of trying to use your stored procedure is efficiency. The default Lookup is going to cache all those values. With the Script Version, say you have 10k rows come through, all with CAN. That's 10k invocations of your stored procedure where the results never change.
You do pay a startup cost as the default Lookup mode is Full Cache which means it's going to run your query and keep all those values local. This is great with your data set: 1000 countries max, 5 or 10 byte per row. That's nothing.
Yes, you can. You'll want to use a couple Execute SQL Tasks to do this.
Use an Execute SQL Task to gather a Result Set of Wrong_Country_Codes.
Add a ForEach Container as a successor to the previous Execute SQL Task. Pass the Result Set to this Container.
Inside that ForEach container, you will have another Execute SQL Task that will call your sproc, using each row (e.g. Wrong_Country_Code) as a variable parameter.
That should work. Only select the columns necessary to pass to your stored procedure.
Edit
In acknowledgement to the other answer, performance is going to be an issue. Perhaps rather than have the stored procedure produce an output, alter the sproc to do the updates for you.
Requirement:
To count the number of times a procedure has executed
From what I understand so far, sys.dm_exec_procedure_stats can be used for approximate count but that's only since the last service restart. I found this link on this website relevant but I need count to be precise and that should not flush off after the service restart.
Can I have some pointers on this, please?
Hack: The procedure I need to keep track of has a select statement so returns some rows that are stored in a permanent table called Results. The simplest solution I can think of is to create a column in Results table to keep track of the procedure execution, select the maximum value from this column before the insert and add one to it to increment the count. This solution seems quite stupid to me as well but the best I could think of.
What I thought is you could create a sequence object, assuming you're on SQL Server 2012 or newer.
CREATE SEQUENCE ProcXXXCounter
AS int
START WITH 1
INCREMENT BY 1 ;
And then in the procedure fetch a value from it:
declare #CallCount int
select #CallCount = NEXT VALUE FOR ProcXXXCounter
There is of course a small overhead with this, but doesn't cause similar blocking issue that could happen with using a table because sequences are handled outside transaction.
Sequence parameters: https://msdn.microsoft.com/en-us/library/ff878091.aspx
The only way I can think of keeping track of number of executions even when the service has restarted , is to have a table in your database and insert a row to that table inside your procedure everytime it is executed.
Maybe add a datetime column as well to collect more info about the execution. And a column for user who executed etc..
This can be done, easily and without Enterprise Edition, by using extended events. The sqlserver.module_end event will fire, set your predicates correctly and use a histogram target.
http://sqlperformance.com/2014/06/extended-events/predicate-order-matters
https://technet.microsoft.com/en-us/library/ff878023(v=sql.110).aspx
To consume the value, query the histogram target (under the reviewing target output examples).
A few months back, I started using a CRUD script generator for SQL Server. The default insert statement that this generator produces, SELECTs the inserted row at the end of the stored procedure. It does the same for the UPDATE too.
The previous way (and the only other way I have seen online) is to just return the newly inserted Id back to the business object, and then have the business object update the Id of the record.
Having an extra SELECT is obviously an additional database call, and more data is being returned to the application. However, it allows additional flexibility within the stored procedure, and allows the application to reflect the actual data in the table.
The additional SELECT also increases the complexity when wanting to wrap the insert/update statements in a transaction.
I am wondering what people think is better way to do it, and I don't mean the implementation of either method. Just which is better, return just the Id, or return the whole row?
We always return the whole row on both an Insert and Update. We always want to make sure our client apps have a fresh copy of the row that was just inserted or updated. Since triggers and other processes might modify values in columns outside of the actual insert/update statement, and since the client usually needs the new primary key value (assuming it was auto generated), we've found it's best to return the whole row.
The select statement will have some sort of an advantage only if the data is generated in the procedure. Otherwise the data that you have inserted is generally available to you already so no point in selecting and returning again, IMHO. if its for the id then you can have it with SCOPE_IDENTITY(), that will return the last identity value created in the current session for the insert.
Based on my prior experience, my knee-jerk reaction is to just return the freshly generated identity value. Everything else the application is inserting, it already knows--names, dollars, whatever. But a few minutes reflection and reading the prior 6 (hmm, make that 5) replies, leads to a number of “it depends” situations:
At the most basic level, what you inserted is what you’d get – you pass in values, they get written to a row in the table, and you’re done.
Slightly more complex that that is when there are simple default values assigned during an insert statement. “DateCreated” columns that default to the current datetime, or “CreatedBy” that default to the current SQL login, are a prime example. I’d include identity columns here, since not every table will (or should) contain them. These values are generated by the database upon table insertion, so the calling application cannot know what they are. (It is not unknown for web server clocks to not be synchronized with database server clocks. Fun times…) If the application needs to know the values just generated, then yes, you’d need to pass those back.
And then there are are situations where additional processing is done within the database before data is inserted into the table. Such work might be done within stored procedures or triggers. Once again, if the application needs to know the results of such calculations, then the data would need to be returned.
With that said, it seems to me the main issue underlying your decision is: how much control/understanding do you have over the database? You say you are using a tool to automatically generate your CRUD procedures. Ok, that means that you do not have any elaborate processing going on within them, you’re just taking data and loading it on in. Next question: are there triggers (of any kind) present that might modify the data as it is being written to the tables? Extend that to: do you know whether or not such triggers exists? If they’re there and they matter, plan accordingly; if you do not or cannot know, then you might need to “follow up” on the insert to see if changes occurred. Lastly: does the application care? Does it need to be informed of the results of the insert action it just requested, and if so, how much does it need to know? (New identity value, date time it was added, whether or not something changed the Name from “Widget” to “Widget_201001270901”.)
If you have complete understanding and control over the system you are building, I would only put in as much as you need, as extra code that performs no useful function impacts performance and maintainability. On the flip side, if I were writing a tool to be used by others, I’d try to build something that did everything (so as to increase my market share). And if you are building code where you don't really know how and why it will be used (application purpose), or what it will in turn be working with (database design), then I guess you'd have to be paranoid and try to program for everything. (I strongly recommend not doing that. Pare down to do only what needs to be done.)
Quite often the database will have a property that gives you the ID of the last inserted item without having to do an additional select. For example, MS SQL Server has the ##Identity property (see here). You can pass this back to your application as an output parameter of your stored procedure and use it to update your data with the new ID. MySQL has something similar.
INSERT
INTO mytable (col1, col2)
OUTPUT INSERTED.*
VALUES ('value1', 'value2')
With this clause, returning the whole row does not require an extra SELECT and performance-wise is the same as returning only the id.
"Which is better" totally depends on your application needs. If you need the whole row, return the whole row, if you need only the id, return only the id.
You may add an extra setting to your business object which can trigger this option and return the whole row only if the object needs it:
IF #return_whole_row = 1
INSERT
INTO mytable (col1, col2)
OUTPUT INSERTED.*
VALUES ('value1', 'value2')
ELSE
INSERT
INTO mytable (col1, col2)
OUTPUT INSERTED.id
VALUES ('value1', 'value2')
FI
I don't think I would in general return an entire row, but it could be a useful technique.
If you are code-generating, you could generate two procs (one which calls the other, perhaps) or parametrize a single proc to determine whther to return it over the wire or not. I doubt the DB overhead is significant (single-row, got to have a PK lookup), but the data on the wire from DB to client could be significant when all added up and if it's just discarded in 99% of the cases, I see little value. Having an SP which returns different things with different parameters is a potential problem for clients, of course.
I can see where it would be useful if you have logic in triggers or calculated columns which are managed by the database, in which case, a SELECT is really the only way to get that data back without duplicating the logic in your client or the SP itself. Of course, the place to put any logic should be well thought out.
Putting ANY logic in the database is usually a carefully-thought-out tradeoff which starts with the minimally invasive and maximally useful things like constraints, unique constraints, referential integrity, etc and growing to the more invasive and marginally useful tools like triggers.
Typically, I like logic in the database when you have multi-modal access to the database itself, and you can't force people through your client assemblies, say. In this case, I would still try to force people through views or SPs which minimize the chance of errors, duplication, logic sync issues or misinterpretation of data, thereby providing as clean, consistent and coherent a perimeter as possible.
In Oracle there is a mechanism to generate sequence numbers e.g.;
CREATE SEQUENCE supplier_seq
MINVALUE 1
MAXVALUE 999999999999999999999999999
START WITH 1
INCREMENT BY 1
CACHE 20;
And then execute the statement
supplier_seq.nextval
to retrieve the next sequence number.
How would you create the same functionality in MS SQL Server ?
Edit: I'm not looking for ways to automaticly generate keys for table records. I need to generate a unique value that I can use as an (logical) ID for a process. So I need the exact functionality that Oracle provides.
There is no exact match.
The equivalent is IDENTITY that you can set as a datatype while creating a table. SQLSERVER will automatically create a running sequence number during insert.
The last inserted value can be obtained by calling SCOPE_IDENTITY() or by consulting the system variable ##IDENTITY (as pointed out by Frans)
If you need the exact equivalent, you would need to create a table and then write a procedure to retun the next value and other operations. See Marks response on pitfalls on this.
Edit:
SQL Server has implemented the Sequence similar to the Oracle. Please refer to this question for more details.
How would you implement sequences in Microsoft SQL Server?
Identity is the best and most scalable solution, BUT, if you need a sequence that is not an incrementing int, like 00A, 00B, 00C, or some special sequence, there is a second-best method. If implemented correctly, it scales OK, but if implemented badly, it scales badly. I hesitate to recommend it, but what you do is:
You have to store the "next value" in a table. The table can be a simple, one row, one column table with just that value. If you have several sequences, they can share the table, but you might get less contention by having separate tables for each.
You need to write a single update statement that will increment that value by 1 interval. You can put the update in a stored proc to make it simple to use and prevent repeating it in code in different places.
Using the sequence correctly, so that it will scale reasonably (no, not as well as Identitiy :-) requires two things: a. the update statement has a special syntax made for this exact problem that will both increment and return the value in a single statement; b. you have to fetch the value from the custom sequence BEFORE the start of a transaction and outside the transaction scope. That is one reason Identity scales -- it returns a new value irrespective of transaction scope, for any attempted insert, but does not roll back on failure. That means that it won't block, and also means you'll have gaps for failed transactions.
The special update syntax varies a little by version, but the gist is that you do an assignment to a variable and the update in the same statement. For 2008, Itzik Ben-Gan has this neat solution: http://www.sqlmag.com/Articles/ArticleID/101339/101339.html?Ad=1
The old-school 2000 and later method looks like this:
UPDATE SequenceTable SET #localVar = value = value + 5
-- change the tail end to your increment logic
This will both increment and return you the next value.
If you absolutely cannot have gaps (resist that requirement :-) then it is technically possible to put that update or proc in side the rest of your trnsaction, but you take a BIG concurrency hit as every insert waits for the prior one to commit.
I can't take credit on this; I learned it all from Itzik.
make the field an Identity field. The field will get its value automatically. You can obtain the last inserted value by calling SCOPE_IDENTITY() or by consulting the system variable ##IDENTITY
The SCOPE_IDENTITY() function is preferred.
As DHeer said there is absolutely no exact match. If you try to build your own procedure to do this you will invariably stop your application from scaling.
Oracle's sequences are highly scalable.
OK, I take it back slightly. If you're really willing to focus on concurrency and you're willing to take numbers out of order as is possible with a sequence, you have a chance. But since you seem rather unfamiliar with t-sql to begin with, I would start to look for some other options when (porting an Oracle app to MSSS - is that what you're doing)
For instance, just generate a GUID in the "nextval" function. That would scale.
Oh and DO NOT use a table for all the values, just to persist your max value in the cache. You'd have to lock it to ensure you give unique values and this is where you'll stop scaling. You'll have to figure out if there's a way to cache values in memory and programmatic access to some sort of lightweight locks- memory locks, not table locks.
I wish that SQL Server had this feature. It would make so many things easier.
Here is how I have gotten around this.
Create a table called tblIdentities. In this table put a row with your min and max values and how often the Sequence number should be reset. Also put the name of a new table (call it tblMySeqNum). Doing this makes adding more Sequence Number generators later fairly easy.
tblMySeqNum has two columns. ID (which is an int identity) and InsertDate (which is a date time column with a default value of GetDate()).
When you need a new seq num, call a sproc that inserts into this table and use SCOPE_IDENTITY() to get the identity created. Make sure you have not exceeded the max in tblIdentities. If you have then return an error. If not return your Sequence Number.
Now, to reset and clean up. Have a job that runs as regularly as needed that checks all the tables listed in tblIdentites (just one for now) to see if they need to be reset. If they have hit the reset value or time, then call DBCC IDENT RESEED on the name of the table listed in the row (tblMySeqNum in this example). This is also a good time to clear our the extra rows that you don't really need in that table.
DON'T do the cleanup or reseeding in your sproc that gets the identity. If you do then your sequence number generator will not scale well at all.
As I said, it would make so many things easier of this feature was in SQL Server, but I have found that this work around functions fairly well.
Vaccano
If you are able to update to SQL Server 2012 you can use SEQUENCE objects. Even SQL Server 2012 Express has support for sequences.
CREATE SEQUENCE supplier_seq
AS DECIMAL(38)
MINVALUE 1
MAXVALUE 999999999999999999999999999
START WITH 1
INCREMENT BY 1
CACHE 20;
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
Results in:
---------------------------------------
1
(1 row(s) affected)
---------------------------------------
2
(1 row(s) affected)
---------------------------------------
3
(1 row(s) affected)
---------------------------------------
4
(1 row(s) affected)
---------------------------------------
5
(1 row(s) affected)
Just take care to specify the right data type. If I hadn't specified it the MAXVALUE you've provided wouldn't be accepted, that's why I've used DECIMAL with the highest precision possible.
More on SEQUENCES here: http://msdn.microsoft.com/en-us/library/ff878091.aspx
This might have already been answered a long time ago... but from SQL 2005 onwards you can use the ROW_NUMBER function... an example would be:
select ROW_NUMBER() OVER (ORDER BY productID) as DynamicRowNumber, xxxxxx,xxxxx
The OVER statement uses the ORDER BY for the unique primary key in my case...
Hope this helps... no more temp tables, or strange joins!!
Not really an answer, but it looks like sequences are coming to SQLServer in 2012.
http://www.sql-server-performance.com/2011/sequence-sql-server-2011/
Not an exact answer but addition to some existing answers
SCOPE_IDENTITY (Transact-SQL)
SCOPE_IDENTITY, IDENT_CURRENT, and ##IDENTITY are similar functions
because they return values that are inserted into identity columns.
IDENT_CURRENT is not limited by scope and session; it is limited to a
specified table. IDENT_CURRENT returns the value generated for a
specific table in any session and any scope. For more information, see
IDENT_CURRENT (Transact-SQL).
It means two different sessions can have a same identity value or sequence number so to avoid this and get unique number for all sessions use IDENT_CURRENT
Exactly because of this
IDENT_CURRENT is not limited by scope and session; it is limited to a specified table.
we need to use SCOPE_IDENTITY() because scope identity will give us unique number generated in our session, and uniqueness is provided by identity itself.