I have a big table that includes a column of floats (1.5million rows) that I join to a slightly smaller table (15k rows) which also has floats, I then multiply various floats.
I have discovered I get significant performance gains (over 10 times faster) using numerics rather than floats in the big table.
Trouble is I don't know the size of the floats in advance so I was hoping to calculate the length of the biggest float and then use that information to cast the float to a numeric using a variable in the declaration i.e. cast(MyFloatColumn as numeric(#varInt,2))
It seems I'm not allowed to do this (Incorrect syntax error) so is there an alternative?
Below is some code that shows what I am trying to do - final statement is where the error is.
Many thanks for your help,
Simon
CREATE TABLE dbo.MyTable
(
MyFloatColumn float
);
GO
INSERT INTO dbo.MyTable VALUES (12345.12041);
INSERT INTO dbo.MyTable VALUES (123.1);
GO
declare #precisionofbiggest int
SELECT #precisionofbiggest = sizeofintpart + 2
FROM (SELECT TOP (1) Len(Cast(Cast(myfloatcolumn AS BIGINT) AS VARCHAR)) AS
sizeofintpart
FROM dbo.mytable
ORDER BY myfloatcolumn DESC) AS atable
SELECT cast(myfloatcolumn AS numeric(#precisionofbiggest,2)) AS anewnumericcolumn
FROM dbo.mytable
(#precisionofbiggest will be 7 in this example so if it worked I would get
aNewNumericColumn
12345.12
123.10
)
the last statement should be dynamic to get the variable value
declare #sql nvarchar(max)
SET #sql = 'SELECT cast(myfloatcolumn AS numeric('
+ CONVERT(VARCHAR(20), #precisionofbiggest)
+ ',2)) AS anewnumericcolumn FROM dbo.mytable'
exec sp_executesql #sql
Related
Hello looking solution to sort sizes inside my column.
Exmaple :
-- CREATE TEMP TABLE
Create Table #MyTempTable (
size varchar(20)
);
-- insert sample data to TEMP TABLE
insert into #MyTempTable
values
('10.5W'),
('10W'),
('11.5W'),
('11W'),
('12W'),
('5.5W'),
('5W'),
('6.5W'),
('6W'),
('7.5W'),
('7W'),
('8.5W'),
('8W'),
('9.5W'),
('9W'),
('4')
select 'BEFORE',* from #MyTempTable
SELECT 'AFTER',size
FROM #MyTempTable
ORDER BY LEN(size)
When i order by LEN there is no good sorting like this :
AFTER 5W
AFTER 6W
AFTER 7W
AFTER 8W
AFTER 9W
AFTER 10W
AFTER 11W
AFTER 12W
AFTER 5.5W
AFTER 7.5W
AFTER 6.5W
AFTER 9.5W
AFTER 8.5W
AFTER 10.5W
AFTER 11.5W
All im' looking for is to sort in proper order. like this :
5W
5.5W
6W
6.5W
7W
7.5W
8W
8.5W
9W
9.5W
10W
10.5W
11W
11.5W
12W
I seearched a lot of stackoverflow and can't find solution for that because there is not only int and also decimal numbers. So don't know how to get it
Assuming each value would always end in just one unit, you may sort on the numeric portion cast to a decimal:
SELECT size
FROM #MyTempTable
ORDER BY CAST(
CASE WHEN size LIKE '%[A-Z]'
THEN LEFT(size, LEN(size) - 1)
ELSE size END AS DECIMAL(10, 2)
);
A couple of other options:
-- if you don't know all of the potential non-numeric characters:
SELECT size FROM #MyTempTable
ORDER BY TRY_CONVERT(decimal(5,2),
SUBSTRING(size,1,COALESCE(NULLIF
(PATINDEX('%[^0-9.]%', size),0),255)-1));
-- if there is a finite set (say, W and D):
DECLARE #KnownChars varchar(32) = 'WD';
SELECT size FROM #MyTempTable
ORDER BY TRY_CONVERT(decimal(5,2),
TRANSLATE(size, #KnownChars, REPLICATE(space(1), LEN(#KnownChars))));
In order by clause first remove W from then cast as NUMERIC data type. Now you can expect sorting like a number.
SELECT *
FROM #MyTempTable AS mtt
ORDER BY CAST(LEFT(mtt.size, LEN(mtt.size) - 1) AS DECIMAL) ASC;
I'm curently storing a list of ids in a column as a CSV string value ('1;2;3').
I'd like to optimize with a better approach (I believe) which would use varbinary(max).
I'm looking for tsql functions
1 . That would merge side by side a set of integer rows into a varbinary(max)
2 . That would split the varbinary field into a set of int rows
Any tips appreciated, thanks
Solution is very questionable. I'd also suggest to normalize the data.
However, if you still want to store your data as VARBINARY, here is the solution:
CREATE FUNCTION dbo.fn_String_to_Varbinary(#Input VARCHAR(MAX))
RETURNS VARBINARY(MAX) AS
BEGIN
DECLARE #VB VARBINARY(MAX);
WITH CTE as (
SELECT CAST(CAST(LEFT(IT,CHARINDEX(';',IT)-1) AS INT) as VARBINARY(MAX)) as VB, RIGHT(IT,LEN(IT) - CHARINDEX(';',IT)) AS IT
FROM (VALUES (#Input)) as X(IT) union all
SELECT VB + CAST(CAST(LEFT(IT,CHARINDEX(';',IT)-1) AS INT) as VARBINARY(MAX)) as VB, RIGHT(IT,LEN(IT) - CHARINDEX(';',IT)) AS IT FROM CTE WHERE LEN(IT) > 1
)
SELECT TOP 1 #VB = VB FROM CTE
ORDER BY LEN(VB) DESC
RETURN #VB
END
GO
DECLARE #Input VARCHAR(MAX) = '421;1;2;3;5000;576;842;375;34654322;18;67;000001;1232142334;'
DECLARE #Position INT = 9
DECLARE #VB VARBINARY(MAX)
SELECT #VB = dbo.fn_String_to_Varbinary(#Input)
SELECT #VB, CAST(SUBSTRING(#VB,4*(#Position-1)+1,4) AS INT)
GO
The function converts string into VARBINARY and then script extracts 9th number from that VARBINARY value.
Do not run this function against a data set with million records and million numbers in each line.
I have a SQL Server table with numbers in column no:
12345670000115
14245670000116
58492010000118
I need a function that will remove one number 1 from right side of number, so result must be like:
1234567000015
1424567000016
5849201000018
I find some solutions to use charindex() with substring(), but my SQL skills are poor so I really need help.
Thanks
Assuming this is varchar data here is an easy way to accomplish this. BTW, I would suggest you not use column names like 'no'. It is a reserved word and it is horribly ambiguous. Does that mean number or the opposite of yes? If it is number as I assume it would be better to name the column with an indication of what the number is. PartNumber, ItemNumber, CatalogNumber whatever...
LEFT(no, len(no) - 2) + RIGHT(no, 1)
Try to use this query:
declare #charToReplace char = '1'
select REVERSE(stuff(REVERSE(no), charindex(#charToReplace, REVERSE(no)), 1, ''))
from table
or
declare #charToReplace char = '1'
declare #tmp_table TABLE (NO varchar(16))
insert into #tmp_table
select REVERSE(NO)
from yourtable
select REVERSE(stuff(NO, charindex(#charToReplace, NO), 1, ''))
For your particular data, if the numbers fit a BIGINT, one easy way is to treat them like numbers:
Setup
create table #tmp (
number VARCHAR(16)
)
insert into #tmp values ('12345670000115'), ('14245670000116'), ('58492010000118')
GO
Script:
select number, cast( (cast(number AS bigint) - 100) / 100 * 10 + cast(number AS bigint) % 100 as VARCHAR(16))
from #tmp
GO
I resolve problem. There is answer in which I remove one character 1 and update whole table. Thanks all for help!
Update myTableName
set barcode=substring(barcode,1,11)+substring(barcode,13,1)
where len(barcode)>= 14
I have variable called #prmclientcode which is nvarchar. The input to this variable can be a single client code or multiple client codes separated by comma. For e.g.
#prmclientcode='1'
or
#prmclientcode='1,2,3'.
I am comparing this variable to a client code column in of the tables. The data type of this column is numeric(6,0). I tried converting the variable data type like below
SNCA_CLIENT_CODE IN ('''+convert(numeric(6,0),#prmclientcode+''')) (The query is inside a dynamic sql).
But when I try executing this I get the error
Arithmetic overflow error converting nvarchar to data type numeric.
Can anyone please help me here!
Thanks!
You need to convert the numeric(6,0) column to nvarchar data type. You can use below scrip to convert it to nvarchar, before processing:
SNCA_CLIENT_CODE IN ('''+convert(cast( numeric(6,0) as nvarchar(max) ),#prmclientcode+'''))
Please try with the below code snippet.
DECLARE #ProductTotals TABLE
(
ProductID int
)
INSERT INTO #ProductTotals VALUES(1)
INSERT INTO #ProductTotals VALUES(11)
INSERT INTO #ProductTotals VALUES(3)
DECLARE #prmclientcode VARCHAR(MAX)='1'
SELECT * FROM #ProductTotals
SELECT * FROM #ProductTotals WHERE CHARINDEX(',' + CAST(ProductID AS VARCHAR(MAX)) + ',' , ',' + ISNULL(#prmclientcode,ProductID) + ',') > 0
Let me know if any concern.
use following code in order to separate your variable:
DECLARE
#T VARCHAR(100) = '1,2,3,23,342',
#I int = 1
;WITH x(I, num) AS (
SELECT 1, CHARINDEX(',',#T,#I)
UNION ALL
SELECT num+1,CHARINDEX(',',#T,num+1)
FROM x
WHERE num+1<LEN(#T)
AND num<>0
)
SELECT SUBSTRING(#T,I,CASE WHEN num=0 THEN LEN(#T)+1 ELSE num END -I)
FROM x
Use can use either table function or dynamic sql query, both options will work.
Let me know if you need more help
I am stuck on converting a varchar column UserID to INT. I know, please don't ask why this UserID column was not created as INT initially, long story.
So I tried this, but it doesn't work. and give me an error:
select CAST(userID AS int) from audit
Error:
Conversion failed when converting the varchar value
'1581............................................................................................................................' to data type int.
I did select len(userID) from audit and it returns 128 characters, which are not spaces.
I tried to detect ASCII characters for those trailing after the ID number and ASCII value = 0.
I have also tried LTRIM, RTRIM, and replace char(0) with '', but does not work.
The only way it works when I tell the fixed number of character like this below, but UserID is not always 4 characters.
select CAST(LEFT(userID, 4) AS int) from audit
You could try updating the table to get rid of these characters:
UPDATE dbo.[audit]
SET UserID = REPLACE(UserID, CHAR(0), '')
WHERE CHARINDEX(CHAR(0), UserID) > 0;
But then you'll also need to fix whatever is putting this bad data into the table in the first place. In the meantime perhaps try:
SELECT CONVERT(INT, REPLACE(UserID, CHAR(0), ''))
FROM dbo.[audit];
But that is not a long term solution. Fix the data (and the data type while you're at it). If you can't fix the data type immediately, then you can quickly find the culprit by adding a check constraint:
ALTER TABLE dbo.[audit]
ADD CONSTRAINT do_not_allow_stupid_data
CHECK (CHARINDEX(CHAR(0), UserID) = 0);
EDIT
Ok, so that is definitely a 4-digit integer followed by six instances of CHAR(0). And the workaround I posted definitely works for me:
DECLARE #foo TABLE(UserID VARCHAR(32));
INSERT #foo SELECT 0x31353831000000000000;
-- this succeeds:
SELECT CONVERT(INT, REPLACE(UserID, CHAR(0), '')) FROM #foo;
-- this fails:
SELECT CONVERT(INT, UserID) FROM #foo;
Please confirm that this code on its own (well, the first SELECT, anyway) works for you. If it does then the error you are getting is from a different non-numeric character in a different row (and if it doesn't then perhaps you have a build where a particular bug hasn't been fixed). To try and narrow it down you can take random values from the following query and then loop through the characters:
SELECT UserID, CONVERT(VARBINARY(32), UserID)
FROM dbo.[audit]
WHERE UserID LIKE '%[^0-9]%';
So take a random row, and then paste the output into a query like this:
DECLARE #x VARCHAR(32), #i INT;
SET #x = CONVERT(VARCHAR(32), 0x...); -- paste the value here
SET #i = 1;
WHILE #i <= LEN(#x)
BEGIN
PRINT RTRIM(#i) + ' = ' + RTRIM(ASCII(SUBSTRING(#x, #i, 1)))
SET #i = #i + 1;
END
This may take some trial and error before you encounter a row that fails for some other reason than CHAR(0) - since you can't really filter out the rows that contain CHAR(0) because they could contain CHAR(0) and CHAR(something else). For all we know you have values in the table like:
SELECT '15' + CHAR(9) + '23' + CHAR(0);
...which also can't be converted to an integer, whether you've replaced CHAR(0) or not.
I know you don't want to hear it, but I am really glad this is painful for people, because now they have more war stories to push back when people make very poor decisions about data types.
This question has got 91,000 views so perhaps many people are looking for a more generic solution to the issue in the title "error converting varchar to INT"
If you are on SQL Server 2012+ one way of handling this invalid data is to use TRY_CAST
SELECT TRY_CAST (userID AS INT)
FROM audit
On previous versions you could use
SELECT CASE
WHEN ISNUMERIC(RTRIM(userID) + '.0e0') = 1
AND LEN(userID) <= 11
THEN CAST(userID AS INT)
END
FROM audit
Both return NULL if the value cannot be cast.
In the specific case that you have in your question with known bad values I would use the following however.
CAST(REPLACE(userID COLLATE Latin1_General_Bin, CHAR(0),'') AS INT)
Trying to replace the null character is often problematic except if using a binary collation.
This is more for someone Searching for a result, than the original post-er. This worked for me...
declare #value varchar(max) = 'sad';
select sum(cast(iif(isnumeric(#value) = 1, #value, 0) as bigint));
returns 0
declare #value varchar(max) = '3';
select sum(cast(iif(isnumeric(#value) = 1, #value, 0) as bigint));
returns 3
I would try triming the number to see what you get:
select len(rtrim(ltrim(userid))) from audit
if that return the correct value then just do:
select convert(int, rtrim(ltrim(userid))) from audit
if that doesn't return the correct value then I would do a replace to remove the empty space:
select convert(int, replace(userid, char(0), '')) from audit
This is how I solved the problem in my case:
First of all I made sure the column I need to convert to integer doesn't contain any spaces:
update data set col1 = TRIM(col1)
I also checked whether the column only contains numeric digits.
You can check it by:
select * from data where col1 like '%[^0-9]%' order by col1
If any nonnumeric values are present, you can save them to another table and remove them from the table you are working on.
select * into nonnumeric_data from data where col1 like '%[^0-9]%'
delete from data where col1 like '%[^0-9]%'
Problems with my data were the cases above. So after fixing them, I created a bigint variable and set the values of the varchar column to the integer column I created.
alter table data add int_col1 bigint
update data set int_col1 = CAST(col1 AS VARCHAR)
This worked for me, hope you find it useful as well.