This is in SQL Server 2005. I have a varchar column and some rows contain trailing space, e.g. abc, def.
I tried removing the trailing space with this command:
update thetable
set thecolumn = rtrim(thecolumn)
But the trailing space remains. I tried to find them using:
select *
from thetable
where thecolumn <> rtrim(thecolumn)
But it returned nothing.
Are there some settings that I am not aware that influences trailing space check?
EDIT:
I know that there is trailing space from SSMS, when I copy paste the value from the grid to the editor, it has trailing space.
Check if the spaces that are not removed have the ASCII code 32.
Try this to replace "hard space" with "soft space":
update thetable set thecolumn = rtrim(replace(thecolumn, char(160), char(32)))
the query was missing equal sign
Are you certain that it is a space (ascii 32) character? You can get odd behavior with other "non-visible" characters. Try running
select ascII(right(theColumn, 1))
from theTable
and see what you get.
Use this Function:
Create Function [dbo].[FullTrim] (#strText varchar(max))
Returns varchar(max) as
Begin
Declare #Ch1 char,#ch2 char
Declare #i int,#LenStr int
Declare #Result varchar(max)
Set #i=1
Set #LenStr=len(#StrText)
Set #Result=''
While #i<=#LenStr
Begin
Set #ch1=SUBSTRING(#StrText,#i,1)
Set #ch2=SUBSTRING(#StrText,#i+1,1)
if ((#ch1=' ' and #ch2=' ') or (len(#Result)=0 and #ch1=' '))
Set #i+=1
Else
Begin
Set #Result+=#Ch1
Set #i+=1
End
End
Return #Result
End
In SQL, CHAR(n) columns are right-padded with spaces to their length.
Also string comparison operators (and most functions too) do not take the trailing spaces into account.
DECLARE #t TABLE (c CHAR(10), vc VARCHAR(10))
INSERT
INTO #t
VALUES ('a ', 'a ')
SELECT LEN(c), LEN(vc), с + vc
FROM #t
--
1 1 "a a"
Please run this query:
SELECT *
FROM thetable
WHERE thecolumn + '|' <> RTRIM(thecolumn) + '|'
and see if it finds something.
It sounds like either:
1) Whatever you are using to view the values is inserting the trailing space (or the appearance thereof- try a fixed-width font like Consolas).
2) The column is CHAR, not VARCHAR. In that case, the column will be padded with spaces up to the length of the column, e.g. inserting 'abc' into char(4) will always result in 'abc '
3) You are somehow not committing the updates, not updating the right column, or other form of user error. The update statement itself looks correct on the face of it.
I had the same issues with RTRIM() AND LTRIM() functions.
In my situation the problem was in LF and CR chars.
Solution
DECLARE #test NVARCHAR(100)
SET #test = 'Declaration status '
SET #test = REPLACE(REPLACE(#test, CHAR(10), ''), CHAR(13), '')
Related
I've tried to use CONCAT function of some fields in a table; in order to get a string that I need to compare onto another field from different table.
However when I use the function it's like it random adds spaces between the fields and then I cannot use this result to compare.
I've tried:
SELECT CONCAT([STC_GL-STC].[ZZGL_Desc_Group_5D],'-',
[STC_GL-STC].[ZZCostCentreGroup],'-',
AS RESULT
FROM [STC_GL-STC];
As an example of result:
'Compras - RM -MATERIA PRIMA -'
(Please note the blank spaces in the second and third (-).
I would need to obtain:
'Compras - RM-MATERIA PRIMA-'
I've checked the values in the fields and there is no blank spaces at the end on fields ZZGL_Desc_Group_5D , ZZCostCentreGroup.
I've also tried:
SELECT CONCAT_WS('-',[ZZGL_Desc_Group_5D],[ZZCostCentreGroup]) AS RESULT
FROM [STC_GL-STC]
With same result.
And finally I tried to remove blank spaces using RTRIM and LTRIM using the following:
SELECT CONCAT(LTRIM(RTRIM([STC_GL-STC].[ZZGL_Desc_Group_5D])),
LTRIM(RTRIM('-')),
LTRIM(RTRIM([STC_GL-STC].[ZZCostCentreGroup]))) AS RESULT
FROM [STC_GL-STC]
ORDER BY RESULT ASC;
And even with LTRIM and RTRIM functions on that field, I still getting the same result.
How to get rid of this behaviour and of the blank spaces? Is there another way to build that string?
Kind Regards and many thanks in advance,
Long time ago I created a udf function to remove white spaces.
It is based on the 'magic' of the XML xs:token data type.
udf
/*
1. All invisible TAB, Carriage Return, and Line Feed characters will be replaced with spaces.
2. Then leading and trailing spaces are removed from the value.
3. Further, contiguous occurrences of more than one space will be replaced with a single space.
*/
CREATE FUNCTION dbo.udf_tokenize(#input VARCHAR(MAX))
RETURNS VARCHAR(MAX)
AS
BEGIN
RETURN (SELECT CAST('<r><![CDATA[' + #input + ']]></r>' AS XML).value('(/r/text())[1] cast as xs:token?','VARCHAR(MAX)'));
END
Test harness
-- DDL and sample data population, start
DECLARE #mockTbl TABLE (ID INT IDENTITY(1,1), col_1 VARCHAR(100), col_2 VARCHAR(100));
INSERT INTO #mockTbl (col_1, col_2)
VALUES (' FL ', ' Miami')
, (' FL ', ' Fort Lauderdale ')
, (' NY ', ' New York ')
, (' NY ', '')
, (' NY ', NULL);
-- DDL and sample data population, end
SELECT *
, col_1n = dbo.udf_tokenize(col_1)
, col_2n = dbo.udf_tokenize(col_2)
, CONCAT_WS('-', dbo.udf_tokenize(col_1), dbo.udf_tokenize(col_2)) AS RESULT
FROM #mockTbl;
I have a column with the name of a person in the following format: "LAST NAME, FIRST NAME"
Only Upper Cases Allowed
Space after comma optional
I would like to use a regular expression like: [A-Z]+,[ ]?[A-Z]+ but I do not know how to do this in T-SQL. In Oracle, I would use REGEXP_LIKE, is there something similar for SQL Server 2016?
I need something like the following:
UPDATE table
SET is_correct_format = 'YES'
WHERE REGEXP_LIKE(table.name,'[A-Z]+,[ ]?[A-Z]+');
First, case sensitivity depends on the collation of the DB, though with LIKE you can specify case comparisons. With that... here is some Boolean logic to take care of the cases you stated. Though, you may need to add additional clauses if you discover some bogus input.
declare #table table (Person varchar(64), is_correct_format varchar(3) default 'NO')
insert into #table (Person)
values
('LowerCase, Here'),
('CORRECTLY, FORMATTED'),
('CORRECTLY,FORMATTEDTWO'),
('ONLY FIRST UPPER, LowerLast'),
('WEGOT, FormaNUMB3RStted'),
('NoComma Formatted'),
('CORRECTLY, TWOCOMMA, A'),
(',COMMA FIRST'),
('COMMA LAST,'),
('SPACE BEFORE COMMA , GOOD'),
(' SPACE AT BEGINNING, GOOD')
update #table
set is_correct_format = 'YES'
where
Person not like '%[^A-Z, ]%' --check for non characters, excluding comma and spaces
and len(replace(Person,' ','')) = len(replace(replace(Person,' ',''),',','')) + 1 --make sure there is only one comma
and charindex(',',Person) <> 1 --make sure the comma isn't at the beginning
and charindex(',',Person) <> len(Person) --make sure the comma isn't at the end
and substring(Person,charindex(',',Person) - 1,1) <> ' ' --make sure there isn't a space before comma
and left(Person,1) <> ' ' --check preceeding spaces
and UPPER(Person) = Person collate Latin1_General_CS_AS --check collation for CI default (only upper cases)
select * from #table
The tsql equivalent could look like this. I'm not vouching for the efficiency of this solution.
declare #table as table(name varchar(20), is_Correct_format varchar(5))
insert into #table(name) Values
('Smith, Jon')
,('se7en, six')
,('Billy bob')
UPDATE #table
SET is_correct_format = 'YES'
WHERE
replace(name, ', ', ',x')
like (replicate('[a-z]', charindex(',', name) - 1)
+ ','
+ replicate('[a-z]', len(name) - charindex(',', name)) )
select * from #table
The optional space is hard to solve, so since it's next to a legal character I'm just replacing with another legal character when it's there.
TSQL does not provide the kind of 'repeating pattern' of * or + in regex, so you have to count the characters and construct the pattern that many times in your search pattern.
I split the string at the comma, counted the alphas before and after, and built a search pattern to match.
Clunky, but doable.
I've created a full-text indexed column on a table.
I have a stored procedure to which I may pass the value of a variable "search this text". I want to search for "search", "this" and "text" within the full-text column. The number of words to search would be variable.
I could use something like
WHERE column LIKE '%search%' OR column LIST '%this%' OR column LIKE '%text%'
But that would require me to use dynamic SQL, which I'm trying to avoid.
How can I use my full-text search to find each of the words, presumably using CONTAINS, and without converting the whole stored procedure to dynamic SQL?
If you say you definitely have SQL Table Full Text Search Enabled, Then you can use query like below.
select * from table where contains(columnname,'"text1" or "text2" or "text3"' )
See link below for details
Full-Text Indexing Workbench
So I think I came up with a solution. I created the following scalar function:
CREATE FUNCTION [dbo].[fn_Util_CONTAINS_SearchString]
(
#searchString NVARCHAR(MAX),
#delimiter NVARCHAR(1) = ' ',
#ANDOR NVARCHAR(3) = 'AND'
)
RETURNS NVARCHAR(MAX)
AS
BEGIN
IF #searchString IS NULL OR LTRIM(RTRIM(#searchString)) = '' RETURN NULL
-- trim leading/trailing spaces
SET #searchString = LTRIM(RTRIM(#searchString))
-- remove double spaces (prevents empty search terms)
WHILE CHARINDEX(' ', #searchString) > 0
BEGIN
SET #searchString = REPLACE(#searchString,' ',' ')
END
-- reformat
SET #searchString = REPLACE(#searchString,' ','" ' + #ANDOR + ' "') -- replace spaces with " AND " (quote) AND (quote)
SET #searchString = ' "' + #searchString + '" ' -- surround string with quotes
RETURN #searchString
END
I can get my results:
DECLARE #ftName NVARCHAR (1024) = dbo.fn_Util_CONTAINS_SearchString('value1 value2',default,default)
SELECT * FROM Table WHERE CONTAINS(name,#ftName)
I would appreciate any comments/suggestions.
For your consideration.
I understand your Senior wants to avoid dynamic SQL, but it is my firm belief that Dynamic SQL is NOT evil.
In the example below, you can see that with a few parameters (or even defaults), and a 3 lines of code, you can:
1) Dynamically search any source
2) Return desired or all elements
3) Rank the Hit rate
The SQL
Declare #SearchFor varchar(max) ='Daily,Production,default' -- any comma delim string
Declare #SearchFrom varchar(150) ='OD' -- table or even a join statment
Declare #SearchExpr varchar(150) ='[OD-Title]+[OD-Class]' -- Any field or even expression
Declare #ReturnCols varchar(150) ='[OD-Nr],[OD-Title]' -- Any field(s) even with alias
Set #SearchFor = 'Sign(CharIndex('''+Replace(Replace(Replace(#SearchFor,' , ',','),', ',''),',',''','+#SearchExpr+'))+Sign(CharIndex(''')+''','+#SearchExpr+'))'
Declare #SQL varchar(Max) = 'Select * from (Select Distinct'+#ReturnCols+',Hits='+#SearchFor+' From '+#SearchFrom + ') A Where Hits>0 Order by Hits Desc'
Exec(#SQL)
Returns
OD-Nr OD-Title Hits
3 Daily Production Summary 2
6 Default Settings 1
I should add that my search string is comma delimited, but you can change to space.
Another note CharIndex can be substanitally faster that LIKE. Take a peek at
http://cc.davelozinski.com/sql/like-vs-substring-vs-leftright-vs-charindex
I have one column for comment and I need to show this for one report.
Here what happen some time, users uses multiple enters in comment box. I can not access code part I need to manage this thing in SQL only.
So I have removed unwanted
1 /r/n
2 /n/n
from using
REPLACE(REPLACE(Desc, CHAR(13)+CHAR(10), CHAR(10)),CHAR(10)+CHAR(10), CHAR(10)) as Desc,
Now I want to remove any \r or \n from starting or ending of the string if any
By the way you meant in your question:(Remove char(10) or char(13) from specific string)
Note: You should see the output result by switching your resultset output to Results to Text(Ctrl+T).
Results to Text
Results to Grid
Use TRIM check here
Example : UPDATE tablename SET descriptions = TRIM(TRAILING "<br>" FROM descriptions)
if you want to replace newline then use something like below
SELECT REPLACE(REPLACE(#str, CHAR(13), ''), CHAR(10), '')
or
DECLARE #testString varchar(255)
set #testString = 'MY STRING '
/*Select the string and try to copy and paste into notepad and tab is still there*/
SELECT testString = #testString
/*Ok, it seems easy, let's try to trim this. Huh, it doesn't work, the same result here.*/
SELECT testStringTrim = RTRIM(#testString)
/*Let's try to get the size*/
SELECT LenOfTestString = LEN(#testString)
/*This supposed to give us string together with blank space, but not for tab though*/
SELECT DataLengthOfString= DATALENGTH(#testString)
SELECT ASCIIOfTab = ASCII(' ')
SELECT CHAR(9)
/*I always use this like a final solution*/
SET #testString = REPLACE(REPLACE(REPLACE(#testString, CHAR(9), ''), CHAR(10), ''), CHAR(13), '') SELECT #testString
/*
CHAR(9) - Tab
CHAR(10) - New Line
CHAR(13) - Carriage Return
*/
Reference
select dbo.trim('abc','c') -- ab
select dbo.trim('abc','a') -- bc
select dbo.trim(' b ',' ') -- b
Create a user-define-function: trim()
trim from both sides
trim any letter: space, \r, \n, etc
Create FUNCTION Trim
(
#Original varchar(max), #letter char(1)
)
RETURNS varchar(max)
AS
BEGIN
DECLARE #rtrim varchar(max)
SELECT #rtrim = iif(right(#original, 1) = #letter, left(#original,datalength(#original)-1), #original)
return iif( left(#rtrim,1) = #letter, right(#rtrim,datalength(#rtrim)-1),#rtrim)
END
I am stuck on converting a varchar column UserID to INT. I know, please don't ask why this UserID column was not created as INT initially, long story.
So I tried this, but it doesn't work. and give me an error:
select CAST(userID AS int) from audit
Error:
Conversion failed when converting the varchar value
'1581............................................................................................................................' to data type int.
I did select len(userID) from audit and it returns 128 characters, which are not spaces.
I tried to detect ASCII characters for those trailing after the ID number and ASCII value = 0.
I have also tried LTRIM, RTRIM, and replace char(0) with '', but does not work.
The only way it works when I tell the fixed number of character like this below, but UserID is not always 4 characters.
select CAST(LEFT(userID, 4) AS int) from audit
You could try updating the table to get rid of these characters:
UPDATE dbo.[audit]
SET UserID = REPLACE(UserID, CHAR(0), '')
WHERE CHARINDEX(CHAR(0), UserID) > 0;
But then you'll also need to fix whatever is putting this bad data into the table in the first place. In the meantime perhaps try:
SELECT CONVERT(INT, REPLACE(UserID, CHAR(0), ''))
FROM dbo.[audit];
But that is not a long term solution. Fix the data (and the data type while you're at it). If you can't fix the data type immediately, then you can quickly find the culprit by adding a check constraint:
ALTER TABLE dbo.[audit]
ADD CONSTRAINT do_not_allow_stupid_data
CHECK (CHARINDEX(CHAR(0), UserID) = 0);
EDIT
Ok, so that is definitely a 4-digit integer followed by six instances of CHAR(0). And the workaround I posted definitely works for me:
DECLARE #foo TABLE(UserID VARCHAR(32));
INSERT #foo SELECT 0x31353831000000000000;
-- this succeeds:
SELECT CONVERT(INT, REPLACE(UserID, CHAR(0), '')) FROM #foo;
-- this fails:
SELECT CONVERT(INT, UserID) FROM #foo;
Please confirm that this code on its own (well, the first SELECT, anyway) works for you. If it does then the error you are getting is from a different non-numeric character in a different row (and if it doesn't then perhaps you have a build where a particular bug hasn't been fixed). To try and narrow it down you can take random values from the following query and then loop through the characters:
SELECT UserID, CONVERT(VARBINARY(32), UserID)
FROM dbo.[audit]
WHERE UserID LIKE '%[^0-9]%';
So take a random row, and then paste the output into a query like this:
DECLARE #x VARCHAR(32), #i INT;
SET #x = CONVERT(VARCHAR(32), 0x...); -- paste the value here
SET #i = 1;
WHILE #i <= LEN(#x)
BEGIN
PRINT RTRIM(#i) + ' = ' + RTRIM(ASCII(SUBSTRING(#x, #i, 1)))
SET #i = #i + 1;
END
This may take some trial and error before you encounter a row that fails for some other reason than CHAR(0) - since you can't really filter out the rows that contain CHAR(0) because they could contain CHAR(0) and CHAR(something else). For all we know you have values in the table like:
SELECT '15' + CHAR(9) + '23' + CHAR(0);
...which also can't be converted to an integer, whether you've replaced CHAR(0) or not.
I know you don't want to hear it, but I am really glad this is painful for people, because now they have more war stories to push back when people make very poor decisions about data types.
This question has got 91,000 views so perhaps many people are looking for a more generic solution to the issue in the title "error converting varchar to INT"
If you are on SQL Server 2012+ one way of handling this invalid data is to use TRY_CAST
SELECT TRY_CAST (userID AS INT)
FROM audit
On previous versions you could use
SELECT CASE
WHEN ISNUMERIC(RTRIM(userID) + '.0e0') = 1
AND LEN(userID) <= 11
THEN CAST(userID AS INT)
END
FROM audit
Both return NULL if the value cannot be cast.
In the specific case that you have in your question with known bad values I would use the following however.
CAST(REPLACE(userID COLLATE Latin1_General_Bin, CHAR(0),'') AS INT)
Trying to replace the null character is often problematic except if using a binary collation.
This is more for someone Searching for a result, than the original post-er. This worked for me...
declare #value varchar(max) = 'sad';
select sum(cast(iif(isnumeric(#value) = 1, #value, 0) as bigint));
returns 0
declare #value varchar(max) = '3';
select sum(cast(iif(isnumeric(#value) = 1, #value, 0) as bigint));
returns 3
I would try triming the number to see what you get:
select len(rtrim(ltrim(userid))) from audit
if that return the correct value then just do:
select convert(int, rtrim(ltrim(userid))) from audit
if that doesn't return the correct value then I would do a replace to remove the empty space:
select convert(int, replace(userid, char(0), '')) from audit
This is how I solved the problem in my case:
First of all I made sure the column I need to convert to integer doesn't contain any spaces:
update data set col1 = TRIM(col1)
I also checked whether the column only contains numeric digits.
You can check it by:
select * from data where col1 like '%[^0-9]%' order by col1
If any nonnumeric values are present, you can save them to another table and remove them from the table you are working on.
select * into nonnumeric_data from data where col1 like '%[^0-9]%'
delete from data where col1 like '%[^0-9]%'
Problems with my data were the cases above. So after fixing them, I created a bigint variable and set the values of the varchar column to the integer column I created.
alter table data add int_col1 bigint
update data set int_col1 = CAST(col1 AS VARCHAR)
This worked for me, hope you find it useful as well.