One of my tables' column contains names, for example as "Obama, Barack" (with double quotes). I was wondering if we can do something to make it appear as Barack Obama in the tables. I think we can do it with declaring a variable but just could not manage to find a solution.
And yes as this table contains the multiple transactions of the same person we also end up with having multiple rows of "Obama, Barack"... a data warehouse concept (fact tables).
What #Ben has said is correct. Having two columns one for first name and one for last name is correct.
However if you wish to update the entire database as it is you could do...
/*This will swap the order round*/
UPDATE TableName SET NameColumn = SUBSTRING(NameColumn, 1, CHARINDEX(',',NameColumn))+SUBSTRING(NameColumn, CHARINDEX(',', NameColumn),LEN(NameColumn)-CHARINDEX('"', NameColumn,2))
/*This will remove the quotes*/
UPDATE TableName SET NameColumn = REPLACE(NameColumn, '"', '')
Edit:- but as I can't see your data you may have to edit it slightly. But the theory is correct. See here http://www.technoreader.com/SQL-Server-String-Functions.aspx
From the question I assume you want to:
Remove the quotes
Remove the comma
Swap the names
So Regexp_replace is probably your best bet
UPDATE tablename
SET column_name = REGEXP_REPLACE( column_name, '^"(\w+), (\w+)"$', '\2 \1' )
So regexp_replace is changing the column value as long as it matches the pattern exactly. What the parts of the expression are
^" means it must start with a double quote
(\w+) means immediately followed by a string of 1 or more alphanumeric characters. This string is then saved as the variable \1 because its the first set of ()
, means immediately followed by a comma and a space
(\w+) means immediately followed by a string of 1 or more alphanumeric characters. This string is then saved as the variable \2 because its the second set of ()
"$ means immediately follwed by a double quote which is the end of the string
\2 \1 is the replacement string, the second saved string followed by a space followed by the first saved string
So anything which does not exactly match these conditions will not be replaced. So if you have an leading or traling spaces, or more than one space after the comma, or many other reasons the text will not be replaced.
A much more flexible (maybe too flexible) option could be:
UPDATE tablename
SET column_name = REGEXP_REPLACE( column_name, '^\W*(\w+)\W+(\w+)\W*$', '\2 \1' )
This is similar but effectively makes the quotes and the comma optional, and deals with any other leading or trailing pubctuation or whitespace.
^\W* means must start with zero or more non-alphanumberics
(\w+)\W+(\w+) means two alphanumberic strings separated by one or more non-alphanumerics. The two strings are saved as described above
\W*$ means must then end with zero or more non-alphanumberics
More info on regexp in oracle is here
http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/ap_posix.htm
http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14251/adfns_regexp.htm
Related
I am working with some geo-spatial (string) data that i have been saving into an SQL server database, whenever my data has had a single quote within the string i am saving, i have tried to escape this by padding the single quote. Though this allows me to save the text into my database, when i try to use a where clause to find locations which have been had an escape sequence applied to them, the where clause does not seem to pick them up.
For example
SELECT * from tb_test WHERE Address = 'Abbey Gardens, St John''s Wood, London, NW8';
Will produce an empty table, even if this record exists. The only want i have been able to get this to work was by using LIKE to match the pattern of my text which does not include the quotation (see below).
SELECT * from tb_test WHERE Address LIKE 'Abbey Gardens'
Is there any reason as to why SQL does not pick up on the full Address ? and if so how can i get it to?
SELECT * from tb_test WHERE Address = 'Abbey Gardens, St John''s Wood,
London, NW8'; Will produce an empty table, even if this record exists.
The quotes are proper in the literal so it seems the actual column value does not match the literal value. This could be due to trailing whitespace (e.g. tab, newline, etc.). Check the raw binary value of the column for unexpected characters:
SELECT CAST(Address AS varbinary(MAX)), *
FROM tb_test
WHERE Address LIKE 'Abbey Gardens%';
I need to find invalid social security numbers in a varchar field in a SQL Server 2008 database table. (Valid SSNs are being defined by being in the format ###-##-#### - doesn't matter what the numbers are, as long as they are in that "3-digit dash 2-digit dash 4-digit" pattern.
I do have a working regex:
SELECT *
FROM mytable
WHERE ssn NOT LIKE '[0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]'
That does find the invalid SSNs in the column, but I know (okay - I'm pretty sure) that there is a way to shorten that to indicate that the previous pattern can have x iterations.
I thought this would work:
'[0-9]{3}-[0-9]{2}-[0-9]{4}'
But it doesn't.
Is there a shorter regex than the one above in the select, or not? Or perhaps there is, but T-SQL/SQL Server 2008 doesn't support it!?
If you plan to get a shorter variant of your LIKE expression, then the answer is no.
In T-SQL, you can only use the following wildcards in the pattern:
%
- Any string of zero or more characters.
WHERE title LIKE '%computer%' finds all book titles with the word computer anywhere in the book title.
_ (underscore)
Any single character.
WHERE au_fname LIKE '_ean' finds all four-letter first names that end with ean (Dean, Sean, and so on).
[ ]
Any single character within the specified range ([a-f]) or set ([abcdef]).
WHERE au_lname LIKE '[C-P]arsen' finds author last names ending with arsen and starting with any single character between C and P, for example Carsen, Larsen, Karsen, and so on. In range searches, the characters included in the range may vary depending on the sorting rules of the collation.
[^]
Any single character not within the specified range ([^a-f]) or set ([^abcdef]).
So, your LIKE statement is already the shortest possible expression. No limiting quantifiers can be used (those like {min,max}), not shorthand classes like \d.
If you were using MySQL, you could use a richer set of regex utilities, but it is not the case.
I suggest you to use another solution like this:
-- Use `REPLICATE` if you really want to use a number to repeat
Declare #rgx nvarchar(max) = REPLICATE('#', 3) + '-' +
REPLICATE('#', 2) + '-' +
REPLICATE('#', 4);
-- or use your simple format string
Declare #rgx nvarchar(max) = '###-##-####';
-- then use this to get your final `LIKE` string.
Set #rgx = REPLACE(#rgx, '#', '[0-9]');
And you can also use something like '_' for characters then replace it with [A-Z] and so on.
There's very little documentation available about escaping characters in SQL Server BULK INSERT files.
The documentation for BULK INSERT says the statement only has two formatting options: FIELDTERMINATOR and ROWTERMINATOR, however it doesn't say how you're meant to escape those characters if they appear in a row's field value.
For example, if I have this table:
CREATE TABLE People ( name varchar(MAX), notes varchar(MAX) )
and this single row of data:
"Foo, \Bar", "he has a\r\nvery strange name\r\nlol"
...how would its corresponding bulk insert file look like, because this wouldn't work for obvious reasons:
Foo,\Bar,he has a
very strange name
lol
SQL Server says it supports \r and \n but doesn't say if backslashes escape themselves, nor does it mention field value delimiting (e.g. with double-quotes, or escaping double-quotes) so I'm a little perplexed in this area.
I worked-around this issue by using \0 as a row separator and \t as a field separator, as neither character appeared as a field value and are both supported as separators by BULK INSERT.
I am surprised MSSQL doesn't offer more flexibility when it comes to import/export. It wouldn't take too much effort to build a first-class CSV/TSV parser.
For the next person to search:
I used "\0\t" as a field separator, and "\0\n" for the end-of-line separator on the last field. Use of "\0\r\n" would also be acceptable if you wish to pretend that the files have DOS EOL conventions.
For those unfamiliar with the \x notation, \0 is CHAR(0), \t is CHAR(9), \n is CHAR(10) and \r is CHAR(13). Replace the CHAR() function with whatever your language offers to convert a number to a nominated character.
With this combination, all instances of \t and \n (and \r) become acceptable characters in the data file. After all, the weakness of the bulk upload system is that tabs and newlines are often legitimate characters in text strings, whereas other low-ASCII characters like CHAR(0), CHAR(1) and CHAR(2) are not legal text - not even appearing in UTF-8.
The only character you cannot have in your data is \0 - UNLESS you can guarantee it will never be followed by \t or \n (or \r)
If your language suffers problems when you use \0 in strings (but depending on how you code, you may still be able to avoid that problem) - AND if you know that your data won't have CHAR(1) or CHAR(2) in it (ie no binary) then use those characters instead. Those low characters are only going to be found when you are trying to store arbitrary binary data in strings.
Note also that you will find bytes 0, 1, 2 in UTF-16, UCS-2 and UTF-32 (aka UCS-4) - BUT - the 2 or 4 byte wide representation of CHAR(0, 1 or 2) is still acceptable and distinct from any legal unicode text. Just make sure you select the correct codepage setting in the format file to suit your choice of a UTF or UCS variant.
A bulk insert needs to have corresponding fields and field count for each row. Your example is a little rough, as its not structured data. As for thecharacters it will interpret them literally, not using escape characters (your string will be as seen in the file.
As for the double quotes enclosing each field, you will just have to use them as field and row terminators as well. So now your you should have:
Fieldterminator = '","',
Rowterminator = '"\n'
Does that make sense? Then after the bulk insert you'll need to take out the prefix double quote with something like:
Update yourtable
set yourfirstcolumn = right(yourfirstcolumn, len(yourfirstcolumn) - 1)
I ran into something a little odd this morning and thought I'd submit it for commentary.
Can someone explain why the following SQL query prints 'equal' when run against SQL 2008. The db compatibility level is set to 100.
if '' = ' '
print 'equal'
else
print 'not equal'
And this returns 0:
select (LEN(' '))
It appears to be auto trimming the space. I have no idea if this was the case in previous versions of SQL Server, and I no longer have any around to even test it.
I ran into this because a production query was returning incorrect results. I cannot find this behavior documented anywhere.
Does anyone have any information on this?
varchars and equality are thorny in TSQL. The LEN function says:
Returns the number of characters, rather than the number of bytes, of the given string expression, excluding trailing blanks.
You need to use DATALENGTH to get a true byte count of the data in question. If you have unicode data, note that the value you get in this situation will not be the same as the length of the text.
print(DATALENGTH(' ')) --1
print(LEN(' ')) --0
When it comes to equality of expressions, the two strings are compared for equality like this:
Get Shorter string
Pad with blanks until length equals that of longer string
Compare the two
It's the middle step that is causing unexpected results - after that step, you are effectively comparing whitespace against whitespace - hence they are seen to be equal.
LIKE behaves better than = in the "blanks" situation because it doesn't perform blank-padding on the pattern you were trying to match:
if '' = ' '
print 'eq'
else
print 'ne'
Will give eq while:
if '' LIKE ' '
print 'eq'
else
print 'ne'
Will give ne
Careful with LIKE though: it is not symmetrical: it treats trailing whitespace as significant in the pattern (RHS) but not the match expression (LHS). The following is taken from here:
declare #Space nvarchar(10)
declare #Space2 nvarchar(10)
set #Space = ''
set #Space2 = ' '
if #Space like #Space2
print '#Space Like #Space2'
else
print '#Space Not Like #Space2'
if #Space2 like #Space
print '#Space2 Like #Space'
else
print '#Space2 Not Like #Space'
#Space Not Like #Space2
#Space2 Like #Space
The = operator in T-SQL is not so much "equals" as it is "are the same word/phrase, according to the collation of the expression's context," and LEN is "the number of characters in the word/phrase." No collations treat trailing blanks as part of the word/phrase preceding them (though they do treat leading blanks as part of the string they precede).
If you need to distinguish 'this' from 'this ', you shouldn't use the "are the same word or phrase" operator because 'this' and 'this ' are the same word.
Contributing to the way = works is the idea that the string-equality operator should depend on its arguments' contents and on the collation context of the expression, but it shouldn't depend on the types of the arguments, if they are both string types.
The natural language concept of "these are the same word" isn't typically precise enough to be able to be captured by a mathematical operator like =, and there's no concept of string type in natural language. Context (i.e., collation) matters (and exists in natural language) and is part of the story, and additional properties (some that seem quirky) are part of the definition of = in order to make it well-defined in the unnatural world of data.
On the type issue, you wouldn't want words to change when they are stored in different string types. For example, the types VARCHAR(10), CHAR(10), and CHAR(3) can all hold representations of the word 'cat', and ? = 'cat' should let us decide if a value of any of these types holds the word 'cat' (with issues of case and accent determined by the collation).
Response to JohnFx's comment:
See Using char and varchar Data in Books Online. Quoting from that page, emphasis mine:
Each char and varchar data value has a collation. Collations define
attributes such as the bit patterns used to represent each character,
comparison rules, and sensitivity to case or accenting.
I agree it could be easier to find, but it's documented.
Worth noting, too, is that SQL's semantics, where = has to do with the real-world data and the context of the comparison (as opposed to something about bits stored on the computer) has been part of SQL for a long time. The premise of RDBMSs and SQL is the faithful representation of real-world data, hence its support for collations many years before similar ideas (such as CultureInfo) entered the realm of Algol-like languages. The premise of those languages (at least until very recently) was problem-solving in engineering, not management of business data. (Recently, the use of similar languages in non-engineering applications like search is making some inroads, but Java, C#, and so on are still struggling with their non-businessy roots.)
In my opinion, it's not fair to criticize SQL for being different from "most programming languages." SQL was designed to support a framework for business data modeling that's very different from engineering, so the language is different (and better for its goal).
Heck, when SQL was first specified, some languages didn't have any built-in string type. And in some languages still, the equals operator between strings doesn't compare character data at all, but compares references! It wouldn't surprise me if in another decade or two, the idea that == is culture-dependent becomes the norm.
I found this blog article which describes the behavior and explains why.
The SQL standard requires that string
comparisons, effectively, pad the
shorter string with space characters.
This leads to the surprising result
that N'' = N' ' (the empty string
equals a string of one or more space
characters) and more generally any
string equals another string if they
differ only by trailing spaces. This
can be a problem in some contexts.
More information also available in MSKB316626
There was a similar question a while ago where I looked into a similar problem here
Instead of LEN(' '), use DATALENGTH(' ') - that gives you the correct value.
The solutions were to use a LIKE clause as explained in my answer in there, and/or include a 2nd condition in the WHERE clause to check DATALENGTH too.
Have a read of that question and links in there.
To compare a value to a literal space, you may also use this technique as an alternative to the LIKE statement:
IF ASCII('') = 32 PRINT 'equal' ELSE PRINT 'not equal'
Sometimes one has to deal with spaces in data, with or without any other characters, even though the idea of using Null is better - but not always usable.
I did run into the described situation and solved it this way:
... where ('>' + #space + '<') <> ('>' + #space2 + '<')
Of course you wouldn't do that for large amount of data but it works quick and easy for some hundred lines ...
As SQL - 92 8.2 comparison predicate saying:
If the length in characters of X is not equal to the length
in characters of Y, then the shorter string is effectively
replaced, for the purposes of comparison, with a copy of
itself that has been extended to the length of the longer
string by concatenation on the right of one or more pad char-
acters, where the pad character is chosen based on CS. If
CS has the NO PAD attribute, then the pad character is an
implementation-dependent character different from any char-
acter in the character set of X and Y that collates less
than any string under CS. Otherwise, the pad character is a
<space>.
How to distinct records on select with fields char/varchar on sql server:
example:
declare #mayvar as varchar(10)
set #mayvar = 'data '
select mykey, myfield from mytable where myfield = #mayvar
expected
mykey (int) | myfield (varchar10)
1 | 'data '
obtained
mykey | myfield
1 | 'data'
2 | 'data '
even if I write
select mykey, myfield from mytable where myfield = 'data' (without final blank)
I get the same results.
how I solved? In this mode:
select mykey, myfield
from mytable
where myfield = #mayvar
and DATALENGTH(isnull(myfield,'')) = DATALENGTH(#mayvar)
and if there is an index on myfield, it'll be used in each case.
I hope it will be helpful.
Another way is to put it back into a state that the space has value.
eg: replace the space with a character known like the _
if REPLACE('hello',' ','_') = REPLACE('hello ',' ','_')
print 'equal'
else
print 'not equal'
returns: not equal
Not ideal, and probably slow, but is another quick way forward when needed quickly.
A table in a MYSQL database has address details- eg...
add1, add2, add3, district, postalTown, country
Ordering by postal town is usually fine, but some details have numbers in the postalTown column. For example 1420 Territet or 3100 Overijse. This will mean these will appear at the top above Aberdeen or Bristol. Is there a way of ordering by postalTown but by the first alphabetical character? That would mean the order of the above would be- Aberdeen, Bristol, Overijse, Territet
Thanks
Write an expression that will return the first alphabetical character, then just Order By [that expression]
Order By substring(LTrim(
Replace(Replace(Replace(Replace(Replace(
Replace(Replace(Replace(Replace(Replace(
colname, '1', ''),'2',''),'3',''),'4,''),'5', ''),
'6',''),'7',''),'8',''),'9',''),'0',''))
1,1)
If you want the rows sorted by the entire city name, and not just by the first character (as question title specifies) then use this:
Order By LTrim(
Replace(Replace(Replace(Replace(Replace(
Replace(Replace(Replace(Replace(Replace(
colname, '1', ''),'2',''),'3',''),'4,''),'5', ''),
'6',''),'7',''),'8',''),'9',''),'0',''))
Above is a guess (I haven't tried it), but the idea is first delete all numeric characters from the column value, then take the first character of whatever is remaining.
Also, if this works, and if you have any development access to the dataabse, (thinking DRY principle), I would add a computed column to this table, (or a separate view against the table), that is defined to use the above expression, so that this "extraction" of the town name is available to all other code that might want to access it without copying this expression everywhere you may need it..
You could write a stored function which returns the remainder of the column starting at the first alphabetic character (perhaps using REGEXP to find that index). Then order by the stored function.
Edit: instead of regexp in your function, depending on data format you could do a 'substring_index' on ' ' (space) and return the index of the first space, then call substring to return the remainder of the string after the first space.
Once you've created a stored function to return the string following the numbers, you can utilize it like this:
order by yourfunctionname(postalTown)
Stored Functions
First thing that comes to mind to me would to do the following on my ORDER BY, obviously, adding numbers 0 through 9. You'll notice crappy schemas produce crappy solutions. :) As the gentleman said above, you should probably think about a redesign of how you are storing your town data.
ORDER BY REPLACE(REPLACE(REPLACE(FieldName, '1', ''),'2',''),'3','') ETC.
Create a view on the table, making whatever translations you need, and then query against the view?