Not sure if this is an appropriate "programming" issue but its relevant.
Is there a way via SQL or excel even to add quotes to a big number of values, for example i want to add values a,b,c,d,e,f,g,j etc etc to a table is there a way i can automatically add quotes to them? as in 'a','b' etc...
i have this issue especially in my select * from table where column in ('value1','value2')...
Thanks...
I usually tackle this sort of issue using Excel - if you enter your source values in a column, you can then just use an Excel formula to concatenate the values with quotes around them (eg =CONCATENATE("'", A1, "'")), and even extend this to build the complete SQL statement.
Not sure if this helps or is what you were asking, but for such queries you should use parameters, i.e. something like
SELECT ... WHERE column IN (?,?)
This is not only an easy approach, but also means that if somebody puts a single-quote in the value they give you, the form of your query can't be altered - that's how SQL Injection security attacks happen.
If you have values in the same column of another table you coud use a select query
select '''' + cast(ValueColumn as nvarchar) + '''' as QuotedVal from MyTable
and then do your copy/paste
Related
I am trying to see if there is an easy answer for this. I have done something similar using multiple pick dropdown parameters in SSRS but this appears to be different.
My scenario is this, so maybe there is an even better answer.
I have a production server that I do not want to make any changes to including temp tables or functions. The production server has a table of clients with about 1600 records. I have set up an SSIS package that will allow transfer of data from production to dev based on a clientid. So my sources would have a query similar to Select Field From Table Where ClientId = ?
This works fine. Now I want to load more than one client, based an data in the clients table. It may be Select ClientId From Clients where Field = A and returns multiple ClientIds.
I am able to populate a comma delimited list from an execute sql task to a SSIS variable, so it maybe 1,4,8.
If I change my source query to use ClientId in (?) I get a conversion error.
I have looked at many posts that advocate a temp table or a function which I want to avoid. Select IN using varchar string with comma delimited values
I have contemplated building the entire sql statement into a variable but this don't seem like the right path as I have many tables to query and transfer where using ClientId = ? works well without having to build each individual SQL statement to a variable.
Is there an easy fix I am missing? I will turn my research now to try to find out how I did this in SSRS but I thought that I should try a post here to see if someone has accomplished this before.
I appreciate any info on this, thank you.
EDIT: Key note is that the column on clients is on the dev server, so I cannot just use a select in the where clause as the column does not exist on the production server.
EDIT: I did not mention that I am specifically looking at OLEDB sources mapping a parameter to ? in the sql statement.
EDIT: Narrowing down on this but having trouble relating SSRS and SSIS functionality. In SSRS its called a multi-value parameter in the following link the key line is
WHERE Production.ProductInventory.ProductID IN (#ProductID)
https://msdn.microsoft.com/en-us/library/dn385719(v=sql.110).aspx
This one looks good as well
https://sqlblogcasts.com/blogs/simons/archive/2007/11/22/RS-HowTo---Pass-a-multivalue-parameter-to-a-query-using-IN.aspx
I will keep researching and thank you for the help so far.
I think this sums it up best
This functionality is limited to strictly using embedded SQL.
What SSRS does is transform your SQL column IN (#value) to column IN
(#selectedvalue1,#selectedvalue2) etc.
You need to forget anything you have about the other ways of passing
lists to SQL i.e. building strings etc. and make sure you declare the
data types are correct for the value of your parameter.
You do not need to use the Join(parameters!,",") trick UNLESS
you are passing the list to a stored procedure.
In which case you then need to use some function to turn the delimited
list into a rowset as you have done.
I hope that helps
The core question is if I can get the same functionality in SSIS as in SSRS. It reminds me of macro substitution..
If you dont want to create a function, you can use the following in your t-sql statement.
Declare #ClientIds nvarchar(50) = '123,456'; --<-- Comma delimited list of Client Ids
Select Field
From Table
Where ClientId IN (
SELECT CAST(RTRIM(LTRIM(Split.a.value('.', 'VARCHAR(100)'))) AS INT) ClientIDs
FROM (
SELECT Cast ('<X>'
+ Replace(#ClientIds, ',', '</X><X>')
+ '</X>' AS XML) AS Data
) AS t CROSS APPLY Data.nodes ('/X') AS Split(a)
)
I'm working with a new REST backend talking to a SQL Server. Our REST api allows for the caller to pass in the columns/fields they want returned (?fields=id,name,phone).
The idea seems very normal. The issue I'm bumping up against is resistance to dynamically generating the SQL statement. Any arguments passed in would be passed to the database using a parameterized query, so I'm not concerned about SQL injection.
The basic idea would be to "inject" the column-names passed in, into a SQL that looks like:
SELECT <column-names>
FROM myTable
ORDER BY <column-name-to-sort-by>
LIMIT 1000
We sanitize all column names and verify their existence in the table, to prevent SQL injection issues. Most of our programmers are used to having all SQL in static files, and loading them from disk and passing them on to the database. The idea of code creating SQL makes them very nervous.
I guess I'm curious if others actually do this? If so, how do you do this? If not, how do you manage "dynamic columns and dynamic sort-by" requests passed in?
I think a lot of people do it especially when it comes to reporting features. There are actually two things one should do to stay on the safe side:
Parameterize all WHERE clause values
Use user input values to pick correct column/table names, don't use the user values in the sql statement at all
To elaborate on item #2, I would have a dictionary where Key is a possible user input and Value is a correponding column/table name. You can store this dictionary wherever you want: config file, database, hard code, etc. So when you process user input you just check a dictionary if the Key exists and if it does you use the Value to add a column name to your query. This way you just use user input to pick required column names but don't use the actual values in your sql statement. Besides, you might not want to expose all columns. With a predefined dictionary you can easily control the list of available columns for a user.
Hope it helps!
I've done similar to what Maksym suggests. In my case, keys were pulled directly from the database system tables (after scrubbing the user request a bit for syntactic hacks and permissions).
The following query takes care of some minor injection issues through the natural way SQL handles the LIKE condition. This doesn't go as far as handling permissions on each field (as some fields are forbidden based on the log-in) but it provides a very basic way to retrieve these fields dynamically.
CREATE PROC get_allowed_column_names
#input VARCHAR(MAX)
AS BEGIN
SELECT
columns.name AS allowed_column_name
FROM
syscolumns AS columns,
sysobjects AS tables
WHERE
columns.id = tables.id AND
tables.name = 'Categories' AND
#input LIKE '%' + columns.name + '%'
END
GO
-- The following only returns "Picture"
EXEC get_allowed_column_names 'Category_,Cat%,Picture'
GO
-- The following returns both "CategoryID and Picture"
EXEC get_allowed_column_names 'CategoryID, Picture'
GO
I have a trigger that takes the columns (and their values) from the inserted table and inserts them as text in an audit table, example:
INSERT INTO audit
(tablename, changes)
SELECT
'mytable',
'id=' + cast(id as nvarchar(50) + ';name=' + name + ';etc...'
FROM
inserted
I have large tables with most columns being non-varchar. In order to concatenate them into a string I need to cast each and every column.
Is it necessary to do so? Is there a better way?
The second (unmarked answer) in this question concatenates the values smartly using xml and cross apply.
Is there a way to expand it to include the column names so that the final result would be:
'id=1;name=myname;amount=100;' etc....
Yes, you need to cast non-character data to a string before you can concatenate it. You might want to use CONVERT instead for data that is susceptible to regional formatting (I'm specifically thinking dates here) to ensure you get a deterministic result. You also need to handle nullable columns, i.e. ISNULL(CAST(MyColumn AS VARCHAR(100)), '') - if you concatenate a NULL it will NULL the whole string.
Why not just save it as xml?
select * from inserted for xml auto
It usually takes less space than a simple text column, and you can treat it as (relatively) normalized data. And most importantly, you don't have to handle converting all the complicated stuff manually (how does your code handle end-lines, quotes...?).
In fact, you can even add indices to xml columns, so it can even be practical to search through. Even without indices, it's going to be much faster searching e.g. all records changed in mytable that set name to some value.
And of course, you can keep the same piece of code for all your audited tables - no need to keep them in sync with the table structures etc. (unless you want to select only some explicit columns, of course).
I have an old SQL Server 2000 database from which I read data. My problem is every query involving a String returns a value the size of that column filled with blank spaces.
e.g: let's say we have a column called NAME CHAR(20). All queries would return:
"John "
instead of just "John".
Is there a configuration or parameter in my database that causes this, or anything at all that can be changed to avoid it? Thank you.
EDIT:
I'd like to clarify, I'm reading my DB using JPA Repositories. I don't want to physically remove the whitespaces from the columns, or trim the values manually using RTRIM/LTRIM/REPLACE. I'm just trying to retrieve the column without trailing spaces, without adding any extra strain to the query or trimming the fields programatically.
you can use REPLACE/RTRIM/LTRIM
select RTRIM(LTRIM(column_name)) from table_name
or
select replace(column_name, ' ', '') from table_name
I've noticed that Visual Studio 2008 is placing square brackets around column names in sql. Do the brackets offer any advantage? When I hand code T-SQL I've never bothered with them.
Example:
Visual Studio:
SELECT [column1], [column2] etc...
My own way:
SELECT column1, column2 etc...
The brackets are required if you use keywords or special chars in the column names or identifiers. You could name a column [First Name] (with a space) – but then you'd need to use brackets every time you referred to that column.
The newer tools add them everywhere just in case or for consistency.
They're handy if your columns have the same names as SQL keywords, or have spaces in them.
Example:
create table test ( id int, user varchar(20) )
Oh no! Incorrect syntax near the keyword 'user'.
But this:
create table test ( id int, [user] varchar(20) )
Works fine.
They are useful if you are (for some reason) using column names with certain characters for example.
Select First Name From People
would not work, but putting square brackets around the column name would work
Select [First Name] From People
In short, it's a way of explicitly declaring a object name; column, table, database, user or server.
During the dark ages of SQL in the 1990s it was a good practice as the SQL designers were trying to add each word in the dictionary as keyword for endless avalanche of new features and they called it the SQL3 draft.
So it keeps forward compatibility.
And i found that it has another nice side effect, it helps a lot when you use grep in code reviews and refactoring.
Regardless of following a naming convention that avoids using reserved words, Microsoft does add new reserved words. Using brackets allows your code to be upgraded to a new SQL Server version, without first needing to edit Microsoft's newly reserved words out of your client code. That editing can be a significant concern. It may cause your project to be prematurely retired....
Brackets can also be useful when you want to Replace All in a script. If your batch contains a variable named #String and a column named [String], you can rename the column to [NewString], without renaming #String to #NewString.
Column names can contain characters and reserved words that will confuse the query execution engine, so placing brackets around them at all times prevents this from happening. Easier than checking for an issue and then dealing with it, I guess.
The brackets can be used when column names are reserved words.
If you are programatically generating the SQL statement from a collection of column names you don't control, then you can avoid problems by always using the brackets.
In addition
Some Sharepoint databases contain hyphens in their names. Using square brackets in SQL Statements allow the names to be parsed correctly.
They are useful to identify each elements in SQL.
For example:
CREATE TABLE SchemaName.TableName (
This would actually create a table by the name SchemaName.TableName under default dbo schema even though the intention might be to create the table inside the SchemaName schema.
The correct way would be the following:
CREATE TABLE [SchemaName].[TableName] (
Now it it knows what is the table name and in which schema should it be created in (rightly in the SchemaName schema and not in the default dbo schema)
I believe it adds them there for consistency... they're only required when you have a space or special character in the column name, but it's cleaner to just include them all the time when the IDE generates SQL.