I'm really bad at SQL and couldn't find anything near what I really need. I'm trying to create a Stored Procedure that should run each night to check if records in my database have an equivalent file on a server with all our data.
Example: Record with a mp4 has: [Spotnumber] -> 0000001. Then my procedure should check (not locally) if the file exists on the other server with this number.
Also the place where it should look could be fore exemple (not locally) C:/Spots. And in this directory there'll be subdirectories like: 2013, 2012, 2011. It should check in each directory if it doesn't exists.
For this I was thinking to make something like this: Single check. But this one searches locally and already has the url in a table-field. This won't be possible for mine.
So my question is: Is it even possible to do this with just a SQL procedure? If yes how should I make it check all the files on another server (what path should I use?) + How can I make it check for each record in each subdirectory?
I would suggest another approach.
Instead of using Sql server to check if the file exists then update the db.
Why don't you use a powershell script checking if a file exist, then in this powershell script update the database. With a little search on google you can find all functions on microsoft blogs explaining how to check if a file exists and update file.
Another solution, you could create an assembly in your database with a .net language and work with that.
Last possibility, i think it can be possible too with SSRS.
If you really need to do that with tsql, you should allow xcmd command on your server, then use xcmd... but it means every body could use xcmd. It's not designed too and not suggested :)
Related
this is one that has me stumped and Ive been doing this a long while.
Migrating to SQL server 2016, large number of ETL. Easy enough.
One of the ETL packages has a simple script task to take a table of files, run a file exists foreach loop.
it uses a project parameter to create the unc ( \servername\share) and then binds that to the file name in the script task.
use an environment config setup in SSISDB
execute in SSDT works fine, deploy to catalog and it cant see the file. i know youll say permissions, but ive permissioned everyone group to share and drive in case its that. SSISDB execution means it should be running under my security context and im domain admin, local admin and creator owner of the share.
even strangeR, i have created simple package to grab the contents of one of the files and import into a dump table in case permissions or pathway were duff ( even though they work in SSDT might be the enviroNment config in SSISDB). THIS WORKS FINE, therefore it cant be the envrionment setup of SSISDB being referenced.
please note this is not running from an agent job yet so wont be due to agent server account issue. need to get it running from ssisdb first then ill create an agent job
So -- script task cant see unc share, built from two variables, that works in ssdt and its running under same credentials...
Go
For what its work the script task code is
Dts.Variables("BolFileExists").Value = File.Exists(Dts.Variables("StrLoadFileLocation").Value.ToString & Dts.Variables("StrCurrentFile").Value.ToString)
This is a slightly different answer as it shows a different approach and removes the script task. I use a foreach to check if the file exists using GUI tools provided by SSIS:
Well I found the answer and I deserve to punch myself in the face.
Tried everything, it was a file variable and path variable being pulled together in the script task so tried concatenation that before the script task, pumped this into a table to ensure it was going to write table.
Literally everything was fine and still didn’t work.
The issue....
Building it as a 2017 package onto a 2016 Sql server.
I’ve not found what was missing dll wise but it must have been one of those that meant the script task couldn’t find the files but weird it didn’t break and just said the files weren’t there!
Thanks all for input, I’m going to go put my head in the door and slam it
My SSIS projects tend to run queries that require changes as they move between environments, like the table schema might change or a value in the Where clause. I've always either put my SQL into a Project Parameter, which is hard to edit since formatting is lost, or just put it directly into the Execute SQL Task/Data Flow Source then manually edited it between migrations which is also not ideal.
I was wonder though if I added my SQL scripts to files within the project, can these be read back in? Example if I put a query like this:
select id, name from %schema%.tablename
I'd like to read this into a variable then it's easy to use an expression as I do with Project Parameters to replace %schema% with the appropriate value. Then the .sql files within the project can be edited with little effort or even tested through an Execute SQL Task that's disabled/removed before the project goes into the deployment flow. But I've been unable to find how to read in a file using a relative path within the project. Also I'm not even sure these get deployed to the SSIS Server.
Thanks for any insight.
I've added a text file query.sql to an SSIS (SQL 2017) Project in Visual Studio, bit I've found no way to pull the contents of query.sql into a variable.
Native tooling approach
For an Execute SQL Task, there's an option to source your query directly from a file.
Set your SQLSourceType to File Connection and then specify a file connection manager in the FileConnection section.
Do be aware that while this is handy, it's also ripe for someone escalating their permissions. If I had access to the file the SSIS package is looking for, I can add a drop database, create a new user and give them SA rights, etc - anything the account that runs the SSIS package can do, a nefarious person could exploit.
Roll your own approach
If you're adamant about reading the file yourself, add two Variables to your SSIS package and supply values like the following
User::QueryPath -> String -> C:\path\to\file.sql
User::QueryActual -> String -> SELECT 1;
Add a Script Task to the package. Specify as a ReadOnly variable User::QueryPath and specify as a ReadWrite variable User::QueryActual
Within the Main you'd need code like the following
string filePath = this.Dts.Variables["User::QueryPath"].Value.ToString();
this.Dts.Variables["User::QueryActual"].Value = System.IO.File.ReadAllText(filePath);
The meat of the matter is System.IO.File.ReadAllText. Note that this doesn't handle checking whether the file exists, you have permission to access, etc. It's just a barebones read of a file (and also open to the same injection challenges as the above method - just this way you own maintaining it versus the fine engineers at Microsoft)
You can build your query by using both Variable and Parameter.
For example:
Parameter A: dbo
Build your variable A (string type) as : "Select * FROM server.DB." + ParameterA + ".Table"
So if you need to change the schema, just change the parameter A will give you the corresponding query in variable A.
I have numerous scripts to run in SQL Server.
Currently I've made many scripts which have been saved as file and can be loaded in SSMS when needed.
I want to know if we have any shortcut(bookmark) in SQL server to script files.
So far I've found nothing on internet. (note that creating stored procedures for these ad-hoc scripts is not an option)
By pressing Cntrl+K, Cntrl+W combination you can see the Bookmark manager.
Here you can see all bookmarks in locally saved sql queries.
Like in the below screenshot I have a bookmark for getLocationBasedMemberVisitsData.Sqlfile.
On how to add a bookmark
goto Edit>Bookmarks>toggle bookmark when you are inside your file with your edit cursor at the line you want to bookmark
P.S.: You can also create a folder hierarchy much like in browsers like Chrome
Store all needed files inside a single solution. Call it Utilities. Much like VS you can browse through the solution files using Solution explorer.
Caveat: The bookmarks on loose sql files are not persisted. So on a restart(close/open) for SSMS tool these are lost.
If you do need to persist bookmarks, create bound files i.e. store the files in a solution first.
Also the bookmarks in Management Studio are consist with Visual Studio.
Addendum:
Based on asker's comment
This just bookmark a line on a script. Is these any way to give these bookmarks a name?
You can rename both the folders as well as bookmark name by double clicking on it or by right click>rename option
On another comment
What do you mean by saving as as solution. I cannot find such option in SSMS. Are you referring to VS?
SQL server management studio like VS supports creation of Solutions(project) which are essentially collection of related queries/SP with connection information and other needed stuff.
To create a new solution press cntrl+shift+N. see screen shot below
Any bookmarks on solution files are sticky. Next time you open the solutions, bookmarks are presented back to you.
The SQL solution can be opened in VS too. The bookmark information along with other user information is stored in a *.ssms_suo file much like a *.suo file of VS.
It is a good idea to store all helpful files in a utilities solution which can be open all the time so that you can easily access it.
I think a Snippet is what you are asking for:
A Transact-SQL.code snippet is a template containing the basic
structure of a Transact-SQL statement or block. You can use snippets
as a starting point when adding statements in the Database Engine
Query Editor. You can insert the pre-defined snippets supplied with
SQL Server, or create your own.
more info here
In SQL Server Management Studio you could use menu -> Tools -> External Tools.
Do you use SQL Prompt? If so, this has a Tab History feature that lets you search for query windows, regardless of whether they have been saved or not.
If not, I'd use the snipppet feature that VV5198722 has referred to, or the Template Browser (invoked from the view menu).
I want Roundhouse to create the db on a second hd (not the C drive which it does right now). Is there any way in Roundhouse to do this ? Iam using sql server 2008 r2 express. I tried the -cds option for custom db creation but it gives an error saying does not provide facility at this time.
thank you
First you must know how you would create it in T-SQL outside of RoundhousE. If you verify that it works with SQL Express, then you can provide the script to rh as text or (with the newest in trunk and soon to be released 0.8.5) you can point it to a script file that it will read.
If you are using an older version of RH, you MUST give the actual text to /cds and not a path to a file. That text cannot contain the word GO in it at all either. Both of these things will be fixed when 0.8.5 is released.
HTH
I'm trying to code a mssql job that does something using the files in a specific directory. But I don't know the name of the file / files, they will vary in time.
I've found xp_cmdshell command, but I can not use it because of security reasons
Is there any other way to check directory if it contains txt files or not (and if yes get the names of them) in tsql.
Thanks in advance,
Without access to the xp_ stored procedures, no. The other way would be to create a COM object using sp_OACreate that creates a COM Scripting.FileSystemObject, but again access to this may well be restricted as it's a security issue.
As your describing this as an MSSQL job, I'm assuming this is going to be a scheduled task of some description? If so, your best option is probably going to be creating a standard Windows batch file (.BAT) that's scheduled in SQL Server agent that does the existential checking and passes whatever files are found in to your SQL script via sqlcmd/osql.