I'm looking to create a text file, which will contain SQL code to create a database and its tables, and to later on, modify the same database.
The text file will be read via an application the user installs, and when it runs, it should read the text file and create, or modify the database if any changes have been applied.
The SQL text file should of course, be somewhat validated in order to not duplicate tables and such.
I'm not asking for any code, just a specific pathway I should follow in order to make this happen.
Thanks for your input.
I'd do database creation via a SQL script which checks for the existence of tables/views/SPs/etc. before creating them, then I'd execute it in the VB application via ADO.NET. I'd ship it with the application in a subdirectory. It's not a big deal to read text files, or to execute a SQL string via ADO.NET.
I'd have a VERSION table in the database that identifies what DB schema version is installed, and when I shipped upgrade scripts which modified the DB, I would have them update the VERSION table. The first version you ship is 1.0, increment as appropriate thereafter.
All the SQL object creation/detection/versioning logic would be in SQL. That's by far the simplest way to do it on the client, it's the simplest thing to develop and to test before shipping (MS SQL Management Studio is a godsend), it's the simplest thing to diff against the previous version, store in source control, etc.
Incidentally, I would also have my application interact with the database strictly via stored procedures, and I would absolutely never, ever feed SQL any concatenated strings. All parameters going to SQL should be delivered via ADO.NET's SqlParameter mechanism, which is very cool because it makes for clean, readable code, and sanitizes all of your parameters for you. Ever use a DB application that crashed on apostrophes? They didn't sanitize their parameters.
If what you are asking is How do I read a text file and make the results execute in SQL
I would use a StreamReader to read the text file into a string variable.
Once you have read it in, go ahead and open a connection to the database and do a ExecuteNonQuery with the value of the string variable.
I would post in the comments but I can't. I think this may be what you are looking for.
Is it possible to execute a text file from SQL query?
Use MS SQL Server Management Studio to perfect your scripts:
http://technet.microsoft.com/en-us/library/ms174173.aspx
SSMS comes with the server installs and is available for the SQL Express versions. (It isn't needed on the client PCs but it may be useful for debugging.
This will most likely be a low security environment and each user will have full control of the DB.
For there it is pretty straight forward to read the text file and run it against the DB. Just get a connection and send the script:
Dim cmd As New Data.SqlClient.SqlCommand
con.Open()
cmd.CommandText = SQL
cmd.Connection = con
cmd.ExecuteNonQuery()
You might want to use a virtual machine on your development PC as it will allow you to quickly do testing of your scripts and code, and return to baseline state.
Related
My SSIS projects tend to run queries that require changes as they move between environments, like the table schema might change or a value in the Where clause. I've always either put my SQL into a Project Parameter, which is hard to edit since formatting is lost, or just put it directly into the Execute SQL Task/Data Flow Source then manually edited it between migrations which is also not ideal.
I was wonder though if I added my SQL scripts to files within the project, can these be read back in? Example if I put a query like this:
select id, name from %schema%.tablename
I'd like to read this into a variable then it's easy to use an expression as I do with Project Parameters to replace %schema% with the appropriate value. Then the .sql files within the project can be edited with little effort or even tested through an Execute SQL Task that's disabled/removed before the project goes into the deployment flow. But I've been unable to find how to read in a file using a relative path within the project. Also I'm not even sure these get deployed to the SSIS Server.
Thanks for any insight.
I've added a text file query.sql to an SSIS (SQL 2017) Project in Visual Studio, bit I've found no way to pull the contents of query.sql into a variable.
Native tooling approach
For an Execute SQL Task, there's an option to source your query directly from a file.
Set your SQLSourceType to File Connection and then specify a file connection manager in the FileConnection section.
Do be aware that while this is handy, it's also ripe for someone escalating their permissions. If I had access to the file the SSIS package is looking for, I can add a drop database, create a new user and give them SA rights, etc - anything the account that runs the SSIS package can do, a nefarious person could exploit.
Roll your own approach
If you're adamant about reading the file yourself, add two Variables to your SSIS package and supply values like the following
User::QueryPath -> String -> C:\path\to\file.sql
User::QueryActual -> String -> SELECT 1;
Add a Script Task to the package. Specify as a ReadOnly variable User::QueryPath and specify as a ReadWrite variable User::QueryActual
Within the Main you'd need code like the following
string filePath = this.Dts.Variables["User::QueryPath"].Value.ToString();
this.Dts.Variables["User::QueryActual"].Value = System.IO.File.ReadAllText(filePath);
The meat of the matter is System.IO.File.ReadAllText. Note that this doesn't handle checking whether the file exists, you have permission to access, etc. It's just a barebones read of a file (and also open to the same injection challenges as the above method - just this way you own maintaining it versus the fine engineers at Microsoft)
You can build your query by using both Variable and Parameter.
For example:
Parameter A: dbo
Build your variable A (string type) as : "Select * FROM server.DB." + ParameterA + ".Table"
So if you need to change the schema, just change the parameter A will give you the corresponding query in variable A.
I am attempting to use a Packaged Solution for my Access 2010 application that has its backend linked to SQL Server. At the moment, I'm using the .accdb file as the frontend, and I would like to distribute my application to some other Windows computers, but the Packaged Solution does not work. I had the package include Access Runtime, so their version of the frontend is running on Runtime and not full Access. However, once the application makes a request to the backend, the application does nothing, as I am not even prompted for the SQL Password as per usual with the full version. I've read on about including a .dsn file in the package can secure the SQL connection (see here), but going through steps of other tutorials to create .dsn files hasn't led to any results. Would anyone know how to correctly generate the .dsn file or if I've done something else wrong at this point?
(And yes, I understanding using Access 2010 in the year 2019 is almost a joke at this point, but I'm doing this for testing purposes. I plan to completely remake the frontend in Angular in the future.)
One other unrelated note... would it be a better idea to have the frontend hosted as a .html file like through the "Publish to Access Services" process? I did read that Access Services was discontinued last year, so would that not be possible?
Edit: This is not a duplicate of "DSN Less Connection (MS Access to SQL2016)" because A) I want to utilize a DSN Connection, not DSN-less and B) I am not using connection strings in my code to hook up with SQL.
You should be able to just create FILE dsn, link your tables, and then distribute the compiled accDE to each desktop.
However, what SQL odbc source provider did you use? If you use the SQL server ODBC provider, then that is by default installed on each computer.
However, if you linked using Native 11 (or later), then that driver is NOT installed on each workstation by default. So, I HIGH recommend you create a FILE dsn (not a user or system DSN), and link the table using that. (Access will create DSN-less links for you)
And you should NOT be seeing a logon prompt with your application. This suggests you forgot or missed the save password option.
So, I would re-link your tables, creating a new FILE DSN. And if you using the linked table manager, then make sure you check the prompt for new location to force creating of a NEW DSN. If you just re-fresh, then you DO NOT get a chance to click on the save password option during the linking process.
So, what odbc driver are you using? The native 11 or later are better, but they are not installed by default on each workstation. However, CAUTION is required here, since the older sql driver does NOT support the newer datetime2 formats. If you used these newer sql column types, they will be returned as string data types in Access and create a mess of issues.
So, first, I would re-link using a FILE dsn.
Make sure you check the save password during the re-link.
You then compile your accDB into an accDE, and then distribute that. You don’t really need to use the package wizard, since once each workstation has the runtime installed, then a simple copy of the accDE to each person’s computer will thus work fine. There is NO special connection between your accDE and the package wizard. Once the runtime is installed, then any and all mdb, accDB, and your accDE can simply be clicked on to launch + run. So for testing, you can skip the package wizard, and just copy the accDE to the target machine, click on it, and see if it works.
Edit
The prompt and check box during this process is this:
So you have to check that box to save the password. Note that you ONLY get this dialog WHEN you create a new FILE dsn.
I wish to write a query that inserts a file that resides on the client (C# web server) into a column in the database server (SQL Server), something like INSERT … SELECT * FROM OPENROWSET(BULK…), but without having to save the file on the server machine first.
Is this even possible in SQL?
Although your context is unstated, I'm assuming that you're intending to run this from SSMS rather than from OSQL, a PowerShell script, or through some other means.
The file doesn't need to reside on the physical box running SQL Server, but SQL Server does need access to it. The typical approach, I believe, would be for an application server to copy the file to a shared repository and then pass it off to SQL Server through a UNC reference. The syntax to do so is relatively trivial and can be found in Importing Bulk Data by Using BULK INSERT or OPENROWSET(BULK...).
If instead you're interested in providing a mechanism for the SQL Server to save a file from some type of stream operation where the client is directly transmitting a file and there is no shared repository, I'm not aware of a way to do that. Even if you use an SQL FILESTREAM object you still need an accessible NTFS location to stream from. See Saving and Retrieving File Using FileStream SQL Server 2008.
At some point, the server will have to have a hand on the file. That does not mean that the server has to keep the file, but the file has to get to the server in order to be read and inserted into the db. Typically, this is achieved with a form and a file-type input. On the server, you can use the uploaded file to create your query, then delete it.
That said, storing files in a database is a debatable practice. Depending on the type and size of files you're storing, your database can quickly balloon in size. For starters, this makes backups slower and more prone to failure, along with a laundry list of other potential pitfalls. Check out this question on SO: Storing Images in DB - Yea or Nay? As you can see from the answers, there are a number of considerations to be made, but a good rule of thumb is to not do this unless you have compelling reasons to do so.
In SQL Server BLOB Data in .NET: Tutorial, Mohammad Elsheimy explains how it can be done:
using(SqlConnection con = new SqlConnection(conStr) )
using(SqlCommand command = new SqlCommand("INSERT INTO MyFiles VALUES (#Filename, #Data)", con) )
{
command.Parameters.AddWithValue("#Filename", Path.GetFileName(filename));
command.Parameters.AddWithValue("#Data", File.ReadAllBytes(filename));
connection.Open();
}
Basically, this way the file is read on the client and sent to the database server without a need for a temporary file on the server machine.
I've got a project where I'm attempting to use SQLite via System.Data.SQLite. In my attempts to keep the database under version-control, I went ahead and created a Database Project in my VS2008. Sounds fine, right?
I created my first table create script and tried to run it using right-click->Run on the script and I get this error message:
This operation is not supported for the provider or data source you are using.
Does anyone know if there's an automatic way to use scripts that are part of database project against SQLite databases referenced by the databases, using the provider supplied by the System.Data.SQLite install?
I've tried every variation I can think of in an attempt to get the script to run using the default Run or Run On... commands. Here's the script in it's most verbose and probably incorrect form:
USE Characters
GO
IF EXISTS (SELECT * FROM sysobjects WHERE type = 'U' AND name = 'Skills')
BEGIN
DROP Table Skills
END
GO
CREATE TABLE Skills
(
SkillID INTEGER PRIMARY KEY AUTOINCREMENT,
SkillName TEXT,
Description TEXT
)
GO
Please note, this is my first attempt at using a Database, and also the first time I've ever touched SQLite. In my attempts to get it to run, I've stripped any and everything out except for the CREATE TABLE command.
UPDATE: Ok, so as Robert Harvey points out below, this looks like an SQL Server stored procedure. I went into the Server Explorer and used my connection (from the Database project) to get do what he suggested regarding creating a table. I can generate SQL from to create the table and it comes out like thus:
CREATE TABLE [Skills] (
[SkillID] integer PRIMARY KEY NOT NULL,
[SkillName] text NOT NULL,
[Description] text NOT NULL
);
I can easily copy this and add it to the project (or add it to another project that handles the rest of my data-access), but is there anyway to automate this on build? I suppose, since SQLite is a single-file in this case that I could also keep the built database under version-control as well.
Thoughts? Best practices for this instance?
UPDATE: I'm thinking that, since I plan on using Fluent NHibernate, I may just use it's auto-persistence model to keep my database up-to-snuff and effectively in source control. Thoughts? Pitfalls? I think I'll have to keep initial population inserts in source-control separately, but it should work.
I built my database using an SQLite SQL script and then fed that into the sqlite3.exe console program like this.
c:\sqlite3.exe mydatabase.db < FileContainingSQLiteSQLCommands
John
Well, your script looks like a SQL Server stored procedure. SQLite most likely doesn't support this, because
It doesn't support stored procedures, and
It doesn't understand SQL Server T-SQL
SQL is actually a pseudo-standard. It differs between vendors and sometimes even between different versions of a product within the same vendor.
That said, I don't see any reason why you can't run any (SQLite compatible) SQL statement against the SQLite database by opening up connection and command objects, just like you would with SQL Server.
Since, however, you are new to databases and SQLite, here is how you should start. I assume you already have SQLite installed
Create a new Windows Application in Visual Studio 2008. The database application will be of no use to you.
Open the Server Explorer by pulling down the View menu and selecting Server Explorer.
Create a new connection by right-clicking on the Data Connections node in Server Explorer and clicking on Add New Connection...
Click the Change button
Select the SQLite provider
Give your database a file name.
Click OK.
A new Data Connection should appear in the Server Explorer. You can create your first table by right-clicking on the Tables node and selecting Add New Table.
I've got a web application that uses Microsoft Sql Management Objects (SMO) dll's. I'm wondering how I go about redistributing the libraries for a remote machine.
As I understand it, these come with SQL server or Sql express - which isn't on the remote (shared) webserver. Asking the host to install them, is probably out of the question, so is it possible to dynamically load them?
See below error-
Could not load file or assembly 'Microsoft.SqlServer.Smo, Version=9.0.242.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified.
Alternatively, if anyone can provide a workaround for the below snippet that would also be useful. The script variable is a SQL install script which has been read to the end. The nice thing about this is the outputting of each of the execute strings from SQL. I could of course just execute it all in one chunk, but that provides no visual feedback to the user line by line, that the sql is executing ok.
Is there a stored procedure that could perform this sort of thing? Or an alternative way to execute an install script without full permissions.
Dim connection As New SqlConnection(Me.ConnectionString)
connection.Open()
connection.Close()
Dim server As New Server(New ServerConnection(connection))
server.ConnectionContext.SqlExecutionModes = SqlExecutionModes.ExecuteAndCaptureSql
Dim commands As StringCollection = server.ConnectionContext.CapturedSql.Text
server.ConnectionContext.ExecuteNonQuery(script, ExecutionTypes.ContinueOnError)
Dim s As String
For Each s In server.ConnectionContext.CapturedSql.Text
AppendMessages(s)
Next
http://www.mattberther.com/2005/04/11/executing-a-sql-script-using-adonet/
from
http://coolthingoftheday.blogspot.com/2006/11/execute-t-sql-scripts-with-gos.html
Was what I was looking for, and did the trick.
you can try using sqlcmd utilty that comes with sql server
No, you can't dynamically load them, and no, you can't install them without permissions. Sorry about that, but no dice there.