I am currently working on a project creating a relational database that in additional to standard data also requires the ability to have attachments that relate to each record.
The database itself will be based in Microsoft SQL Server, with Microsoft Access (2016) serving as the front end.
After receiving some advice from a previous question, I believe utilizing a FileTable would be best for this.
The idea would be that the automatically generated ID for each record would automatically insert a corresponding directory to the file table, which could then be accessed by the user to drop or read files from.
I was able to set up the FileTable, and as long as I am using it through SSMS or the directory it works fine.
The idea for this I think should work, but I am having a few issues:
I am pretty sure I need to set the connection between Access and SQL as OLEDB instead of ODBC, but I cannot seem to find a straightforward way to do this in Access. I was able to figure out how to list the provider etc., but not sure where to put it.
Maybe related to the previous point, but I am not able to load the table in a way that allows for it to be edited. Ideally I am thinking it would work by inserting the ID as 'is_directory = 1' to the table whenever a unique ID is made, but Access does not seem to like that the filepath (large binary object) is the key.
These are the errors I am seeing:
When trying to load the filetable itself via ODBC connection:
'Invalid field definition 'path_locator' in definition of index or
relationship.'
When trying to access the file_stream/path_locator in Access: 'A
problem occurred while Microsoft Access was communicating with the
OLE server or ActiveX Control. Close the OLE server and restart it
outside of Microsoft Access. Then try the original operation again in
Microsoft Access.'
Any help on this would be appreciated, if there is anything I can clarify please let me know.
So I ended up doing a little bit of a workaround on this.
Basically, I used the following vba code to make it so Access will automatically make a directory with the name of the ID (unless it already exists), and then open that same directory.
This achieves the same thing I was trying to accomplish without the need to connect to the FileTable directly.
The function creates a new directory at the designated string. The button click command uses that function with the designated parent directory, with the new folder being named after the Unique Identifier of the current record (if it does not exist). It then opens that folder so attachments etc. can be added.
Sub MakeFolder(strPath As String)
If Dir(strPath, vbDirectory) > vbNullString Then
Exit Sub
End If
MkDir strPath
End Sub
Private Sub Command5_Click()
MakeFolder "\\ServerName\MSSQLSERVER\FileStream\DocumentStore\" & Me.ID
Shell "explorer.exe" & " " & "\\ServerName\MSSQLSERVER\FileStream\DocumentStore\" & Me.ID, vbNormalFocus
End Sub
The end result is then present on the SQL FileTable as I originally wanted.
Related
My SSIS projects tend to run queries that require changes as they move between environments, like the table schema might change or a value in the Where clause. I've always either put my SQL into a Project Parameter, which is hard to edit since formatting is lost, or just put it directly into the Execute SQL Task/Data Flow Source then manually edited it between migrations which is also not ideal.
I was wonder though if I added my SQL scripts to files within the project, can these be read back in? Example if I put a query like this:
select id, name from %schema%.tablename
I'd like to read this into a variable then it's easy to use an expression as I do with Project Parameters to replace %schema% with the appropriate value. Then the .sql files within the project can be edited with little effort or even tested through an Execute SQL Task that's disabled/removed before the project goes into the deployment flow. But I've been unable to find how to read in a file using a relative path within the project. Also I'm not even sure these get deployed to the SSIS Server.
Thanks for any insight.
I've added a text file query.sql to an SSIS (SQL 2017) Project in Visual Studio, bit I've found no way to pull the contents of query.sql into a variable.
Native tooling approach
For an Execute SQL Task, there's an option to source your query directly from a file.
Set your SQLSourceType to File Connection and then specify a file connection manager in the FileConnection section.
Do be aware that while this is handy, it's also ripe for someone escalating their permissions. If I had access to the file the SSIS package is looking for, I can add a drop database, create a new user and give them SA rights, etc - anything the account that runs the SSIS package can do, a nefarious person could exploit.
Roll your own approach
If you're adamant about reading the file yourself, add two Variables to your SSIS package and supply values like the following
User::QueryPath -> String -> C:\path\to\file.sql
User::QueryActual -> String -> SELECT 1;
Add a Script Task to the package. Specify as a ReadOnly variable User::QueryPath and specify as a ReadWrite variable User::QueryActual
Within the Main you'd need code like the following
string filePath = this.Dts.Variables["User::QueryPath"].Value.ToString();
this.Dts.Variables["User::QueryActual"].Value = System.IO.File.ReadAllText(filePath);
The meat of the matter is System.IO.File.ReadAllText. Note that this doesn't handle checking whether the file exists, you have permission to access, etc. It's just a barebones read of a file (and also open to the same injection challenges as the above method - just this way you own maintaining it versus the fine engineers at Microsoft)
You can build your query by using both Variable and Parameter.
For example:
Parameter A: dbo
Build your variable A (string type) as : "Select * FROM server.DB." + ParameterA + ".Table"
So if you need to change the schema, just change the parameter A will give you the corresponding query in variable A.
First of all - my VB skills are almost non-existent so any suggestion/ideas should be a little more elaborate if possible. Here is the problem I am facing -
We have linked database tables and queries in Ms Access 2010. Now a copy of the database has been created on a new SQL server. So I want to re-link the tables so that access points to the tables on the new SQL server. Right now, all the tables are corresponding to the database on the old server. Here is what I have tried -
Open XYD_Outreach_Dev.accdb on the network (this is the dev version of access that we use)
Form a new ODBC connection which creates a new_sql_server.dsn file corresponding to the new SQL server (2012)
Go to External Data -> ODBC Database -> Link to the data source by creating......
Link to a table say - tblAbriContacts
This creates a new linked table dbotblAbriContacts
I rename it to tblAbriContacts at which point Access asks me -> table already exists. Do you want to replace it? I click yes and the table gets replaced
I hit the save button, close the application and restart it. The tblAbriContacts is pointing to the old database again
What could be the issue? Please note that I have to do this for all the database, lookup and junction tables.
Do I have to change some connection string which the access points to as soon as it is launched?
If I try the Linked table manager, nothing happens, it says "All selected tables were successfully refreshed". But nothing happens. I guess that's because the tables haven't been moved from the old database. A new copy of everything has been created on the new server.
Please help me out with this. My work is stuck because of this reason.
Can you please check if there is a module or other vba file created and file's on_load event has some code which to assign the connection string for all objects (tables and SPs) and refreshes it every time you restart the application.
If you see the connection string there, you might need to replace it with the new one.
I have had Packages in the past where I was looping through multiple Text files in a folder and loading into sql server tables.
Now I am asked to create a package which will loop through Multiple Excel Files in a folder and load them into sql server table.
I went through the following steps to create this package assuming it shouldn't be much different from what I have in other packages where it loops through multiple Flat file.
Added an Execute Sql Task, Truncating my staging table, A simple Truncate table statement.
Added a Foreach Loop Container. Selected Foreach File Enumerator and created a variable called File_Path with data type string.
Added a Data Flow Task.
Added an Excel Data Source. and configured the Excel Connection manager By selecting any one 'Excel' File in the destination folder. (At this point is configured correctly as it is not showing any red cross or warring messages.)
Then I selected the Excel File Connection Manager and in Properties windows Under Expressions, Selected Connection String property and Used the User Variable #User::File_Path.
At this point the Excel Data source is showing a Red Cross as it needs further configuration.
I have tried a few things Like changing the Data Access Mode from Table name to Table Name or View Name Variable, And passing variable #User::File_Path but it gives me the following error.
Can someone please have a look and advice where I am going wrong and how I can fix this? Any Advice or a pointer in the right direction is much appreciated.
Thank you.
You shouldn't use an expression on the ConnectionString property, but on the ExcelFilePath property.
I'm looking to create a text file, which will contain SQL code to create a database and its tables, and to later on, modify the same database.
The text file will be read via an application the user installs, and when it runs, it should read the text file and create, or modify the database if any changes have been applied.
The SQL text file should of course, be somewhat validated in order to not duplicate tables and such.
I'm not asking for any code, just a specific pathway I should follow in order to make this happen.
Thanks for your input.
I'd do database creation via a SQL script which checks for the existence of tables/views/SPs/etc. before creating them, then I'd execute it in the VB application via ADO.NET. I'd ship it with the application in a subdirectory. It's not a big deal to read text files, or to execute a SQL string via ADO.NET.
I'd have a VERSION table in the database that identifies what DB schema version is installed, and when I shipped upgrade scripts which modified the DB, I would have them update the VERSION table. The first version you ship is 1.0, increment as appropriate thereafter.
All the SQL object creation/detection/versioning logic would be in SQL. That's by far the simplest way to do it on the client, it's the simplest thing to develop and to test before shipping (MS SQL Management Studio is a godsend), it's the simplest thing to diff against the previous version, store in source control, etc.
Incidentally, I would also have my application interact with the database strictly via stored procedures, and I would absolutely never, ever feed SQL any concatenated strings. All parameters going to SQL should be delivered via ADO.NET's SqlParameter mechanism, which is very cool because it makes for clean, readable code, and sanitizes all of your parameters for you. Ever use a DB application that crashed on apostrophes? They didn't sanitize their parameters.
If what you are asking is How do I read a text file and make the results execute in SQL
I would use a StreamReader to read the text file into a string variable.
Once you have read it in, go ahead and open a connection to the database and do a ExecuteNonQuery with the value of the string variable.
I would post in the comments but I can't. I think this may be what you are looking for.
Is it possible to execute a text file from SQL query?
Use MS SQL Server Management Studio to perfect your scripts:
http://technet.microsoft.com/en-us/library/ms174173.aspx
SSMS comes with the server installs and is available for the SQL Express versions. (It isn't needed on the client PCs but it may be useful for debugging.
This will most likely be a low security environment and each user will have full control of the DB.
For there it is pretty straight forward to read the text file and run it against the DB. Just get a connection and send the script:
Dim cmd As New Data.SqlClient.SqlCommand
con.Open()
cmd.CommandText = SQL
cmd.Connection = con
cmd.ExecuteNonQuery()
You might want to use a virtual machine on your development PC as it will allow you to quickly do testing of your scripts and code, and return to baseline state.
I've got a project where I'm attempting to use SQLite via System.Data.SQLite. In my attempts to keep the database under version-control, I went ahead and created a Database Project in my VS2008. Sounds fine, right?
I created my first table create script and tried to run it using right-click->Run on the script and I get this error message:
This operation is not supported for the provider or data source you are using.
Does anyone know if there's an automatic way to use scripts that are part of database project against SQLite databases referenced by the databases, using the provider supplied by the System.Data.SQLite install?
I've tried every variation I can think of in an attempt to get the script to run using the default Run or Run On... commands. Here's the script in it's most verbose and probably incorrect form:
USE Characters
GO
IF EXISTS (SELECT * FROM sysobjects WHERE type = 'U' AND name = 'Skills')
BEGIN
DROP Table Skills
END
GO
CREATE TABLE Skills
(
SkillID INTEGER PRIMARY KEY AUTOINCREMENT,
SkillName TEXT,
Description TEXT
)
GO
Please note, this is my first attempt at using a Database, and also the first time I've ever touched SQLite. In my attempts to get it to run, I've stripped any and everything out except for the CREATE TABLE command.
UPDATE: Ok, so as Robert Harvey points out below, this looks like an SQL Server stored procedure. I went into the Server Explorer and used my connection (from the Database project) to get do what he suggested regarding creating a table. I can generate SQL from to create the table and it comes out like thus:
CREATE TABLE [Skills] (
[SkillID] integer PRIMARY KEY NOT NULL,
[SkillName] text NOT NULL,
[Description] text NOT NULL
);
I can easily copy this and add it to the project (or add it to another project that handles the rest of my data-access), but is there anyway to automate this on build? I suppose, since SQLite is a single-file in this case that I could also keep the built database under version-control as well.
Thoughts? Best practices for this instance?
UPDATE: I'm thinking that, since I plan on using Fluent NHibernate, I may just use it's auto-persistence model to keep my database up-to-snuff and effectively in source control. Thoughts? Pitfalls? I think I'll have to keep initial population inserts in source-control separately, but it should work.
I built my database using an SQLite SQL script and then fed that into the sqlite3.exe console program like this.
c:\sqlite3.exe mydatabase.db < FileContainingSQLiteSQLCommands
John
Well, your script looks like a SQL Server stored procedure. SQLite most likely doesn't support this, because
It doesn't support stored procedures, and
It doesn't understand SQL Server T-SQL
SQL is actually a pseudo-standard. It differs between vendors and sometimes even between different versions of a product within the same vendor.
That said, I don't see any reason why you can't run any (SQLite compatible) SQL statement against the SQLite database by opening up connection and command objects, just like you would with SQL Server.
Since, however, you are new to databases and SQLite, here is how you should start. I assume you already have SQLite installed
Create a new Windows Application in Visual Studio 2008. The database application will be of no use to you.
Open the Server Explorer by pulling down the View menu and selecting Server Explorer.
Create a new connection by right-clicking on the Data Connections node in Server Explorer and clicking on Add New Connection...
Click the Change button
Select the SQLite provider
Give your database a file name.
Click OK.
A new Data Connection should appear in the Server Explorer. You can create your first table by right-clicking on the Tables node and selecting Add New Table.