The package that I am working on moves data from a .DBF file onto a table on SQL server.
I used provider for the source as Microsoft Jet 4.0 OLE DB Provider and set extended properties in the all tab of the connection manager to "dBASE IV".
(which is required for .DBF files - pulled this information while building package)
I have given environments - DEV, QA and PROD and have created project parameters to access respective paths.
For the source, went into properties and gave an expression to pick up filepath for respective servers (DEV, QA and PROD) and ran into below error while running the package.
The connection string format is not valid. It must consist of one or more components of the form X=Y, separated by semicolons. This error occurs when a connection string with zero components is set on database connection manager.
Any help is much appreciated.
Based on this article, the DBF connection string is like the following:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\folder;Extended Properties=dBASE IV;User ID=Admin;Password=;
You have to specify the folder that contains the database not the .DBF file
Additional Information
SSIS Connect to a dBASE or Other DBF File
Importing dBF files to SQL Server using SSIS
Related
On our servers (windows 2016, SQL reporting server 2016, Microsoft Access Database Engine 2016) we run 2 SSIS packages. 1 imports data from an excel file to the database and 1 exports data from the database to an excel file. Both are xlsx files.
We run this exact package on TST, ACC, RES and PRD (same server and access setup). We didn't have any issues until a week ago the packages on PRD just kept on getting stuck in the "beginning validation phase" of the Dataflow Task. The other environments are fine.
We've determined that it is not a problem in the application since a simple read package that we created for this issue, gave the same problem. It doesn't seem to be an access issue either. The account that runs the script is sysadmin in SQL and local admin on the fileserver.
We also tried
• Only using one import flows instead of two in Data Flow task: no change https://social.msdn.microsoft.com/Forums/sqlserver/en-US/781c855f-833e-4578-a43a-1729482bbabd/dtspipeline-validation-phase-is-beginning-but-never-stop?forum=sqlintegrationservices
• Set connection managers for OLE DB sources are all set toDelayValidation to True: no change SSIS pre-evaluation phase taking long
• Set ValidateExternalMetadata is set to false for Excel Sources: no change SSIS pre-evaluation phase taking long
• Reinstall Microsoft Access Database Engine on server: no change
• Tested reading a flat file (txt) which worked without issue.
We're fresh out of ideas so any help would be greatly appreciated.
UPDATE:
When manually trying to run the import/export wizard (and selecting excel file) I get "The operating system is not presently configured to run this application". Investigating this message as well.
If you had no problem reading a text file then that would point me to the excel driver (32 or 64), however I would think if that was the issue you would see a connection error. Do this as test.
Go to the console and open excel on the server. This will let you know if you have some licensing issue or something preventing excel from opening on the server.
Import a small amount of data into SQL server from excel using the import data into a test database (or just make a test table). Be sure to use the same driver you are using in the SSIS package.
Currently I am building a database in SQL Server Management Studio (SSMS). Several data sources are imported in this database, via the import/export wizard. Upon completion of the import, not only am I running the import, I also save an SSIS package. This SSIS package is to schedule a weekly refresh of the data source.
On my Windows Server 2012 R2 the Express edition is installed, therefore I have manually created several batch files that run every week (as scheduled via Task Scheduler). This works fine for most tables, however for some tables I encounter some strange (?) behaviour.
This is as follows: when creating the SSIS package via import/export wizard, and directly running the import, the table shows up correctly in the database. That is, with all column names and the thousands of rows it contains.
The strange thing is that, when executing the SSIS package (via the batch file), the table is empty (column names are correct though). For some tables, I do not encounter this behaviour. For others, this behaviour is encountered all the time.
The batch script is as follows (quite straightforward):
"C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\DTExec.exe" /F "C:\Users\username\ssispackage.dtsx"
The batch file seems to run correctly at all times, as the table 'creation_date' changes when I run the batch file. Moreover, for all the tables that do correctly 'refresh', these same batch files do the job.
Some settings of the SSIS package:
Data source: Oracle OLE DB provider
Destination: SQL Server Native Client / MS OLE DB Provider for SQL Server (tried both)
Data via query (as I am querying several tables from Oracle); query is parsing correctly
Mappings: Create destination table & Drop and re-create destination table
Dropping and re-creating is done, because the data source is rather small, and has some changes weekly/monthly (to some rows).
For most data sources imported (and refreshed weekly) via this method, the tables are correctly showing each week (simply dropping the previous table, and re-creating the source).
I hope someone can explain to me why this issue occurs. If some more information is needed from my side, please ask.
Thanks in advance!
UPDATE:
When looking at the log of the batch file, this is (part) of the output:
Source: ..... "SourceConnectionOLEDB"
Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "OraOLEDB" Hresult: 0x80004005 Description: **"ORA-01005: null password given; logon denied".**
End Error
.....next error.... "SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.
Thus, it seems that the password is not remembered/saved correctly in the SSIS package?
This is strange though, as for most tables it does correctly store the password (as those do refresh correctly).
When setting the properties of the data source, namely Oracle Provider for OLE DB, I select the option "Allow saving password". So it should store the password correctly?
Found the answer after all.. The .dtsx file that is saved (the SSIS package) contains the variables for the connection string, it shows that the Password (Sensitive="1") is there. But in the wizard, I did not select 'Save sensitive data with user key'. When selecting this option, an encryption string was added. Now the SSIS packages run well!
I'm having an issue exporting a large dataset (500k+) to Excel via SSIS, where the output file ends up with 0 rows exported. Before saying that I shouldn't be exporting that many records to Excel, let me state that I know and normally wouldn't. Accounting does not want a CSV and is unwilling to open a CSV in Excel.
Using Visual Studio 2012 SSDT, here are the components involved.
Execute SQL Task -> Creates the empty file with headers
Data Flow Task ->
OLE DB Source -> SQL Query
Excel Destination
While the package is running, you can see records flowing from the source to the destination. The package completes without error, but when you open the file, it's empty. The only thing in there is the header.
If I select the Top 1000 records and export to Excel, it works as intended.
Some things I've tried:
Export to Excel on the network
Export to Excel locally
Export to CSV to Excel on both network and locally
Export to Ole DB Destination using Office Access Database Engine 12.0 with "Excel 12.0" extended properties.
Tried running as different users
All with the same outcome.
Can anyone provide any insight into why this may be happening and how to proceed?
We experienced a similar behaviour, when runnig the ETL in a SQL Server Agent job. Debugging it in Visual Studio, worked, however. So I do not know, whether this solution applies to you.
The reason was that the user, under which the package ran, did not have access to C:\users\Default.
I found this out by using sysinternals process monitor.
I was inspired by that post: Empty Excel File permissions issue: SSIS Excel Destination buffers large record sets through C:\Users\Default
[I explained my search for the bug in my blog: https://www.csopro.de/biblog/2018/04/ssis-fehlerbehebung-bei-excel-destination-schreibt-keine-zeilen/ Unfortunately it is in German]
I have an old SSIS package built under 2008R2. Other devs have tweaked it over the years. When I open it and manipulate it I have an issue where it WILL NOT persist the password in the connection manager. Otherwise I can more or less edit it.
When I save the package and deploy just the dtsx file to a test server, it is using a connection to a different server. I've opened up the raw XML and find no references to that server. The only connection in the package points to the server it is supposed to use.
How do I figure out where it is pulling this phantom connection string from?
Most likely its stored as part of a package configuration which has many possible sources from a SQL or a separate XML config file. Check the command line options for a configuration option where you are executing the package to determine the source. Package configurations override the default values of parameters used in a package, and abstract environment specific values (conn strs, file paths, credentials,...) for easier testing/QA, without hard coding values.
I'm learning ASP.NET MVC 3 with Entity Framework Code First. I'm following a tutorial and I downloaded the corresponding solution for testing on my local machine. Now, something I didn't understand very well is about the automatic creation of the database (if this one didn't exist yet on disk). The very first time I run the application, the database is created for me. That's ok.
Here is the section in Web.config
<add name="BlogContext"
connectionString="Data Source=.\SQL2008;Initial Catalog=CodeFirstMVC.mdf;Integrated Security=SSPI"
providerName="System.Data.SqlClient" />
But I have two problems:
1st. For testing purpose, I deleted the database on disk and run again the solution. I thought that the database would be automatically created but I was wrong: I got the error message below:
{"Unable to open the physical file \"c:\\Program Files\\Microsoft SQL Server\\MSSQL10.SQL2008\\MSSQL\\DATA\\CodeFirstMVC.mdf.mdf\". Operating system error 2: \"2(failed to retrieve text for this error. Reason: 15105)\".\r\nCannot open database \"CodeFirstMVC.mdf\" requested by the login. The login failed.\r\nLogin failed for user 'sa'.\r\nFile activation failure. The physical file name \"c:\\Program Files\\Microsoft SQL Server\\MSSQL10.SQL2008\\MSSQL\\DATA\\CodeFirstMVC.mdf_log.LDF\" may be incorrect."}
I noticed that if I changed the file name in my Web.config then the database is again successfully created. Can you explain me? Why do I have to change the database name to get it running again?
2nd. The database is created in the folder located in C:\Program Files\Microsoft SQL Server\MSSQL10.SQL2008\MSSQL\DATA. I would like to store my database in the App_Data directory. How can I proceed?
Initial catalog is not path to file. It is the name of database. AttachDbFilename is used to specify the file so your connection string should look like:
Data Source=.\SQL2008;Initial Catalog=CodeFirstMVC;AttachDbFilename=|DataDirectory|CodeFirstMVC.mdf;Integrated Security=SSPI
Where |DataDirectory| instructs SQL server to use local application data directory instead of global SQL Server data directory. Local data directory for web application is App_Data.
Edit:
I just noticed that you are probably using full SQL server instead of SQL server express. As I know creating database in App_Data automatically is feature of SQL server express. That also explains first error because SQL server created database called CodeFirstMVC.mdf and stored the database in its global data directory within CodeFirstMVC.mdf.mdf file and transaction log in CodeFirstMVC.mdf.ldf file. It also registered that database internally. By deleting files you didn't remove database from SQL server. You just break its functionality but SQL server still believes that the database exists. That is also reason why you have to change the name to make it work.