i would like to ask maybe basic question. My SSIS uses SQL query, result is exported to *.csv file. Everything works fine for months (sql select, sql records/results are in csv export) - but yesterday i have issue: SSIS ran without error, but in *.csv export one column was without any records(column header is in csv file, but records not) - in another columns there were records (records are from same table as missing column:(). when i ran SSIS manually again (cca 4 min after), data were in column. SSIS is deployed and run on same sever as database. my question is - if there is no error in SSIS Execution report, data flow task ran correctly, now query returns all data withou editing .... is there any chance to find what caused data loss? maybe on server log?
thank you
Related
I have created an SSIS package for my database running on SQL Server 2014.
This SSIS package extracts data from an Excel workbook (residing in a shared folder on my Windows Server) and feeds a table in my database. I have tested the package (through a SQL Job) and it works fine.
The logic behind the creation of this package is to pull data on a daily basis from that Excel workbook (the latter being updated on a daily basis). It does have a date column; so, data updated are tagged with the relevant date.
Since this SQL job will be executed on a daily basis, the table will be filled with duplicates (that is, data from Day 1 will be repeated on Day 2 and so on).
How can I deal this issue? I was thinking on adding a T-SQL flow to my SSIS package to delete the contents of the Table before it pulls the data from the Excel Workbook. I guess this would work but I am on the look out for a more elegant solution.
Any help would be appreciated.
I have a table in a db in SQL Server that reads data from a csv file that's being uploaded to a ftp each night. The table shows data for the past 30 days, but suddenly it stopped showing entries past a certain date.
I've checked the ftp dump and the csv file - everything here looks fine here (nothing's changed). The table itself is created using SSIS, and I've found various script for extract, load and transform. However, I'm unsure of how and where to start troubleshooting.
I realize that this is somewhat a broad question, so I'm looking for a way or narrowing down the problem?
Every time I had an SSIS issue like this I would open the package and run it manually from within SSIS because I've found the logging at that level better than when it runs as a SQL job.
For large and complicated packages I would have to select parts of and run piece by piece
I am developing on Visual studio 2015 (with data tools extensions) a simple SSIS package: it has a connection (Sql server 2008), an OLEDB Origin, containing a query, connected to a flat file destination.
The db connection is tested an it works;
The oleDb origin works, there are no variables in the query;
The query inside the OleDb origin returns about 700 records
The preview in the OleDb returns 200 records (as configured)
The mapping between query and file is 1:1, all the fields from query are included in the resulting csv in the same order.
When I run the package the csv file is created, it contains exactly the needed fields but only 169 records (expected about 700).
I compared the contents of the csv with the data from the query and noticed that:
Records are not in the same order
The records not exported are seemingly random placed in the sql results (I supposed that the exported were the first ones but it is not the case)
The data in the missing records has nothing strange (no long string or null values, or out of range values)
Repeating the execution several times the extracted and missing records are exactly the same, in the same order.
Do you have any suggestion to solve this? I need to export all records in the same order. It seems that the Order by clause in the query is ignored by the export and that records exported are a sample of the original ones.
I discovered the issue, my fault.
I checked and saved the data from sql management studio, then developed the dtsx.
Meanwhile that data (that I believed to be unmodifiable) has been modified, therefore the dtsx showed the updated version and I checked it against the old version.
I have an Excel file linked to MS SQL Server through ADO connections. Within this file I can update data in the database. IE each row in Excel is a row in SQL Server.
Everything works great, except for some reason certain column headers in my SQL Server get written into every row of data and I'm not sure why. For example, my column header is Construction Date. What I see in every row of that column is construction date.
This problem seems to only occur from one of the Excel files I have. I have multiple Excel files accessing the same table in SQL Server but the only fields that seem to have this problem are the ones in that specific Excel file. The code is the same across all Excel files, the only difference is the users. That leaves me to assume the problem is because Excel crashes.
Anyone know why this occurs?
There is no select statement. The excel data is just a linked table to SQL server. The update statement is based on primary keys and is hard coded. The data that has changed is stored on a seperate tab and the update runs when you click a button.
I have a strange problem when extracting rows from our MS CRM. Let me explain the flow:
I have a SSIS package that extracts data from CRM and loads it to another database. From this database the data is transformed and loaded to a data warehouse.
If this SSIS package runs at the same time as CRM is inserting/updating many rows i get a strange result: The data that is extracted (using a normal OLD DB Source) in SSIS contains duplicate rows (the ModifiedOn date is different though).
If I run the exact same query tha extracts the data from CRM manually in SSMS i get no duplicates.
So.... it seems like the connection from SSIS somehow makes reads of rows that are updated - both before and after the update (dirty reads of some kind).
Anybody experienced this before?
Thx a lot
/Nicolaj