I have an XML file and based on it I am looking to insert some information into my database. But first I need to split each node from the XML, to a source destination in SSIS. So far so good.
When I tried to execute the steps to see if I haven't missed something, I noticed that the XML Source step is taking too long, way too long, for at least 20 minutes before asking for help here. And each time I execute the package, a command prompt window pops up and quickly disappears.
I added some print screen below:
Package quick review:
Data flow task review:
Progress review
Does anyone know what might be the issue?
I've had this before with an export column transform. It may be a visual studio issue. Try "Start without Debugging" (Ctrl+F5), if it works fine there then it'll work once deployed.
Related
Two years ago a SQL expert opened a SSMS and showed all queries as they were happening in real time. That way he saw which SQL statements were running fast and which took some time to be ran. I remember the queries to be displayed in a "CMD" look alike window. I can't remember if new queries were displayed at the top or were they shown at the bottom of the window.
For the past month I have been trying to figure out how he got this working. I looked everywhere in monitor, but I can't find anything similar to what he showed me than.
The results were similar to the "claymore eth miner window" ...
Can someone point me in the direction of getting this?
You can use sp_whoisactive to get the current running queries. It is very useful to see if currently there is blocking, locking, long running statements.
In order to get better picture of what was going on in the past, you can enable the Query Store. There are some predefined reports:
and various statistics and it is user friendly:
Today I came across a bug that I'd like to share with everyone.
When trying to execute an SSIS package in Visual Studio (2015 and 2017 the following can happen (note this package was executed sucessfully before).
While on the bottom it clearly says the SSIS package is finished, the data flow task is still in progress (and will never finish). What also happens (don't know if consistently) a CMD window pops open with "SQLDUMPER.EXE".
This is not due to the way the steps are configured, since executing them individually might still lead to the proper results.
Note as well that in my case this problem makes Visual Studio incredibly unstable. It is not uncommon that after or during every execution Visual Studio crashes completely and is automatically shut down by Windows.
There are no error messages and I had no idea what the reason is this happened untill... (answer below)
After a lot of googling i've found a reason whya multitude of people seem to have this problem.
Right click the project and then click on properties
Click "Debugging"
Set "Run64BitRunTime" to "True"
Apparently for most people this fixes the problem.
For me however I had to come up with another solution.
Right click the project and click on properties
Click "General"
Set "TargetServerVersion" to either the target SQL server and run the package again OR set it to any SQL server version, run the package to see if it works, and if it doesn't set it to another version.
These solutions are counter intuitive because this problem seems to arrise at a random moment and the target sql server version might have been the same throughout the entire development.
I have not tried to see what happens when I deploy the bugged project to a server and see if it runs there, so any and all extra information on this problem would be appreciated.
I spent a good bit of time googling this problem. It looks like it is being reported by many people around the world. It is usually down to either x32/x64 compatibility or some miniscule issue with sorting or data sizes or compiling a custom C# script (or combination thereof).
None of these things worked for me so I rebuilt the entire package from scratch and the issue was gone.
It is far from perfect, especially if your package is large and/or complex but if nothing else works, this is your last resort.
"Run64BitRunTime" was already set to "True", but I was getting this error. so I restarted the visual studio and problem resolved.
None of the proposed solutions worked for me. I had to rebuild both the solution and the project a couple of times and this fixed it. Of course, I opened and closed the VS a couple of times in between. I guess this is a bug and you have to tinker with it to get it to work.
I've looked all over and can't find an answer to my question. I'm trying to do the simplest database I can. I want to be able to add/edit/delete items and have it save them to the database, so they're in there the next time I open it.
I'm currently using VS 2013 Express and have the 2012 express still installed. was going through the MSDN tutorials and even tried the Northwind Database, but it said I had to upgrade it, but that it wouldn't allow me to. So now I'm stuck.
Currently, I've created a database, then a new project. I connected the new project to the database and drag-n-dropped one of the tables onto the form in gridview. In the interest of saving space, I won't go into a lot of detail. I basically followed the msdn tutorial on creating two tables called Customers and Orders, with a few variables and 3 entries a piece. There is a primary key in customers and a foreign key in orders.
I used the code MSDN provided, that was similar to what was already in the save buttons auto-generated code, except in a try/catch code. What I have so far is:
`Private Sub CustomersBindingNavigatorSaveItem_Click(sender As Object, e As EventArgs) Handles CustomersBindingNavigatorSaveItem.Click
' Me.Validate()
' Me.CustomersBindingSource.EndEdit()
'Me.TableAdapterManager.UpdateAll(Me.SampleDatabaseDataSet)
Try
Me.Validate()
Me.CustomersBindingSource.EndEdit()
Me.CustomersTableAdapter.Update(Me.SampleDatabaseDataSet.Customers)
MsgBox("Update successful")
Catch ex As Exception
MsgBox("Update failed")
End Try
End Sub`
I apologize for the code not all being in the block, I couldn't get it to work properly. So, from what I can determine, it looks like this code updates the dataset and not the database. Is that true? If so, how do I update the database? When I run the program and add a row, I can click save and it returns a message box, saying it ran successfully. However, I can't rerun the program and have it display, much less shut the VS Express down and bring it back up to have it show the changes. Can anyone please tell me what I'm doing wrong here? Thanks.
Okay, I think I've found the answer to this question. It appears that what's happening is that there are two versions of the database. One version is stored in the root of the project folder, the other is stored in the bin/debug folder.
Apparently, for whatever reason, when you run your program to test your code it pulls from the root database. Any changes you make and save are copied to the one in bin/debug. I assume this is so that you can have stock information in your database and not have it altered by code that may or may not work the way you want. I understand that, but I think there is just as much reason to want it to save changes. I wish there were an option to make it that way and if anyone knows how, please share that.
I've spent a couple of years playing with databases, on and off, thinking I was doing something wrong, when in reality it's just the way VS was handling it. You can do one of two things to test if this is your issue. One is that you can go into the bin/debug and copy the .mdf and the .ldf, and paste them into the root folder, overwriting the two there. I'm not sure what the .ldf does, but it wouldn't run without it. Then when you go to runtime, your changes will be reflected. Another option is build your project and run the .exe in your bin/debug folder. It will run the program and any changes you make will stick, but again, only in the bin/debug version. I assume that a published project will save to the root directory.
I found the article that led to this breakthrough here: http://visualstudiomagazine.com/blogs/tool-tracker/2012/05/dealing-with-local-databases-or-why-your-updates-dont-stick.aspx
Can anyone please tell me how to perform a simple batch process in SSIS?
I know how to do such a thing using T-SQL and/or .NET code but what I wish to do is to use the SSIS GUI to do this entirely. I am not sure if it is possible and all I can find on Google are over complicated solutions.
To explain a bit more - I am reading from a flat file and I want to insert its entire contents into a Sql Server table. Pretty simple huh... But I want to do it 50,000 records at a time. I expect this to be as simple as setting a property somewhere or at the most using some kind of loop tool from the toolbox.
Thanks
You are right. It is a property of the Data Flow task.
Right click on the task, click on the "Properties..." command, look for the DefaultBufferMaxRows property and set the desired value.
References #MSDN.
I had a working SSIS package last week. I came back into the office this morning, opened up the package and tried to open up one of the Data Flow tasks. Double clicking did nothing. I closed the project and reopened it and received an error message that there was a "Catastrophic Error" and that the layout could not be displayed. The Control Flow diagram now only showed my two Sequence Containers.
I made a copy of the dtsx file then went into it manually with a text editor and found that some of the XML was duplicated. I removed the "extra" copy and reopened my package. The empty Sequence Containers are now gone, but I still can't see any of my components. I looked at the Package Explorer and everything is still there, but it's just not displaying in the designer. I was even able to run the package successfully.
Is there any way to rebuild the package so that I can see everything in the designer? Any other suggestions?
Thanks!
I'm not sure why BIDS has decided the layout is junk but you can get around the issue by removing the layout and forcing BIDS to regenerate that information. The graphical layout stuff is a cool idea for conceptualizing how a package is organized but the implementation of storing that information, serialized XML inside XML, is pants.
There are various articles available about how to remove the XML, but this is a more recent example of how to use do this. http://joshrobi.blogspot.com/2012/04/editing-existing-ssis-package-via-ezapi.html
As always, ensure you have a good copy of the package before editing the XML directly.
Do you have a backup of your package - try and restore that to another location first to make sure that is not corrupted.
How are you accessing the package, are you connecting remotely to a server hosting SSIS, or are you using tools located locally on the server?
I've encountered issues when trying to access packages across networks before.
Also, as a word of warning it's not good practice to edit package contents before carrying out further diagnosis!