How do I retrieve a more detailed error message?
I ran an individual task to upload a file:
And the response I got was:
Where do I go to get more details on what actually failed?
Please note that I am able to login and perform this operation manually through an FTP client like filezilla.
Have you considered using EventHandlers for OnError events and logging to a table?
http://consultingblogs.emc.com/jamiethomson/archive/2005/06/11/SSIS_3A00_-Custom-Logging-Using-Event-Handlers.aspx
EMC (the company that hired the blogger who wrote the original blog post) took down his migrated content. So here is the above post on waybackmachine -- SSIS: Custom Logging Using Event Handlers
I've used this exact method on a few SSIS packages, and it's pretty easy to setup. The OnPostExecute is really excessive, but great for demonstration purposes.
NB: I'm not sure it will give you detailed FTP error codes if it's not showing them in debug mode.
For FTP troubleshooting -- if you're using any variable/dynamic connection parameters that are changed at run time, you might want to pop them up in a message box, or save them to a file as they are set so you can review them later.
Related
We have some to critical jobs in our production system. We receiving email about of job status succeded or failed and also we want to set send text message from SQL Server. I searched but looks not so possible, has anybody tried before ?
I have done this a long time ago using a third-party tool that offers exactly what you need. You can find out more about it here.
And here you can find detailed information on how to send and receive messages from SQL Server, using this tool.
In the website it is stated that you can only use this for free for the first 30 days.
Basically you will need to host their small database on your SQL Server and then insert a row into a Messages table with the message you want to send along with the phone number to which it has to be delivered.
"The SMS Messaging Server service application will pickup any newly added messages and send them out through the appropriate channel automatically." as stated in the documentation.
However, because it is a third party tool and not open source, I don't know if it will be suitable for you. I am not aware of any free tools for the same purpose.
I have found this link where it is explained how to turn emails into text messages, it has an inconvenient, however: you have to know the cellular provider for each of the phones you want to send message.
https://www.mssqltips.com/sqlservertip/2922/sql-server-alerts-with-text-messaging-from-sql-server-database-mail/
Best regards,
Given help from this microsoft link, I am aware of many tools related to SSIS diagnostics:
Event Handlers (in particular, "OnError")
Error Outputs
Operations Reports
SSISDB Views
Logging
Debug Dump Files
I just want to know what is the basic, "go to" approach for (non-production) diagnostics setup with SSIS. I am a developer who WILL have access to the QA and UAT servers where I will be performing diagnostics.
In my first attempt to find the source of an error, I used SSMS to view operational reports. All I saw was this:
I followed the instructions shown above, but all it did was lead me in a circle. The overview allows me to see the details, but the details show the above message and ask me to go back to the overview. In short, there is zero error information beyond telling me which task failed within the SSIS package.
I simply want to get to a point where I can at last see SOMETHING about the error(s).
If the answer is that I first need to configure an OnError event in my package, then my next question is: what would the basic, "go to" designer-flow look like for that OnError event?
FYI, this question is similar to "best way of logging in SSIS"
I also noticed an overall strategy for success with SSIS in this answer. The author says:
Instrument your code - have it make log entries, possibly recording diagnostics such as check totals or counts. Without this, troubleshooting is next to impossible. Also, assertion checking is a good way to think of error handling for this (does row count in a equal row count in b, is A:B relationship really 1:1).
Sounds good! But I'd like to have a more concrete example...particularly for feeding me the basics of what specific errors were generated.
I'm trying to avoid learning ALL the SSIS diagnostic approaches, just for the purpose of picking one good "all around" approach.
Update
Per Nick.McDermaid suggestion, in the SSISDB DB I run this:
SELECT * FROM [SSISDB].[catalog].[executions] ORDER BY start_time DESC
This shows to me the packages that I manually executed. The timestamps correctly reflect when I ran them. If anything is unusual(?) it is that the reference_id, reference_type and environment_name columns are null. All the other columns have values.
Update #2
I discovered half the answer I'm looking for. The reason no error information is available, is because by default the SSIS package execution logging level is "none". I had to change the logging level.
Nick.McDermaid gave me the rest of the answering by explaining that I don't need to dive into OnError tooling or SSIS logging provider tooling.
I'm not sure what the issue with your reports are but in answer to the question "Which SSIS diagnostics should I learn", I suggest the vanilla ones out of the box.
In other words use built in SSIS logging (which does not require any additional code) to log failures. Then use the built in reports (once you get them working) to check those logs.
vanilla functionality requires no maintenance. Custom functionality (i.e. filling your packages up with OnError events) requires a lot more maintenance.
You may find situations where you need to learn some of the SSISDB tricks to troubleshoot but in the first instance, try to get everything you can out of the vanilla reports.
If you need to maintain an SQL 2012 or after existing system, then all of this logging is built in. Manual OnError additions are not guaranteed to be built in
The only other thing to be aware of is that script tasks never yield informative errors. I actually suggest you avoid the use of script tasks in SSIS. I feel that if you have to use a script task, you might be using the wrong tool
Adding to the excellent answer of #Nick.McDermaid.
I use SSIS Catalog error reporting. In most cases, it is sufficient and have the following functionality for error analysis. Emphasis is on the following:
Usually the first or second error message contains meaningful information on error. The latter is some error occurred in the dataflow....
If you look at the first/second error message at All Messages report at Error Messages section, you will see Error Context hyperlink. Invoking it will show you environment, connection managers and some variables at the moment of package crash.
Good error analysis is more an approach and practice than a mere tool selection. Here are my recommendations:
SSIS likes to report error code instead of meaningful explanation. So, Integration Services Error and Message Reference is your friend.
SSIS includes in error context (see above) dump those variables which have Include in ErrorDump property set to true.
I build SQL commands for SQL Task or DataFlow Source in variables. This allows to display SQL command executed at error in error context, when you set incude in Dump property on these variables.
Structure your variables well. If some variable is used only at some task - declare it on this task. Otherwise a mess of dumped variables will hurt you more than do any good.
I'm currently working on a web application which generates daily error (and non error) logs.
The current system outputs a log per task to a text file, and outputs critical errors as well as "start" and "finish" type messages to an email account.
The current workflow is as follows: scour the email box for errors, then go and find the .txt file to look at the associated errors and find the cause.
There are around 30 txt files split across about 5 servers.
This system was set up before me, but I'm looking for any advice on how to deal with the situation.
I have control of the script forming the error logs so can do pretty much anything - but I'm lost where to start: I'd considered some kind of web facing dashboard tool, maybe output the files to RSS or something?
Are there any external or internal tools I should be using?
Of course you may use the SQL Server Reporting Services or review this comparison table, there are some packages which may support SQL Server but they may be overwhelming for your task.
It's not really clear what your problem is or what you want to do, but if I understand correctly, your biggest problem is that some messages are logged to a log file but others are sent by email. Therefore, there is no single location that has all error messages in it and that makes analysis and troubleshooting difficult.
The best solution would be to use a logging framework that supports multiple logging destinations (file, DB, email) and severities. That would allow you to specify a configuration like "all errors are logged to a text file and critical ones are also sent by email", so you can ensure that you have everything in one place for general analysis but critical errors are also handled with priority.
You didn't mention what programming language you use, but assuming it's .NET-based then log4net and Enterprise Library are two common frameworks and there are many questions about them here on SO. Googling should give you a good idea of the pros and cons for your situation. If you're using a different language then you can look for the equivalent package: log4j (Java), logging (Python) etc.
I want to be able to copy the file I have which comes in as XML into a new folder location on the server. Essentially I want to hold a back up of the input files in a new folder.
What I have done so far is try to follow what has been said on this forum post - link text
At first I tried the last method which didn't do anything (file renaming while reading). So I tried one of the other options and altered the orchestration and put a Send shape just after the Receive shape. So the same message that comes in is sent out to the logical port. I export the MSI, and I have created a Send Port in the Admin console which has been set to point to my copy location. It copies the file but it continues to create one every second. The Event Viewer also reports warnings saying "The file exists". I have set the Copy Mode of the port to 'overwrite' and 'Create New', both are not working.
I have looked on Google but nothing helps - BTW I support BizTalk but I have no idea how pipelines, ports work. So any help would be appreciated.
thanks for the quick responses.
As David has suggested I want to be able to track the message off the wire before BizTalk does any processing with it.
I have tried to the CodePlex link that Ben supplied and its points to 'Atomic-Scope's BizTalk Message Archiving Pipeline Component' which looks like my client will have to pay for. I have downloaded the trial and will see if I have any luck.
David - I agree that the orchestration should represent the business flow and making a copy of a file isn't part of the business process. I just assumed when I started tinkering around I could do it myself in the orchestration as suggested on the link I posted.
I'd also rather not rely on the BizTalk tracking within the message box database as I suppose the tracked messages will need to be pruned on a regular basis. Is that correct or am I talking nonsense?
However is there a way I can do what Atomic-Scope have done which may be cheaper?
**Hi again, I have figured it out from David's original post as indicated I also created a Send port which just has a "Filter" expression like - BTS.ReceivePortName == ReceivePortName
Thanks all**
As the post you linked to suggests there are several ways of achieving this sort of result.
The first question is: What do you need to track?
It sounds like there are two possible answers to that question in your case, which I'll address seperately.
You need to track the message as received off the wire before BizTalk touches it
This scenario often arises where you need to be able to prove that your BizTalk solution is not the source of any message corruption or degradation being seen in messages.
There are two common approaches to this:
Use a pipeline component such as the one as Ben Runchey suggests
There is another example of a pipeline component for archiving here on codebetter.com. It looks good - just be careful if you use other components, and where you place this component, that you are still following BizTalk streaming model proper practices. BizTalk pipelines are all forwardonly streaming, meaning that your stream is readonly once, and all the work on them the happens in an eventing manner.
This is a good approach, but with the following caveats:
You need to be careful about the streaming employed within the pipeline component
You are not actually tracking the on the wire message - what your pipeline actually sees is the message after it has gone through the BizTalk adapter (e.g. HTTP adapter, File etc...)
Rely upon BizTalk's out of the box tracking
BizTalk automatically persists all messages to the message box database and if you turn on BizTalk tracking you can make BizTalk keep these messages around.
The main downside here is that enabling this tracking will result in some performance degradation on your server - depending on the exact scenario, this may not be a huge hit, but it can be signifigant.
You can track the message after it has gone through the initial receive pipeline
With this approach there are two main options, to use a pure messaging send port subscribing to the receive port, to use an orchestration send port.
I personally do not like the idea of using an orchestration send port. Orchestrations are generally best used to model the business flow needed. Unless this archiving is part of the business flow as understood by standard users, it could simply confuse what does what in your solution.
The approach I tend to use is to create a messaging send port in the BizTalk admin console that subscribes to your receive port. The send port will then just use a standard BizTalk file adapter, with a pass through pipeline.
I think you should look at the Biztalk Message Archiving pipeline component. You can find it on Codeplex (http://www.codeplex.com/btsmsgarchcomp).
You will have to create a new pipeline and deploy it to your biztalk group. Then update your receive pipeline to archive the file to a location that the host this receive location is running under has access to.
I'm trying to fix a broken SSP on a MOSS 2007 site. The problem I am running into manifests itself as follows...
In the SSP "Search Settings" page I get this message:
The search service is currently offline. Visit the Services on Server page in SharePoint Central Administration to verify whether the service is enabled. This might also be because an indexer move is in progress.
In the SSP "User Profiles and Properties" page I get this in red at the top:
An error has occurred while accessing the SQL Server database or the Office SharePoint Server Search service. If this is the first time you have seen this message, try again later. If this problem persists, contact your administrator.
I have contacted my administrator, but that is currently me and it turns out I don't know any more than I do about the problem.
In the Event Log I get the following message:
The Execute method of job definition Microsoft.Office.Server.Search.Administration.IndexingScheduleJobDefinition (ID 8714973c-0514-4e1a-be01-e1fe8bc01a18) threw an exception. More information is included below.
Search application '{0}' is not ready.
The Event ID is 6398, which isn't as useful as I had hoped, but I don find the message interesting in that it looks like a String.format call where the substituted value is missing. Unfortunately no interesting in that it tells me how to fix the problem.
Sharepoint's own log offers this:
UserProfileConfigManager.GetImportStatus() failed to obtain crawl status: System.InvalidOperationException: Search application '{0}' is not ready.
at Microsoft.Office.Server.Search.Administration.SearchApi..ctor(WellKnownSearchCatalogs catalog, SearchSharedApplication application)
at Microsoft.Office.Server.Search.Administration.SearchSharedApplication.get_SearchApi()
at Microsoft.Office.Server.UserProfiles.UserProfileConfigManager.c__DisplayClass3.b__0()
at Microsoft.Office.Server.Diagnostics.FirstChanceHandler.ExceptionFilter(Boolean fRethrowException, TryBlock tryBlock, FilterBlock filter, CatchBlock catchBlock, FinallyBlock finallyBlock)
I have tried stopping and starting the search service, removing and re-adding it from the administration panel, and pretty much every other thing I could find to do with Sharepoint's own administrative tools, which leads me to believe the problem here may be database or permissions related.
There was a second SSP set up on the same server, which I think may have been part of the original cause of the problem, but removing it has made no difference.
Maybe you can make sense of this - I'm new to sharepoint, so it makes little sense to me:
"Service Shared, after looking for the solution much encontre this forum where a person tapeworm the same problem. After reading a infinity of commentaries, which I made to solve the problem was to create a new shared service, later it assigns the other applications to him and later I put it like predetermined, it initiates the import of profiles, and later the hearings, clearly first I did it in a site of tests just in case something happened, later eliminates the First Shared Service and finally the error I am solved. The snapshot of the Registry of the configuration of the application in the data base has been stored correctly. Context: application `SharedServices2 ′"
You didn't mention anything about tapeworms, so maybe you're running a newer version.
Translation of:
http://tecnologiainformaticait.wordpress.com/2008/11/21/error-sharepoint-search-application-0-is-not-ready/
Personally, I'd try the msdn forums.
So it seems that the problem was a corrupted Shared Service Provider ( no idea how it came about, but there you go ) and the only working solution I could find was to delete it and start again.
I suspect there may have been a more elegant fix by changing something in the database somewhere, but I don't know the Sharepoint Database model well enough to find it in the time available.
As an additional warning to this, if you do delete your SSP you may find that it doesn't delete cleanly so that you get a bunch of SQL server tasks that still try to run on an empty database, which can cause problems if you have anything else running on the same database server.
Same problem. My DBA delete correctly the search database and it still doesn't work.
I'll post the solution on my blog when I found something.
For the moment, we open a MS call.
Created a new SSP
2- In central admin, click on shared Services Administration
3- Click on "Change Associations" and move all the web apps to the new SSP
Choose a new search_DB and select the good server that will index if you are in a farm
Problems created by this operation:
We notice that we lose statistics information for our sites.
if you tried this solution, give us your feed back too
Thanks.
http://dejacquelot.blogspot.com/