Biztalk suspended messages in database - database

I was wondering if someone knows where I can see the data of a suspended message in the biztalk database.
I need this because about 900 messages have been suspended because of a validation and I need to edit all of them, resuming isn't possible.
I know that info of suspended messages are shown in BizTalkMsgBoxDb in the table InstancesSuspended and that the different parts of each message are shown in the table MessageParts. However I can't find the table where the actual data is stored.
Does anyone have any idea where this can be done?

I found a way to do this, there's no screwing up my system when I just want to read them.
How I did it is using the method "CompressionStreams" using Microsoft.Biztalk.Pipeline.dll.
The method to do this:
public static Stream getMsgStrm(Stream stream)
{
Assembly pipelineAssembly = Assembly.LoadFrom(string.Concat(#"<path to dll>", #"\Microsoft.BizTalk.Pipeline.dll"));
Type compressionStreamsType = pipelineAssembly.GetType("Microsoft.BizTalk.Message.Interop.CompressionStreams", true);
return (Stream)compressionStreamsType.InvokeMember("Decompress", BindingFlags.Public | BindingFlags.InvokeMethod | BindingFlags.Static, null, null, new object[] { (object)stream });
}
Then I connect with my database, fill in a dataset and stream out the data to string, code:
String SelectCmdString = "select * from dbo.Parts";
SqlDataAdapter mySqlDataAdapter = new SqlDataAdapter(SelectCmdString, "<your connectionstring">);
DataSet myDataSet = new DataSet();
mySqlDataAdapter.Fill(myDataSet, "BodyParts");
foreach (DataRow row in myDataSet.Tables["BodyParts"].Rows)
{
if (row["imgPart"].GetType() != typeof(DBNull))
{
SqlBinary binData = new SqlBinary((byte[])row["imgPart"]);
MemoryStream stm = new MemoryStream(binData.Value);
Stream aStream = getMsgStrm(stm);
StreamReader aReader = new StreamReader(aStream);
string aMessage = aReader.ReadToEnd();
//filter msg
//write msg
}
}
I then write each string to an appropriate "txt" or "xml" depending on what u want, you can also filter out certain messages with regular expression, etc.
Hope this helps anyone, it sure as hell helped me.
Greetings

Extract Messages from suspended instances
Scenario:
BizTalk 2010 and SQL 2008 R2 is the environment we have used fore this scenario.
You have problem with some integrations, 1500 suspended instances inside BizTalk and you need to send the actual messages to a customer, and then you properly do not want to manually save out this from BizTalk Administrator.
There are a lot of blogs and Internet resources pointing out vbs, powershell scripts how to do this, but I have used BizTalk Terminator to solve this kind of scenarios.
As you now BizTalk terminator is asking you 3 questions when the tool starts
I.1.All BizTalk databases are backed up?
II.2.All Host Instances is stopped?
III.3.All BizTalk SQL Agents is stopped?
This is ok when you are going to actually change something inside BizTalk databases but this is not what you are going to do in this scenario you are only using the tool to read from BizTalk databases. But you should always have backups off BizTalk databases.
You are always responsible for what you are doing, but when we have used this tools in the way I describe we have not have any problem with this scenario.
So after you have start Terminator tool please click yes to the 3 questions(you dont need to stop anything in this scenario) then connect to the correct environment please do this in your test environment first so you feel comfortable with this scenario, the next step is to choose a terminator task choose Count Instances(and save messages) after this you have to fill in the parameter TAB with correct serviceClass and Hostname and set SaveMessages to True and last set FilesaveFullPath to the correct folder you want to save the messages to.
Then you can choose to click on the Execute Button and depending of the size and how many it can take some time, after this disconnect Terminator do NOT do anything else.
You should now if you have filled in the correct values in the parameter TAB have the saved messages inside the FilesaveFullPath folder.
Download BizTalk terminator from this address:
http://www.microsoft.com/en-us/download/details.aspx?id=2846

This is more than likely not supported by Microsoft. Don't risk screwing up your system. If you have a need to have a edit and resubmit, it needs to be built into the orchestration. Otherwise, your best bet is to use WMI to write a script to:
pull out all of the suspended messages
terminate them
edit them
resubmit them

you can find it through the HAT tool you just need to specify the schema ,port and the exact date
with the exact time and it will show you the messages right click on the desired one and save .

Related

Fetching ElasticSearch Results into SQL Server by calling Web Service using SQL CLR

Code Migration due to Performance Issues :-
SQL Server LIKE Condition ( BEFORE )
SQL Server Full Text Search --> CONTAINS ( BEFORE )
Elastic Search ( CURRENTLY )
Achieved So Far :-
We have a web page created in ASP.Net Core which has a Auto Complete Drop Down of 2.5+ Million Companies Indexed in Elastic Search https://www.99corporates.com/
Due to performance issues we have successfully shifted our code from SQL Server Full Text Search to Elastic Search and using NEST v7.2.1 and Elasticsearch.Net v7.2.1 in our .Net Code.
Still looking for a solution :-
If the user does not select a company from the Auto Complete List and simply enters a few characters and clicks on go then a list should be displayed which we had done earlier by using the SQL Server Full Text Search --> CONTAINS
Can we call the ASP.Net Web Service which we have created using SQL CLR and code like SELECT * FROM dbo.Table WHERE Name IN( dbo.SQLWebRequest('') )
[System.Web.Script.Services.ScriptMethod()]
[System.Web.Services.WebMethod]
public static List<string> SearchCompany(string prefixText, int count)
{
}
Any better or alternate option
While that solution (i.e. the SQL-APIConsumer SQLCLR project) "works", it is not scalable. It also requires setting the database to TRUSTWORTHY ON (a security risk), and loads a few assemblies as UNSAFE, such as Json.NET, which is risky if any of them use static variables for caching, expecting each caller to be isolated / have their own App Domain, because SQLCLR is a single, shared App Domain, hence static variables are shared across all callers, and multiple concurrent threads can cause race-conditions (this is not to say that this is something that is definitely happening since I haven't seen the code, but if you haven't either reviewed the code or conducted testing with multiple concurrent threads to ensure that it doesn't pose a problem, then it's definitely a gamble with regards to stability and ensuring predictable, expected behavior).
To a slight degree I am biased given that I do sell a SQLCLR library, SQL#, in which the Full version contains a stored procedure that also does this but a) handles security properly via signatures (it does not enable TRUSTWORTHY), b) allows for handling scalability, c) does not require any UNSAFE assemblies, and d) handles more scenarios (better header handling, etc). It doesn't handle any JSON, it just returns the web service response and you can unpack that using OPENJSON or something else if you prefer. (yes, there is a Free version of SQL#, but it does not contain INET_GetWebPages).
HOWEVER, I don't think SQLCLR is a good fit for this scenario in the first place. In your first two versions of this project (using LIKE and then CONTAINS) it made sense to send the user input directly into the query. But now that you are using a web service to get a list of matching values from that user input, you are no longer confined to that approach. You can, and should, handle the web service / Elastic Search portion of this separately, in the app layer.
Rather than passing the user input into the query, only to have the query pause to get that list of 0 or more matching values, you should do the following:
Before executing any query, get the list of matching values directly in the app layer.
If no matching values are returned, you can skip the database call entirely as you already have your answer, and respond immediately to the user (much faster response time when no matches return)
If there are matches, then execute the search stored procedure, sending that list of matches as-is via Table-Valued Parameter (TVP) which becomes a table variable in the stored procedure. Use that table variable to INNER JOIN against the table rather than doing an IN list since IN lists do not scale well. Also, be sure to send the TVP values to SQL Server using the IEnumerable<SqlDataRecord> method, not the DataTable approach as that merely wastes CPU / time and memory.
For example code on how to accomplish this correctly, please see my answer to Pass Dictionary to Stored Procedure T-SQL
In C#-style pseudo-code, this would be something along the lines of the following:
List<string> = companies;
companies = SearchCompany(PrefixText, Count);
if (companies.Length == 0)
{
Response.Write("Nope");
}
else
{
using(SqlConnection db = new SqlConnection(connectionString))
{
using(SqlCommand batch = db.CreateCommand())
{
batch.CommandType = CommandType.StoredProcedure;
batch.CommandText = "ProcName";
SqlParameter tvp = new SqlParameter("ParamName", SqlDbType.Structured);
tvp.Value = MethodThatYieldReturnsList(companies);
batch.Paramaters.Add(tvp);
db.Open();
using(SqlDataReader results = db.ExecuteReader())
{
if (results.HasRows)
{
// deal with results
Response.Write(results....);
}
}
}
}
}
Done. Got the solution.
Used SQL CLR https://github.com/geral2/SQL-APIConsumer
exec [dbo].[APICaller_POST]
#URL = 'https://www.-----/SearchCompany'
,#JsonBody = '{"searchText":"GOOG","count":10}'
Let me know if there is any other / better options to achieve this.

How to compare the same db at new moments in time

By developing client server applications in Delphi + SQL Server I continuously face the problem to have an exact report of what an action caused on db.
A simple example is:
BEFORE: start "capturing" the DB
user changes a value in an editbox
the user presses a "save" button
AFTER: stop "capturing" the DB
I would like to have a tool that compares the before and after status (I manually capture the BEFORE and AFTER status).
I googled for this kind of tools but all I found are tools for doing data or schema comparison between more datasources.
Thanks.
The following is an extract for an application we have. This code is in a BeforePost event handler that is linked to all of the Dataset components in the application. This linking is done using code as there are a lot of datasets. This doesn't actually log the changes (just lists the fields) but it should be simple enough to change to meet your objective. I don't know if this is exactly right for what you are trying to achieve since you ask for a tool but it would be an effective way of creating a report of all changes
CurrentReport := Format('Table %s modified', [DataSet.Name]);
for i := 0 to DataSet.FieldCount - 1 do
begin
XField := DataSet.Fields[i];
if (XField.FieldKind = fkData) and (XField.Value <> XField.OldValue) then
begin
CurrentReport := CurrentReport + Format(', %s changed', [XField.FieldName])
end;
end;
Note that our code collects a report but only logs it after the post has been successfully completed

How to query Alert System Enable mail profile properties

In order to send automated job failure notifications from SQL Server Agent, you must configure Database Mail and then to to SQL Agent properties > Alert System > Mail Session > Enable mail profile to configure the mail system and mail profile to be used to send the email notifications. We have many servers, and would like to setup a central cross-server job that ensures that the Enable mail profile option is checked across various servers, because otherwise, the scheduled jobs fail silently without sending an email notification.
Is there a supported way to query the msdb database to get to these settings using T-SQL (or by some other way programmatically)?
Running a SQL Profiler trace while bringing up the properties page in the SQL Server Agent UI shows references to msdb.dbo.sp_get_sqlagent_properties which is an undocumented procedure (I would prefer to use documented objects to help future-proof our solution), and several calls to master.dbo.xp_instance_regread which I would imagine the reg keys could change per each SQL Server instance installation.
Does anyone know of a way to query to check whether the enable mail profile option is configured, and also retrieve the mail profile that is designated in the SQL Agent Alert System configs? Most of our servers are SQL Server 2008 R2, with some SQL 2012. I would prefer SQL 2008+ support.
Thanks in advance!
Future-proofing is a very wise idea :). As you noted, both xp_regread and xp_instance_regread are also undocumented. And http://social.msdn.microsoft.com/Forums/sqlserver/en-US/b83dd2c1-afde-4342-835f-c1debd73d9ba/xpregread explains your concern (plus, it offers you an alternative).
Your trace and your run of sp_helptext 'sp_get_sqlagent_properties' are a good start. The next thing to do is run sp_helptext 'sp_helptext', and note its reference to sys.syscomments. BOL sys.syscomments topic redirects you to sys.sql_modules, and that points to the next step. Unfortunately for your needs, just one row (for 'sp_get_sqlagent_properties') will be returned by running USE msdb; SELECT object_name(object_id) FROM sys.sql_modules WHERE definition LIKE '%sp_get_sqlagent_properties%'. I thus assume you are out of luck - there appears to be no alternative, publicly documented, module (sproc). My assumption could be wrong :).
I deduce that xp_reg% calls exist for client (SMO, SSMS, etc.) needs, such as setting/getting agent properties. More importantly (for your needs), your sp_helptext run also reveals SSMS (a client) is using a registry store (i.e. not a SQL store). Unfortunately, I must deduce (based upon an absence of proof from a library search) that those keys (and their values) are also not documented...
The above appears to put you in a pickle. You could decide "if we are going to rely upon undocumented registry keys, we might as well rely on the undocumented calls to read them", but I won't recommend that:). You could also file a feature request at https://connect.microsoft.com/ (your need is clear), but because your need concerns a client-side feature request, I do not recommend holding your breath while waiting for a fix :).
Perhaps it is time to step back and take a look at the bigger picture:
How often can that key be changed, and how often will this process poll for that change?
Email uses a mail primitive. Sender: "Dear recipient, did you get my mail?" Recipient: "Dear sender, did you send me mail?" Disabling an email profile is not the only reason for an email failure.
Would a different approach be more useful, when compared to periodically checking a key?
One approach would be to periodically send "KeepAlive" email. If the "KeepAlive" email isn't periodically received, maybe that key was tweaked, maybe the Post Office is on a holiday, or maybe something else (equally bad) happened. Plus, this approach should be fully supported, documented, and be future-proof. Who knows how and what keys will be used in the next version of SQL Server Agent?
The first bullet isn't addressed (neither would it be addressed by periodically checking a key), and perhaps you have additional needs (worth mentioning on MS connect).
I finally found a way to do this using PowerShell and Microsoft.SqlServer.Management.Smo.Agent.JobServer.
Here is the PowerShell script I wrote to check if SQL Agent mail alerts are enabled, and to make sure that SQL Agent is set to Auto startup when the server reboots. This works works with local or remote SQL instances.
# usage examples
Check-SQLAgentConfiguration -InstanceName 'localhost\sql2014'
Check-SQLAgentConfiguration -InstanceName 'RemoteServerName'
function Check-SQLAgentConfiguration{
param([string]$InstanceName='localhost')
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | out-null;
$smosrv = new-object Microsoft.SqlServer.Management.Smo.Server($InstanceName);
$smosrv.ConnectionContext.ConnectTimeout = 5; #5 seconds timeout.
$smosrv.ConnectionContext.ApplicationName = 'PowerShell Check-SQLAgentConfiguration';
"Server: {0}, Instance: {1}, Version: {2}, Product Level: {3}" -f $smosrv.Name, $smosrv.InstanceName, $smosrv.Version, $smosrv.ProductLevel;
# NOTE: this does not seem to ever fail even if SQL Server is offline.
if(!$smosrv){"SQL Server Connection failed"; return $null;}
$agent = $smosrv.JobServer;
if(!$agent -or $agent -eq $null){
throw "Agent Connection failed";
return -2;
}
$agentConfigErrMsg = "";
if($agent.AgentMailType -ne "DatabaseMail"){ $agentConfigErrMsg += " AgentMailType: " + $agent.AgentMailType + "; "; }
if(!$agent.DatabaseMailProfile){$agentConfigErrMsg += " DatabaseMailProfile: " + $agent.DatabaseMailProfile + "; ";}
if($agent.SqlAgentAutoStart -ne "True"){$agentConfigErrMsg += " SqlAgentAutoStart: " + $agent.SqlAgentAutoStart + " ServiceStartMode: " + $agent.ServiceStartMode + "; "; }
if($agentConfigErrMsg.length -gt 0){
$agentConfigErrMsg = "Invalid SQL Agent config! " + $agentConfigErrMsg;
throw $agentConfigErrMsg;
return -1;
}
<##
#for debugging:
"Valid: "
"AgentMailType:" + $agent.AgentMailType;
"DatabaseMailProfile: " + $agent.DatabaseMailProfile;
"ServiceStartMode: " + $agent.ServiceStartMode;
"SqlAgentAutoStart: " + $agent.SqlAgentAutoStart;
#"SqlAgentMailProfile: " + $agent.SqlAgentMailProfile;
#>
return 0;
}
SQL 2008R2 uses Service-broker like queues for mail processing (http://technet.microsoft.com/en-us/library/ms175887%28v=sql.105%29.aspx). In our environments I check that the corresponding queue exists and is active.
SELECT * FROM msdb.sys.service_queues
WHERE name = N'ExternalMailQueue'
AND is_receive_enabled = 1;
This table is listed online (http://technet.microsoft.com/en-us/library/ms187795%28v=sql.105%29.aspx).
Testing shows that this makes the required transition as we went from new instance -> enabled mail -> mail switched off and back again.

how to connect MS Access database with matlab (transfer data from GUI and save in database )

Hello ppl I am trying to work with databases and I am new to Matlab.
I want to manipulate databeses created in MS Access but I don't know(I hope find a way to enter data from GUI (this GUI created using matlab ) and save in database)
I've designed the user interface in MATLAB, and create a database in MS Access
The problem I do not know how I connect between the database and MATLAB
I find some code to how connect between it.
dbpath = ['C:\Users\Esra\Documents\Esra.accdb'];
conurl = [['jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DSN='';DBQ='] dbpath];
con = database('','','','sun.jdbc.odbc.JdbcOdbcDriver', conurl);
I hope find good code or book about this .
final , i don not know if it is the correct place to my question or not , if not ,please put my question in correct place
You need to run SQL queries on the database; you can do this with database.fetch (and a few other friends).
The example query from the docs:
conn = database('dbtoolboxdemo','','');
setdbprefs('DataReturnFormat','cellarray')
results = fetch(conn, 'select productdescription from producttable')
% Not in the example in the docs: this syntax, which I prefer, is also supported
results = conn.fetch('select productdescription from producttable');
Note that you will also need to know how to write SQL. For that, there are plenty of resources online - you just have to search for them.

How can I get notification when a mirrored SQL Server database has failed over

We have a couple of mirrored SQL Server databases.
My first problem - the key problem - is to get a notification when the db fails over. I don't need to know because, erm, its mirrored and so it (almost) all carries on working automagically but it would useful to be advised and I'm currently getting failovers when I don't think I should be so it want to know when they occur (without too much digging) to see if I can determine why.
I have services running that I could fairly easily use to monitor this - so the alternative question would be "How do I programmatically determine which is the principal and which is the mirror" - preferably in a more intelligent fashion than just attempting to connect each in turn (which would mostly work but...).
Thanks, Murph
Addendum:
One of the answers queries why I don't need to know when it fails over - the answer is that we're developing using ADO.NET and that has automatic failover support, all you have to do is add Failover Partner=MIRRORSERVER (where MIRRORSERVER is the name of your mirror server instance) to your connection string and your code will fail over transparently - you may get some errors depending on what connections are active but in our case very few.
Right,
The two answers and a little thought got me to something approaching an answer.
First a little more clarification:
The app is written in C# (2.0+) and uses ADO.NET to talk to SQL Server 2005.
The mirror setup is two W2k3 servers hosting the Principal and the Mirror plus a third server hosting an express instance as a monitor. The nice thing about this is a failover is all but transparent to the app using the database, it will throw an error for some connections but fundamentally everything will carry on nicely. Yes we're getting the odd false positive but the whole point is to have the system carry on working with the least amount of fuss and mirror does deliver this very nicely.
Further, the issue is not with serious server failure - that's usually a bit more obvious but with a failover for other reasons (c.f. the false positives above) as we do have a couple of things that can't, for various reasons, fail over and in any case so we can see if we can identify the circumstance where we get false positives.
So, given the above, simply checking the status of the boxes is not quite enough and chasing through the event log is probably overly complex - the answer is, as it turns out, fairly simple: sp_helpserver
The first column returned by sp_helpserver is the server name. If you run the request at regular intervals saving the previous server name and doing a comparison each time you'll be able to identify when a change has taken place and then take the appropriate action.
The following is a console app that demonstrates the principal - although it needs some work (e.g. the connection ought to be non-pooled and new each time) but its enough for now (so I'd then accept this as "the" answer"). Parameters are Principal, Mirror, Database
using System;
using System.Data.SqlClient;
namespace FailoverMonitorConcept
{
class Program
{
static void Main(string[] args)
{
string server = args[0];
string failover = args[1];
string database = args[2];
string connStr = string.Format("Integrated Security=SSPI;Persist Security Info=True;Data Source={0};Failover Partner={1};Packet Size=4096;Initial Catalog={2}", server, failover, database);
string sql = "EXEC sp_helpserver";
SqlConnection dc = new SqlConnection(connStr);
SqlCommand cmd = new SqlCommand(sql, dc);
Console.WriteLine("Connection string: " + connStr);
Console.WriteLine("Press any key to test, press q to quit");
string priorServerName = "";
char key = ' ';
while(key.ToString().ToLower() != "q")
{
dc.Open();
try
{
string serverName = cmd.ExecuteScalar() as string;
Console.WriteLine(DateTime.Now.ToLongTimeString() + " - Server name: " + serverName);
if (priorServerName == "")
{
priorServerName = serverName;
}
else if (priorServerName != serverName)
{
Console.WriteLine("***** SERVER CHANGED *****");
Console.WriteLine("New server: " + serverName);
priorServerName = serverName;
}
}
catch (System.Data.SqlClient.SqlException ex)
{
Console.WriteLine("Error: " + ex.ToString());
}
finally
{
dc.Close();
}
key = Console.ReadKey(true).KeyChar;
}
Console.WriteLine("Finis!");
}
}
}
I wouldn't have arrived here without a) asking the question and then b) getting the responses which made me actually think
Murph
If the failover logic is in your application you could write a status screen that shows which box you're connected by writing to a var when the first connection attempt fails.
I think your best bet would be a ping daemon/cron job that checks the status of each box periodically and sends an email if one doesn't respond.
Use something like Host Monitor http://www.ks-soft.net/hostmon.eng/ to monitor the Event Log for messages related to the failover event, which can send you an alert via email/SMS.
I'm curious though how you wouldn't need to know that the failover happened, because don't you have to then update the datasources in your applications to point to the new server that you failed over to? Mirroring takes place on different hosts (the primary and the mirror), unlike clustering which has multiple nodes that appear to be a single device from the outside.
Also, are you using a witness server in order to automatically fail over from the primary to the mirror? This is the only way I know of to make it happen automatically, and in my experience, you get a lot of false-positives where network hiccups can fool the mirror and witness into thinking the primary is down when in fact it is not.

Resources