We have an old legacy SQL 2000 server (the last in the farm) – We’ve been unable to get rid of this server as it uses xp_findnextmsg, xp_readmail etc to monitor a mailbox (via mapi) and import all email to that address into a database. The database contains simple tables that store "from", "to", "subject", "body", "Sent Date" & so on.
As you may know, the procs above are no longer in use in SQL 2005+
This table is read from dozens of internal systems, for instance emails to this mailbox can be automatically picked up by our helpdesk systems & create calls etc.
My question is this: what it the easiest / modern way of doing this in SQL 2008+? Is it going to be a case of writing a .net binary / service that will use smtp or something to connect to a mailbox and insert the data into SQL or is there a simpler way to do it? (SSIS / 3rd party tools / pre-existing code / projects?)
Just thought I’d ask before i start writing something – no point re-inventing the wheel as it were.
PS: The Mailbox in question is an exchange 2010 mailbox.
Edit: This functionality was hinted to be re-introduced in 2008 & dbmail: http://connect.microsoft.com/SQLServer/feedback/details/126167/xp-readmail-replacement-for-sql-2005 but it looks like it failed to materialise!
Edit 2: I've just found a decent code sample here that utilises the new web services in exchange 2007+: http://social.msdn.microsoft.com/forums/en-US/sqltools/thread/dd2b465b-b1d2-4c0d-82ec-c36c6c482d5d - experimenting in progress (has anyone ever worked with SQL and the Exchange web services?)
Edit 3: All done! I knocked up a .net service that sits on our exchange server and monitors a mailbox & pushes any new mail into SQL. Incase others have a similar question and need some sample code to get started - here is some rough code (chopped out of my service - replaced parameterised SQL with basic dynamic SQL for easy reading): (Note: you’ll need the EWS API 1.1 dll)
Imports Microsoft.Exchange.WebServices.Data
Dim ExchangeUrl As String = "https://DOMAIN.co.uk/ews/exchange.asmx"
Dim service As New ExchangeService(ExchangeVersion.Exchange2010_SP1)
service.Url = New Uri(ExchangeUrl)
service.Credentials = New WebCredentials("USER#DOMAIN.CO.UK", "PASSWORD")
Dim findResults As FindItemsResults(Of Item) = service.FindItems(WellKnownFolderName.Inbox, New ItemView(1000))
If findResults.Count > 0 Then
service.LoadPropertiesForItems(findResults.Items, New PropertySet(BasePropertySet.FirstClassProperties))
End If
For Each item As Item In findResults.Items
Dim CurrentEmail As EmailMessage = item
'#### Grab Email Information
E_ID = CurrentEmail.InternetMessageId.ToString()
If CurrentEmail.Sender.Address.ToString() <> "" Then
E_From = Replace(CurrentEmail.Sender.Address, "'", "''")
Else
E_From = Replace(CurrentEmail.Sender.Name, "'", "''")
End If
E_From = Replace(CurrentEmail.Sender.Address, "'", "''")
E_To = Replace(CurrentEmail.DisplayTo, "'", "''")
E_CC = Replace(CurrentEmail.DisplayCc, "'", "''")
E_Subject = Replace(CurrentEmail.Subject, "'", "''")
E_Body = Replace(CurrentEmail.Body.Text, "'", "''")
E_Received = CurrentEmail.DateTimeReceived.ToString("dd/MM/yyyy HH:mm:ss")
E_Sent = CurrentEmail.DateTimeSent.ToString("dd/MM/yyyy HH:mm:ss")
'#### Save the email into SQL
If SqlQuery("INSERT INTO tbl_Emails ([MessageID], [From], [To], [CC], [Subject], [Body], [Received], [Sent]) VALUES ('" & E_ID & "', '" & E_From & "', '" & E_To & "', '" & E_CC & "', '" & E_Subject & "', '" & E_Body & "', CONVERT(DATETIME, '" & E_Received & "', 103), CONVERT(DATETIME, '" & E_Sent & "', 103))") = True Then
item.Delete(DeleteMode.HardDelete)
End If
Next
The first thing that comes to mind for me is SQL CLR. MAPI is not specifically supported in .net (at least to my knowledge), although there are work arounds.
Reading from an exchange mailbox, luckily, is supported in host of situations.
BTW,I've found working with email in .net to be relatively pain free.
Related
I see there are a lot of questions on this issue, but I thought I'd add to it again. I'm trying to run Pass Through queries to put the load on the server instead of wimpy Access. Almost every table I have is stored in a SQL server, but I have a large table I have to loop through and it's much faster if I copy it to a local table and then loop through it. Otherwise, everything is faster or fast enough when going through the pass through functions.
The table in question currently holds about 25k lines and if I just write a query to have access copy the local table to SQL server it takes about 1 hour. However, if I use a pass through query with insert, I can copy it over in about 47 seconds. My problem seems to come when I try to pass too long of a string of text.
Here is my function that run the pass through query
Public Sub RunPassThruQdf(sqlCode As String, Optional isTestDB As Boolean = False)
Dim qdf As QueryDef
Set qdf = CurrentDb.QueryDefs("vbaSQL")
If isTestDB Then
qdf.Connect = "ODBC;DSN=DataWarehouse_Test;Description=DataWarehouse_Test;UID=**username**;PWD=**password**;APP=Microsoft Office;DATABASE=DataWarehouse_test"
'The above line is where the error happens when I debug
Else
qdf.Connect = "ODBC;DSN=DataWarehouse;Description=DataWarehouse;UID=**username**;PWD=**password**;APP=Microsoft Office;DATABASE=DataWarehouse"
End If
qdf.ReturnsRecords = False
qdf.sql = sqlCode
Do Until InStr(qdf.sql, " ") = 0
qdf.sql = Replace(qdf.sql, " ", " ")
Loop
qdf.Execute
qdf.Close
End Sub
You'll notice that it connects with a special login (redacted) because I can't make changes to the server myself, but that user account can. And that last loop removes double spaces until everything is separated by a single space to reduce the string size as much as possible.
And this is the function that copies the local table to the server table
Public Sub LoadUnidentifiedFromLocal()
Dim rst As New RecordsetClass: rst.OpenR "tblUnidentifiedParts_Local"
Dim dtm As Date: dtm = Now
Dim baseSQL As String: baseSQL = "INSERT INTO [DataWarehouse_test].[dbo].[mT_SalesAttributeDB_UnidentifiedParts] (ID, PartNumber, FamilyNumber, AutoNote, ManualNote, Created, Updated) VALUES "
Dim sql As String
RunPassThruQdf "SET IDENTITY_INSERT [DataWarehouse_test].[dbo].[mT_SalesAttributeDB_UnidentifiedParts] ON", True
Dim i As Integer: i = 1
Do Until rst.EOF
If sql = vbNullString Then sql = baseSQL
If i Mod 500 = 0 Then
RunPassThruQdf sql, True
'Debug.Print i & ": " & Format(Now - dtm, "hh:mm:ss") & " (" & Format(Len(sql), "#,##0") & ")"
'DoEvents
sql = baseSQL
End If
Dim addSQL As String: addSQL = "(" & rst.Fields("ID") & ", '" & rst.Fields("PartNumber") & "', '" & rst.Fields("FamilyNumber") & "', '" & rst.Fields("AutoNote") & "', '" & rst.Fields("ManualNote") & "', '" & rst.Fields("Created") & "', '" & rst.Fields("Updated") & "')"
If sql = baseSQL Then
sql = sql & addSQL
Else
sql = sql & ", " & addSQL
End If
rst.MoveNext
i = i + 1
Loop
If sql <> vbNullString Then
RunPassThruQdf sql, True
Debug.Print i & " " & Format(Now - dtm, "hh:mm:ss")
End If
RunPassThruQdf "SET IDENTITY_INSERT [DataWarehouse_test].[dbo].[mT_SalesAttributeDB_UnidentifiedParts] OFF", True
rst.CloseR
End Sub
Where I have that mod 500 is me trying to figure out how many lines I can copy over until I get that System resources exceeded error. You'll probably also notice I have a special rst class I made. But just know that it mimics the normal one, I just have some special functions in there so it was my version of inheritance since VBA doesn't support that.
The first time I got it I googled and found that someone was able to fix their error by change the max lock to 1 million. And that worked for me too, until I exceeded it. DAO.DBEngine.SetOption dbMaxLocksPerFile, 1000000
I know it's not a problem with my computer, it's 6 core Xeon W-10855M with 64GB of ram. But I will see that error even after restart for a while and then it will just stop and work again until I try to send it too much and then I'm stuck seeing it for a bit. What is weird is even restarting my computer will not fix the issue. And I've tried compress and repair and that won't fix it either. So I'm not clear what changes so that it stops reporting that.
But my first question is if increasing the max locks per file helped, is there a way to clear what locks are currently there? My second question would be how big of a string can I send with a pass through, I think I saw somewhere in the neighborhood of 65k before I got the message (That was when I had it set at i mod 500.
I am having issues with my code when I try to insert data into my database, the section where it checks for matching username keeps popping up the error to select another username. I don't know what I am doing wrong, any help will be appreciated.
This is my code:
'Connecting to SQL Database and executing Query
Dim Strconn As String = "Data Source=.\SQLEXPRESS;AttachDbFilename=|DataDirectory|\phermacy.mdf;Integrated Security=True;User Instance=True"
Dim Strcmd As String = "INSERT INTO tblstaff_info(id,fname,lname,sex,nationality,status,dob,age,nationality,state,address,phone,email,dateemp,username,password) VALUES ('" & txtregid.Text & "','" & txtfname.Text & "','" & txtlname.Text & "','" & cmbstaffsex.Text & "','" & cmbstatus.Text & "','" & dtpickerdob.Text & "','" & txtage.Text & "','" & txtnationality.Text & "','" & txtstateofo.Text & "','" & rtbaddress.Text & "','" & txtphone.Text & "','" & txtemail.Text & "','" & dtpdateemp.Text & "','" & txtusername.Text & "','" & txtpassword.Text & "');"
Dim da As New SqlDataAdapter
Dim ds As New DataSet
Dim sqlcmd As SqlCommand
sqlconn = New SqlConnection(Strconn)
Try
sqlconn.Open()
Catch ex As Exception
MsgBox("Could not connect to DataBase. Application will close now!", vbCritical, "Database Error")
End
End Try
sqlcmd = New SqlCommand(Strcmd, sqlconn)
da.SelectCommand = sqlcmd
sqlcmd.Dispose()
sqlconn.Close()
'Exception Handling-----------------------
Dim exc As Exception = Nothing
Try
da.Fill(ds)
Catch ex As Exception
exc = ex
Finally
If Not (exc) Is Nothing Then
MsgBox("User Name Already Exist. Please select a different User Name!", vbExclamation, "Already Exist")
txtusername.Focus()
Else
MsgBox("Save info Successful.", vbInformation, "Successful")
'Me.Close()
'loginform.Show()
End If
End Try
The whole User Instance and AttachDbFileName= approach is flawed - at best! When running your app in Visual Studio, it will be copying around the .mdf file (from your App_Data directory to the output directory - typically .\bin\debug - where you app runs) and most likely, your INSERT works just fine - but you're just looking at the wrong .mdf file in the end!
If you want to stick with this approach, then try putting a breakpoint on the connection's .Close() call - and then inspect the .mdf file with SQL Server Mgmt Studio Express - I'm almost certain your data is there.
The real solution in my opinion would be to
install SQL Server Express (and you've already done that anyway)
install SQL Server Management Studio Express
create your database in SSMS Express, give it a logical name (e.g. Pharmacy - check your spelling!)
connect to it using its logical database name (given when you create it on the server) - and don't mess around with physical database files and user instances. In that case, your connection string would be something like:
Data Source=.\\SQLEXPRESS;Database=Pharmacy;Integrated Security=True
and everything else is exactly the same as before...
Also see Aaron Bertrand's excellent blog post Bad habits to kick: using AttachDbFileName for more background info.
im currently looking for a way to connect to a Microsoft SQL Server Database via VBA (ADODB) with the focus on a minimal risk in harming, block and change the structure of the database. Therefor the access is readonly.
My attemp is the following:
Set DBConn = New ADODB.Connection
Set TmpRecset = New Recordset
DBConn.ConnectionString = pConnStr
DBConn.Open
On Error GoTo TermConnection
With TmpRecset
.ActiveConnection = DBConn
.Source = pQuery
.LockType = adLockReadOnly
.CursorType = adOpenForwardOnly
.CursorLocation = adUseClient
.Open
End With
On Error GoTo TermRecordset
//Doing something useful with TmpRecset
On Error GoTo 0
TermRecordset:
TmpRecset.Close
Set TmpRecset.ActiveConnection = Nothing
TermConnection:
DBConn.Close
Set DBConn = Nothing
End Sub
And I'm using the following connection string:
"Provider=SQLOLEDB;Data Source=IP\Database;Initial Catalog=Databasename;Trusted_connection=yes;"
I used the manual error handling to ensure, that the recordset and the database is closed whatever happens. Via the parameters of the recordset I define the readonly access.
Are there some other mechanisms to make sure, that the integrity of the Database will be ensured?
Best regards
In my opinion there is no reasonable security in Excel. All security should reside on the server. If you want to prevent accidental or malicious changes to the database then the database on the server should be read-only or all users should have read-only access to the SQL server. Furthermore, you can implement traces on the server, SQL audit C2, or make use of extended properties. Yet, all of this is on the side of the SQL server. The things you can do on the "client" side (such as Excel in this case) are only support functions. And so the question is (to me) what kind of support functions can I implement in Excel to ensure SQL server safety. Here are some of the things I do:
(1) Make the connection string dynamic using global variables or storing the string on a hidden sheet. Then you can automatically switch between development server and production server. Example:
Dim conRCServer As ADODB.Connection
Dim rstResult As ADODB.Recordset
Dim strSQL As String
Set conRCServer = New ADODB.Connection
conRCServer.ConnectionString = "PROVIDER=SQLOLEDB; " _
& "DATA SOURCE=" & Ref.Range("C2").Value2 & ";" _
& "INITIAL CATALOG=" & Ref.Range("C4").Value & ";" _
& "Integrated Security=SSPI "
On Error GoTo SQL_ConnectionError
conRCServer.Open
On Error GoTo 0
(2) Have a seperate error handler for connecting to the server and handling SQL syntax errors. Example:
Set rstResult = New ADODB.Recordset
strSQL = "set nocount on; "
strSQL = strSQL & "/* #" & ActiveWorkbook.Path & "/" & ActiveWorkbook.Name & "{" & WorksheetUsers.Name & "}btnDownloadUserDataFromServer */"
strSQL = strSQL & "select v.LastName, "
strSQL = strSQL & " v.FirstName "
strSQL = strSQL & "from vUsers as v "
strSQL = strSQL & "order by v.LastName, v.FirstName "
rstResult.ActiveConnection = conRCServer
On Error GoTo SQL_StatementError
rstResult.Open strSQL
On Error GoTo 0
Here is an error handler for the SQL syntax and in the above example is a seperate handler for the possible SQL connection error.
(3) Incorporate self-identification within the SQL syntax. As you can see in the above example I am also letting the server know which file, which sheet (within the file) and which function within the sheet the user called to execute this statement. If you capture this data on the server with a trace then you can see who is writing their own queries, who is using your standard files and which functions are used (and their respective impact).
(4) If an error occurs you might want to consider writing automated error emails. Example:
SQL_ConnectionError:
Y = MsgBox("Cannot connect to the server. Please make sure that you have a working internet connection. " & _
"Also ensure that are connected to the corporate network and are allowed to access the server. " & _
"Do you want me to prepare an error-email?", 52, "Problems connecting to Server...")
If Y = 6 Then
Set OutApp = CreateObject("Outlook.Application")
Set OutMail = OutApp.CreateItem(0)
With OutMail
.to = Ref.Range("C7").Value2
.CC = Ref.Range("C8").Value2
.Subject = "Problems connecting to database '" & Ref.Range("C4").Value & "' on server '" & Ref.Range("C2").Value & "'"
.HTMLBody = "<span style=""font-size:10px"">---Automatically generated Error-Email---" & _
"</span><br><br>Error report from the file '" & _
"<span style=""color:blue"">" & ActiveWorkbook.Name & _
"</span>' located and saved on '<span style=""color:blue"">" & _
ActiveWorkbook.Path & "</span>'.<br>" & _
"Excel is not able to establish a connection to the server. Technical data to follow." & "<br><br>" & _
"Computer Name: <span style=""color:green;"">" & Environ("COMPUTERNAME") & "</span><br>" & _
"Logged in as: <span style=""color:green;"">" & Environ("USERDOMAIN") & "/" & Environ("USERNAME") & "</span><br>" & _
"Domain Server: <span style=""color:green;"">" & Environ("LOGONSERVER") & "</span><br>" & _
"User DNS Domain: <span style=""color:green;"">" & Environ("USERDNSDOMAIN") & "</span><br>" & _
"Operating System: <span style=""color:green;"">" & Environ("OS") & "</span><br>" & _
"Excel Version: <span style=""color:green;"">" & Application.Version & "</span><br>" & _
"<br><span style=""font-size:10px""><br>" & _
"Possible reasons for this error include: (1) no Internet connection, (2) no working VPN connection to the corporate network, " & _
"(3) the server is currently offline, (4) DNS authentication problems, (5) ... other reasons ..., " & _
"(6) the user does not have the required permission to connect to the underlying database on the server." & _
"<br><br>---Automatically generated Error-Email---"
.Display
End With
Set OutMail = Nothing
Set OutApp = Nothing
End If
Exit Sub
I also looked into your approach of changing the connection parameters. But in most corporate environments I have worked for these connection parameters have been overridden (for example ADODB.Connection.CommandTimeout is overridden by the server's SQL timeout per user or Windows corporate presets if they exist). So, they did not work for me. But the above worked rather well for me and the companies I worked for over the last couple of years.
Let me know if this is the kind of answer you've been looking for.
I have created a macro/some VBA to UPDATE a SQL Server table which works fine.
In short the code pulls a defined amount of records from the the table to excel and then the end user updates some specific information and then clicks update. A connection is created to the table and an SQL update statement runs which updates the relevant records.
The problem is where the user has not had to update a NULL field (NULL is in the SQL Server table but shows as 'empty' in Excel), when the use clicks update the SQL statement is forcing the NULL to an 'empty' entry.
To get round this I would like my code in the For Each statement to check if the cell/record is NULL or Empty and to skip to the NEXT row so the SQL Execute command is not carried out.
Here is the VBA in question:
cnn.Open cnnstr
Dim row As Range
For Each row In [tbl_data].Rows
uSQL = "UPDATE BREACH_DATA SET [VAL_BREACH_REASON] = '" & (row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value) _
& "' ,[VAL_BREACH_DETAIL] = '" & (row.Columns(row.ListObject.ListColumns("VAL_BREACH_DETAIL").Index).Value) _
& "' ,[VAL_VALID] = '" & (row.Columns(row.ListObject.ListColumns("VAL_VALID").Index).Value) _
& "' ,[VAL_NOTES] = '" & (row.Columns(row.ListObject.ListColumns("VAL_NOTES").Index).Value) _
& "' WHERE [ATD_NUMBER] = '" & (row.Columns(row.ListObject.ListColumns("ATD_NUMBER").Index).Value) & "'"
'Debug.Print uSQL
cnn.Execute uSQL
Next
cnn.Close
Set cnn = Nothing
Any suggestions
Kind Regards
Dino
You are updating SQL Server data directly with strings from a cell. This is a classic example of opening a door for injection attacks - users can do all kinds of bad, bad things to your database. But given that you fix that here is a way to check that each cell is not empty or null (I assume if one of the fields are not empty or null you want to update...):
if not
(
(isempty(row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value)
and isnull(row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value)
and do same for the other cell values....
)
then update....
I've searched high and low to resolve this one, but can't seem to fix it on my own. I'm new to classic ASP but a very long time PhP dev.
I'm getting
ADODB.Recordset error '800a0e78'
Operation is not allowed when the object is closed.
/sdd/fx_mlogin.asp, line 54
While logging in to the site i'm working on. The line's that's its referencing is:
Set rs = cmd.Execute
If RS.BOF or RS.EOF then
More relevant code:
Set cn = Server.CreateObject("ADODB.Connection")
cn.Open MM_ap_connect_STRING
Set cmd = Server.CreateObject("ADODB.Command")
Set cmd.ActiveConnection = cn
cmd.CommandText = "sd_member_login " & "'" & guid & "', " & """" & trim(user) & """, " & "'" & trim(pw) & "', " & "'" & uip & "', " & "'" & uagent & "', " & "'" & logintrack & "'"
Set rs = cmd.Execute
If rs.BOF or rs.EOF then
The kicker is that the site worked before and we're moving hosts. The connection is apparently working, but I'm suspicious that its still the issue. My connection string is
MM_ap_connect_STRING = "Provider=SQLOLEDB;Data source=sql2103.shared-servers.com,1087;Initial catalog=database;User Id=username;Password=password;"
But obviously with the username, password, and database fields filled in. I'm also connecting to a SQL 2005 database. Any help would be appreciated! Let me know if I need to provide any more information.
Try adding the T-SQL line:
SET NOCOUNT ON
to the top of the SQL being executed.
I had this exact issue. For me the answer was my stored procedure had an owner of DBO. Which my user for the previous database was set to DBO. On our destination database there was a different DBO, which caused a execute permissions issue. Just changed my db user to DBO and it's working now. Hope that helps.