Just looking at the requirements of a new project and I wanted to make sure this use case was sound:
user fills in InfoPath (2003) form locally on their PC
a button within the InfoPath form titled 'submit' brings up a new outlook (2003) email message with the infopath form attached. User presses sends and email is sent to an exchange mailbox.
sql server preiodically checks this mailbox, downloading any new submissions with the infopath form attached
sql server parses the attachment and the fields within the infopath form.
Is SQL Server capable of parsing mail attachments this way? Any caveats with this approach?
The attraction to using Outlook as the submission technology is that the process for the user is the same if they are offline. Outlook will then automatically sync when they come back online. It is essential that users have some way to fill the forms in offline, 'submit' them, and then have then synced automatically with the server when they next come online.
edit: to clarify, I am not looking for a way to cache form data from the server->client. I am looking to cache the completed form. Building a separate application to cache the completed reports on the client is not an option.
Later versions of SQL Server are capable of running .NET code within them, and as such you might be able to poll a mailbox from SQL Server and process an InfoPath form. However, I'm not sure I'd do it this way.
It might be better to consider writing a Windows Service that does this work. The Windows Service would start up, inspect the mail box of a "service account", read the mails, extract the attachments, process the xml and, eventually, write the data to SQL. It could also, presumably, respond to that mail with a confirmation or errors if business rules or validation errors occurred.
I'm not sure I'd put all of the above logic into SQL - for one thing, I suspect you'd have issues with accounts (having to have the account SQL was running under be able to access the Exchange mailbox account).
Your mileage may vary, and you should prototype this to determine what works best for you, but I'd try and keep the code the uses Exchange as a "work queue" separate from SQL and only put the code that deals with writing data into tables in SQL.
I would not use the approach you outlined above. There are several approaches that appear to me to be more desirable than having SQL Server looking at an Exchange Mailbox. The major point that you make and an important requirement is that the InfoPath form be allowed to work in offline mode. I would think of the "offline mode" and the "data transfer" parts of your project as two distinct and separate pieces: 1) The form and the data should be stored on the client until a connection to the Internet is available and 2) once the connection is available the form and data should be transferred to the server.
You can setup your InfoPath form to submit directly to the SQL Server and bypass the Exchange "middleman" entirely. The setup in InfoPath when you are designing your form is pretty straight forward: 1) you enable "Submit data" for the connection and 2) you configure the submit options. This article has the details about how to do that. Furthermore, your connection to the SQL Server may be setup for offline use, as it's discussed in this article. The only caveat with this approach is that you may need to change your database schema to support it.
Another approach is to have your InfoPath form submit to a SQL Server 2005 HTTP Endpoint. The InfoPath client is just a glorified XML Editor and the HTTP Endpoint is basically a different name for a web service. You receive the form data at the HTTP endpoint into a staging table where the data is stored as XML and then you can do your parsing of that data from that staging area. Still you will have to setup the InfoPath connection for offline use. The major caveat with this approach is that Microsoft will deprecate HTTP Endpoint in SQL Server 2008 in favor of WCF.
And the other approach I would like to suggest is to use WCF itself to receive the XML form data from the InfoPath client. This approach would require you to connect the form's data source to you WCF web service at design time and then also setting up the form for offline use.
I hope that this will be useful to you and at the very least point you in the right direction.
I've seen similar projects that resorted to an Express edition on the client, save the infopath (or app data) in Express and use Service Broker to deliver to center, because of guaranteed delivery semantics of SSB vs. mail. This gives you an all SQL solution easier to sell to IT and you don't need polling on the server. Also you will not have to deal with MIME parsing, is all straight forward XML processing. It is not for the faint of heart though, getting SSB up and running is a challenge.
If you decide to go with mail delivery, an external service will be arguably easier to build, debug and troubleshoot. There are some finer point issues you should have an answer for:
-How will you keep consistent the mail dequeue operations and the table write operations? Your component must engage the Exchange read/delete and the SQL insert into one distributed transaction.
- Is your logic prepared to deal with infopath docs coming out of order? mail transport makes absolutely no guarantee about the order of delivery, so you may see an 'order delete' doc before the 'order create' doc
- How are you going to detect missing documents (not delivered by mail)? Are you going to implement a sender sequence number and end up reinventing TCP on top of mail?
- Does your processing allow for parallel process of correlated documents? If thread 1 picks up doc 1 and thread 2 picks up doc 2 from same sender and doc 2 is correlated with doc 1 (ie. refer to same business transaction), what will happen at the database write? Will it deadlock, will it loose an update, will one be rolled back?
There are many dragons under the bridge ahead...
Related
I am currently been assigned to develop a sync application for my company. We have SQL server on our database server which will be synced with the client database. Client databases are not known, they can be SQLite or MYSQL or whatever.
What this sync app does is, detect changes that occur on server & client databases. Save these changes and sync. If changes occur on server database it will be synced with the client database and vice versa.
I did some research on it and came to know many solutions. One of them is to use a Microsoft Sync Framework. But I hardly found a good implementation example on it for syncing with remote databases.
Then I came across Change Data Capture(CDC) on SQL Server 2008. CDC works by detecting the change on the source table through triggers and put these changes on a separate table called sync_table, this table is then used for syncing.
Since, I cannot use the CDC feature because I don't have sufficient database rights on my machine, I have started to develop my own solution which works like how CDC does. I create separate sync_table for each source table, create triggers to detect data change and put this data in the sync_table.
However, I am advised to do some more research on it for choosing the best implementation methodology.
I need to keep the following things in mind,
Databases may/may not be on the same network.
On server side, the user must be able to select which tables will take part in the sync process.
Devices that will sync with the server database need to be registered first. Meaning that all client devices will be registered by the user before they can start syncing.
As usual any help will be appreciated :)
There is an open source project called SymmetricDS with many of the same goals. Take a look at the documentation and data model to see how the problem was solved, and maybe you will get some ideas. Instead of a separate shadow table for each source table, there is a single sym_data table where all the data is captured in comma separated value format. The advantage is one place to look for captured data and retrieve changes that were part of the same transaction. The table is kept small by purging it often after data is transferred successfully. It uses web protocols (HTTP) for data transfer. The advantage is leveraging existing web servers for performance, administration, and known filtering through firewalls. There is also a registration protocol used before clients are allowed to sync. The server admin "opens registration" for a client ID, which allows the client to connect for the first time. It supports many different databases, so you'll find examples of how to write triggers and retrieve unique transaction IDs on those systems.
My company is planning to turn an older Winforms application into a WPF/Silverlight Client/Server app.
The idea of having a small server app is to have a list of the accessible data bases combined with the user type that may access each of the databases, instead of having to manage databases in each client's admin control. Additionally, it would be great if the SQL request would be handled by the server which would then return the result.
The app is supposed to work on a arbitrary set of databases which will be "registered" with the server and users get a list of databases according to their authentication rights. They can then do practically everything on those databases what one can imagine. The system should be able to handle up to 2 million rows.
The databases are very different, there can be many of them, they can be MS Access, Oracle, SQL Server etc., so no way for me to specify them all before. On top of that, communication with a SQLite cache is needed.
I already have everything I need for the SQL queries from the Winforms app.
I was thinking:
1) A simple WCF server specifying in a config file the available databases per user type.
2) Interface that specifies all necessary SQL queries that can be made to the server.
3) Client...
The idea is:
a client-server application, where the client uses WCF services to execute SQL queries (INSERT, UPDATE, SELECT, etc.) on tables by invoking services methods.
The service should ideally be consumable for both the WPF and the Silverlight app.
Is that the way to go? Which exisiting technologies might I want to make use of regarding formats, communication, services etc.
If that is problematic, I would consider going back to a desktop app, but then how to ease the user type/database access problem for each client?
I would stick with ADO.NET and start with the DbProviderFactory class. This will let you determine the proper database access based on information supplied by the provider using the Factory Design Pattern. So instead of having to create a specialized objects for each database type and database, you can abstract that logic with the DbProviderFactory.
Here's a link that shows some examples: http://msdn.microsoft.com/en-us/library/wda6c36e(v=VS.100).aspx
This question is an updated version of a previous question I have asked on here.
I am new to client-server model with SQL Server as the relational database. I have read that public access to SQL Server is not secure. If direct access to the database is not a good practice, then what kind of layer should be placed between the server and the client? Note that I have a desktop application that will serve as the client and a remote SQL Server database that will provide data to the client. The client will input their username and password in order to see their data. I have heard of terms like VPN, ISA, TMG, Terminal Services, proxy server, and so on. I need a fast and secure n-tier architecture.
P.S. I have heard of web services in front of the database. Can I use WCF to retrieve, update, insert data? Would it be a good approach in terms of security and performance?
A web-service tier is pretty common for smart-clients as a layer between the user-client and the server. This allows:
simple networking (http only)
you have an app-layer in which to put validation etc without upsetting the db
you can have security that isn't tied to the db
the db can run as fewer accounts (app accounts), allowing greater connection pooling
you can "scale out" the app layer
you can cache etc above the db
you can have a richer app layer, with more services than sql server provides
the client has a known API, and never knows about the db (which is an implementation detail)
You can use WCF to talk to the app layer, but you shouldn't think in terms of "INSERT", "UPDATE" etc - you should think in terms of operations that make sense to your domain model - the "CreateOrder" operation, etc. ADO.NET Data Services allows an API more similar to your "INSERT" etc, but it isn't necessarily as controlled as you might like for a secure service.
Performance is really a factor of "what queries am I running?" and "how much data am I transferring?". As long as you keep the operations sane (i.e. don't fetch the entire "Orders" data over the wire just to find the most recent order-date), then you should be OK.
I've got a SQLServer Database and an application (.NET) that configures data on that database. On the other hand, I've got an application (VC++) that reads that data from the database and must be 'informed' a soon as possible of any change in the data saved on the database.
I would like to receive a signal from the database engine when some data had been changed by the first app. Of course, I can code some kind of message that comunicates both apps but my question is if exists that mechanism (some kind of ultra-trigger that warns the other app) in SQLServer or in any API for accessing SQLServer (Native, OLE DB, or even ODBC)
PS: Apologize for my english
If your requirements include guaranteed delivery (e.g. no lost events) - you might want to consider an MSMQ based strategy.
Accessing MSMQ from Microsoft SQL Server
http://www.codeproject.com/KB/database/SqlMSMQ.aspx
Microsoft Support Knowledge Base Article: 555070
Posting Message to MSMQ from SQL Server
http://support.microsoft.com/kb/555070
You can use a SQL Agent Job that will notify the other app starting it with sp_start_job. Check this out.
Is it possible to get notifications using SQL Server Reporting Services? Say for example I have a report that I want by mail if has for example suddenly shows more than 10 rows or if a specific value drop below 100 000. Do I need to tie Notification Services into it and how do I do that?
Please provide as much technical details as possible as I've never used Notification Services before.
Someone also told me that Notifications Services is replaced by new functionality in Reporting Services in Sql Server 2008 - is this the case?
I'd agree with Simon re Notification Services
Also, data driven SSRS Subscriptions are not available unless you use Enterprise Edition (and isn't available if you use SharePoint Integrated Mode).
An alternate way would be to create an Agent job that runs a proc. The proc could check the conditions you require and kick off the subscription if they are met using:
exec ReportServer.dbo.AddEvent #EventType='TimedSubscription', #EventData='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx'
Where the #EventData is the ID of the subscription in dbo.Subscriptions.
This will drop a row in [dbo].[Event]. The Service polls this table a few times a minute to kick off subscriptions.
Really, this isn't far from what happens when you set up a new Subscription, might even be easier to create a Subscription in the Report Server site, find which agent job was created (the ones with GUID names) and edit the T-SQL.
Hope this helps
I wouldn't go down the ntofications services route - it is pretty much a deprecated feature of SQL Server and even if it is around in future it will stagnate. So don't build a dependency on it.
Depending on your needs a data driven SSRS subscription to e-mail you the report would probably work.
http://msdn.microsoft.com/en-us/library/ms159150(SQL.90).aspx
Sending mail using SSRS subscription to your data driven report
A data-driven subscription is composed of multiple parts. The fixed aspects of a data-driven subscription are defined when you create the subscription, and these include the following:
The report for which the subscription is defined (a subscription is always associated with a single report).
The delivery extension used to distribute the report. You can specify report server e-mail delivery, file share delivery, the null delivery provider used for preloading the cache, or a custom delivery extension. You cannot specify multiple delivery extensions within a single subscription.
The subscriber data source. You must specify a connection string to the data source that contains subscriber data when you define the subscription. The subscriber data source cannot be specified dynamically at run time.
The query that you use to select subscriber data must be specified when you define the subscription. You cannot change the query at run time.