I have an application using Rebus with SQL Server as a transport layer (yes, I know this is not the ideal transport layer) and I'm, trying to find an out-of-the-box tool to do real-time monitoring of the queues. I tried Rebus Snoop but I found that it is not compatible with SQL Server.
Does anyone know a way to monitor Rebus queues?
Thank you very much.
"Rebus queues" is not really a thing, so you most likely want to turn to whatever broker you are using and use some kind of monitoring tool for that.
E.g. if you're using Azure Service Bus, you can use the Azure Portal's built-in metrics to figure out queue lengths, request rates, etc.
But since it looks you're using SQL Server as the transport, there doesn't really (to my knowledge) exist any kind of compatible "queue monitoring tool" - you'd probably find loads of tools (e.g. from RedGate) that can help with looking at your SQL Server's performance metrics, but it doesn't know that you happen to be using your SQL Server as a message broker.
You can of course also turn to Rebus Pro that comes with Fleet Manager, which can help with some of the things. It's a commercial product though, so it might not be the answer you were hoping for. 😉
Related
My first ever question on stack overflow so please go easy. I have a long running windows application that continually processes sql server commands. I also have a web front end that users use occasionally use to update the same db. I've noticed that sometimes (depending on what the windows application is processing at the time) that if a user submits something to the db I receive out of memory exceptions on the server. I realise I need to dig around a bit more and optimise the code. However I cannot afford the server to go down and expect that in the future i'll be allowing more and more users on the frontend. What i really need is a system that will queue the users requests (they are not time critical) and process them when the db is ready.
I'm using SQL 2012 express.
Is SQL Service Broker the best solution, i've also looked into MSMQ.
If so can someone point me in the right direction for it would be appreciate. In my search i'm just finding a lot of things it does that i don't think i need.
Cheers
It depends on where you're doing the persistence work, and / or calculations. If you're doing the hard work in your Windows Application, then using a Service Broker queue won't be worthwhile, as all you will be doing is receiving a message from the Service Broker queue in your Windows Application, doing your calculations and / or queries from the Windows Application, and then persisting the results to the database: as your database is already under memory pressure, this seems like an unnecessary extra load as you could just as easily queue and retrieve the message from MSMQ (or any other queueing technology).
If however you are doing all the work in the database and your Windows Application just acts as a marshalling service - eg taking the request and palming it off to a stored procedure for actioning - then Service Broker Queues may be worth using: because they are already operating within the context of the database, they can be very efficient at persisting amd querying data.
You would also want to take into failure modes, depending on whether or not you can afford to lose any messages. To ensure message persistence in MSMQ you have to use Transactional Messaging: Service Broker is more efficient at transactional queue processing than MSMQ (because it has transaction support built in, unlike MSMQ which has to use DTC, which adds an overhead) - but if your volume of messages is low, this may not be an issue.
I'm in need of an Enterprise Service Bus/Message Queueing solution for publisher/subscriber functionality. I know MANY exist... MSMQ, MS Series, RabbitMQ, NServiceBus, etc etc etc...
My one requirement is that in a shared hosting solution, the only dependency that I can guarantee will exist is SQL 2005 and later... this leads me directly to SQL Service Broker.
If it sounds like I'm trying to shoehorn ESB functionality into SSB... I suppose I am...
My question is: does anyone know of a .NET API or framework that sits on top of SQL Service Broker and already provides much of the plumbing?
If I were to use pure ADO.net, I could add items to the queues by calling a stored procedure, but then:
Do to the nature of conversations, would I make one conversation per message?
If so, do I lose sequential message processing?
How do I receive messages (I know the receive syntax in t-SQL), do I call a stored procedure repeatedly in a message loop to try to
get a message off the queue?
Or would I WAITFORever? Keeping the connection open and executing the stored procedure forever?
SQL Service Broker doesn't support monologue conversations, but I read they can be implemented...
It's these kind of questions that make me wish there existed a .net solution that already managed all of this.
There was an effort to package a WCF Transport Channel for SQL Server Service Broker but, afaik, is abandonware.
But NServiceBus supports Service Broker as a transport, see Using NServiceBus and ServiceBroker.net and there are github projects like A simple wrapper API for SQL Service Broker and an ITransport plugin for NServiceBus. While not exactly mainstream, some support and community effort does exists.
As an ESB I think you will have problems due to lack of true pub-sub and broadcast. SQL Server 2012 has the ability to SEND a message to multiple targets, see How to Multicast messages with SQL Server Service Broker, but you will still have to implement the pub-sub infrastructure (publishing topics, subscribers etc) from scratch. MySpace did that and was a major effort, see Scale out SQL Server by using Reliable Messaging. My observation reffers to the low level direct use of SSB, I have never used NServiceBus so I cannot tell how well does it abstracts/expose unicast/broadcast/multicast/pub-sub over SSB.
As for your specific questions, I recommend reading Writing Service Broker Procedures and Reusing Conversations.
Recently my clients have asked me if they can use they’re application remotely, disconnected from the local network and the company server.
One solution is to place the database in the cloud, but a connection to the database, and the cloud and an internet connection must be always available.
There not always the case.
So my question is - Is there any database sync system, or a synchronization library so that I can work disconnected with local database and when I connect synchronize the changes I have made and receive changes others have made?
Update:
The application is under Windows (7/xp) ( for now )
It's in Delphi 2007 win32
All client need to have Read/Write access
All Clients have internet connection, but not always ON
Security is not critical, but the Sync service should encrypt the communication
When in the presence of the companies network the system should sync and use the Server Database and not the local one.
You have a host of issues with thinking about such a solution. First, there are lots of possible solutions, such as:
Using database replication within a database, to mimic every update (like a "hot" backup)
Building an application to copy the database periodically (every night)
Using a third-party tool (which is what you are asking, I think)
With replication services, the connection does not have to always be up. Changes to the database are logged when the connection is not available and then applied when they can be sent.
However, there are lots of other issues when you leave a corporate network. What about security of the data and access rights? Do you have other options, such as making it easier to access the database from within the network? Do the users need only read-access to the database or read-write access? Would both versions need to be accessed at the same time. Would there be updates to both at the same time?
You may have other options that are more secure than just moving a database to the cloud.
I believe RemObjects DataAbstract allows offline mode and synchronization by using what they call Briefcases. All your other requirements (security, encrypted connections, etc.) are also covered.
This is not a drop-in replacement, thought, and may need extensive rewrite/refactoring of your application. There are lots of upsides, thought; business rules can/should be enforced on the server (real security), scriptable business rules, multiplatform architecture, etc.
There are some products available in the Java world (SymmetricDS lgpl license) - apart from actually being a working system it is documents how it achieved synchronization. Connects to any db with jdbc support. . There is a pro version but the user guide (downloadable pdf) gives you the db schema plus rules on push pull syncing. Useful if you want to build your own.
Btw there is a data replication so tag that would help.
One possibility that is free is the Microsoft Sync Framework: http://msdn.microsoft.com/en-us/sync/bb736753.aspx
It may be possible for you to use it, but you would need to provid some more detail about your application and operating environment to be sure.
IS it possible to share a database like .mdb and work fine? i try but sometimes the file where the databse is changes from DB to DB1 i use delphi Xe4 and Google Drive .
Thank´s
I am a junior software engineer,I like this site and people in it.So,I want to ask my problems to see your answers and suggestions.Then,please don't mad at me for asking without searching.
My problem is about Distributed database communication(MSSQL Server 2008).In picture,I need a Main Server in center of star topology and other small servers that holding less data than center.Small servers job is normal web service with small database changes.On the other hand,Main Server should communicate with others periodicly(Once in hour,or twice in a day) and gather distrubited data changes in other small databases.
According to this plan,Main server is trustable,secured and backed up.Here is my question:
I plan to communicate on web services level.Main server should have a methods for controlling and checking databases.Is there any tools for this usage?I am looking forward to your suggestions,visions.
my kind regards and thanks in advance.
Take a look at SQL Server Replication. From the description you have provided, it sounds as though a solution using remote Subscribers that utilise Push updates could provide the functionality you require.
In the first instance, and in the interest of broadening your knowledge, I suggest you familiarise yourself with the varying flavours of solution that are available to you through SQL Server Replication technology and study their corresponding architectures.
I am about to start working on a new system which will need to support multiple users and potentially allow the database to be accessed over the Internet.
The system will be win32, not web based, the database will just be in an office and accessible anywhere. I am not sure if this is a dangerous approach security wise, am open to suggestions
The database will be SQL Server and the system will be implemented in Delphi 6
Does anyone know how I go about starting this? I will need to take into account record locking as well.
If anyone could provide links to good articles that would be appreciated.
Cheers
Paul
IMHO, the easiest way for you is to create a VPN exposing securely your database over Internet.
Security will be very good, because access to the database will be available only through a trusted VPN connection.
And your database will be available from anywhere, using the Internet just as a tunnel to transport your database packets safely.
So your Delphi code will connect to the database just as usual, using TCP/IP connection, via the VPN secure tunnel.
No need to add additional Delphi-only artifacts, like Indy components and such.
And you will be able to connect to your database for not-Delphi client, which could be a good idea to use some database browsing tool.
Exposing the database on the Internet is a security risk. Security flaws could be easily exploitable remotely.
Solutions are:
VPN, as said in other answers. Simple and secure, but requires some setup on both end-ponts (clients and VPN server), and may require proper software on the server - or a VPN router/appliance - and on the client as well if you're not using standard VPN protocols).
A n-tier application, where only the application server is exposed to the internet. You still have to protect the application server properly and the transmission channel. May require less setup on the client side. Delphi 6 offers Datasnap as a n-tier library (it also still supports CORBA, but it was dropped since D7). DCOM is not very firewall friendly (but can be configured to work across them) but can secure the channel on its own, the other two options (socket and HTTP) are easier to setup but a little less secure (they work using DCOM proxies, thereby the client identity is lost, and require custom code or certificates to secure the channel).
A third solution could be to let user connect remotely via remote desktop, but it requires licenses and a machine able to sustain the remote sessions load.
Record locking is handled by the database itself - read the documentation about SQL Server locking mode carefully to avoid bad surprises later. If the connection is not fast enough you may choose to cache some data on the client side (TClientDataset works well for that) and it can also reduce locking issues, but it can introduce udpate conflicts.
You probably mean a client server system that communicates trough TCP/IP.
You can create this using the Indy components. Be sure to check the examples because they are not easy to use, but you can create almost anything network related with them.
Actually, there are dozens of techniques possible, depending on your experiences, preferences and tools that you have available. I would advise you to use ADO to connect to the database and not the BDE, though. To do this, you can use the ADO components that are part of Delphi or import the msado15.dll type library into your project to use raw ADO API calls. The latter will require a lot more experience!SQL Server is able to just expose itself to the Internet, although this creates a security risk. Still, someone who wants to access it will need a username and password to get a connection and you would need to open the ports that SQL Server uses. But technically speaking, to use ADO over the Internet, all you need to know is the IP address of a working server, plus login information. It's a security risk, though. And for that reason, most developers will not expose SQL Server to a database but just write web services to wrap around the specific database functions that you want to expose.Record locking is something SQL Server will do for you, and if you use transactions you can make it even a bit more secure.In the end, the things you need to learn and read about depend heavily on the things you want to do in your application. So before you even start to write some code, start writing a functional design to get an overview of what you want and what you would need for this. From this document, start writing technical documents to describe more precisely what your code needs to do. Once you have this, you can ask more direct questions about the things you need, yet don't know at the moment.