We have a managed deployment of dynatrace and I would like ot know if anyone has configured monitoring for Snowflake with the performance tool?
Please advise and thank you !
We have Dynatrace monitoring our ELT processes (which is pointing at snowflake), and monitoring our Reading processes (which is pointing at snowflake), and monitoring our Snowflake monitoring (which is collecting usage stats from snowflake), so in that sense we can see it in dynatrace, but we are not monitoring it like one of our own services.
Related
I have an application using Rebus with SQL Server as a transport layer (yes, I know this is not the ideal transport layer) and I'm, trying to find an out-of-the-box tool to do real-time monitoring of the queues. I tried Rebus Snoop but I found that it is not compatible with SQL Server.
Does anyone know a way to monitor Rebus queues?
Thank you very much.
"Rebus queues" is not really a thing, so you most likely want to turn to whatever broker you are using and use some kind of monitoring tool for that.
E.g. if you're using Azure Service Bus, you can use the Azure Portal's built-in metrics to figure out queue lengths, request rates, etc.
But since it looks you're using SQL Server as the transport, there doesn't really (to my knowledge) exist any kind of compatible "queue monitoring tool" - you'd probably find loads of tools (e.g. from RedGate) that can help with looking at your SQL Server's performance metrics, but it doesn't know that you happen to be using your SQL Server as a message broker.
You can of course also turn to Rebus Pro that comes with Fleet Manager, which can help with some of the things. It's a commercial product though, so it might not be the answer you were hoping for. 😉
I'm experiencing periodical Azure SQL Database connection slow downs. As recommended in Wait statistics, or please tell me where it hurts article I ran sys.dm_db_wait_stats (analogue of sys.dm_os_wait_stats for Azure SQL Database) aggregation script which show me that longest waits are of type XE_FILE_TARGET_TVF. Average wait time is 54 seconds.
XE_FILE_TARGET_TVF didn't mentioned in documentation as in any other online resources that I know. I suspect that "XE" means "Extended Events", "TVF" is "table-valued functions", "FILE_TARGET" is probably indicator that something is being written to some file.
So, what kind of wait it is?
Use Azure SQL Database Query Performance Insight for troubleshooting SQL Azure DBs. To begin with, wait stats alone are not a correct performance troubleshooting approach. Read How to analyse SQL Server performance for a more thorough approach, learn to analyze also where the CPU is spent, not only where the elapsed time is spent waiting. Aggregate wait stats are often misleading, filtering out the 'benign' wait stats is a chasing the red lights game.
Unfortunately not everything is actionable in SQL Azure DB environment. Start with the Query Performance Insight to see if you can correlate the performance issues with queries issued by your app. Use the SQL Database Index Advisor to get index recommendations for your workload.
If you cannot find the application problems and suspect this is caused by the platform, you will have to open a support case. Twitting #AzureSupport is very effective in getting help.
To answer your question: a lot of Azure SQL DB monitoring relies on Extended Events and they store data to files. This specific wait stat is unlikely to be related to the cause of your performance problems.
I believe that it's a async process so possibly nothing to worry around and could be a red herring. However I think this is a fabric related task behind the DB. Does this event seem to be consistently causing you issues?
I have gone through the documentation on the Logstash server to find out that we require a redis server which will act as a broker.
Here is the link:
http://logstash.net/docs/1.1.12/tutorials/getting-started-centralized
But what is not clear to me is why we use Redis at all as a broker?
We could rather simply directly ship the logs to the elastic search from logstash itself, that would save us the need of the redis broker. Then why do we go for a shipper and a indexer ?
Need clear explanation.
Thanks.
I believe you can find an answer here:
https://groups.google.com/forum/#!topic/logstash-users/VakCOAzZI8k
Redis basically acts as a temporary key value store for raw shipper information which is then parsed by the indexer. Then the log info is ultimately stored in elasticsearch, and not in redis.
Apparently the idea is to offload indexing to a server dedicated to such tasks, as indexing is CPU intensive. Redis being called a broker seems appropriate, i guess.
When using Logstash with Redis, you can configure Redis to only store all the log entries in memory which would like a in memory queue (like memcache).
You mat come to the point where the number of logs sent will not be processed by Logstash and it can bring down your system on constant basis (observed in our environment).
If you feel Redis is an overhead for your disk, you can configure it to store all the logs in memory until they are processed by logstash.
I have an application which has some intensive db operation at some point, along with writing blobs. Some time it takes a a lot of time. Is there a tool which can identify if its some network issue and db being slow.
Any pointers would also be helpful.
By using Wireshark, you can sniff the network traffic between your client and server; this might give you an indication of what's going on.
Most database products have monitoring solutions (Oracle has products from TOAD) which provide realtime insight into what's going on in your database.
I have transactional replication running between two servers over a dedicated VPN connection. The databases are fairly large, so I initially use the backup and restore method to get the initial snapshot over to the subscriber machine and then let it apply the incremental transactions from there.
Everything runs fine until the VPN line gets flaky (which it does occassionally) at which point the replication process is prone to locking up. When I look on the subscriber side, there are a few SQL processes which appear to be hung and have locks held on the subscriber database and tables. The crazy thing is that those processes are coming from the replication service. I can assure you (from trial and error) that no other processes are locking this database except for replication itself.
So why would the replication process trip over its own feet like that? Why would it get hung just because of a loss of network connectivity? Any suggestions for somehow making it more reliable?
I have heard of issues like this over vpn connections. There is a post here that might help you.
Another option, if you have persistent problems, and depending on your requirements for speed and functionality, might be to use log shipping. In my humble opinion this can provide a more resilient way of moving data - at least from a networking perspective.
With SQL Server 2005 they allow you to replicate using a web service. This might not allow you to ditch the VPN but since web services are less connection driven that might help fix the problem. I haven't tried this myself so I don't know what the results may be.
As for the locks we've had a scare thinking alot of things were locked but it turned out that the replication monitor was just locking on its self so make sure you don't have that open when looking at the locks. That doesn't sound like your problem though.
I'll ask some questions and maybe they can give you some ideas as I don't have a clue here either.
Is there a way for the replicator to test for connectivity before attempting to start copying? Is there a way to put a connectivity test into whatever script you're using to perform replication? Is there a way to have the script bail in case of failure?