Where/How do we check if my snowflake database was down on particular day/Time?
any information_Schema object OR MEta table where its being logged?
you can check snowflake's current operational status here:
https://status.snowflake.com/#
If you want to see where your user is logged in you can check it here:
https://docs.snowflake.com/en/sql-reference/functions/current_region.html
There is no "restarted" as Snowflake is not a singular instance. And for "down" there is only outages, which can be subscribed to.
To subscribe, go to the Status Page page and click “SUBSCRIBE TO UPDATES.” To ensure that you only receive notifications that impact you, please subscribe to your specific region. You can find more detailed instructions on how to subscribe here.
There can be "lack of service" if you incorrectly configure to WAREHOUSE's to not auto start, AND someone stops the WAREHOUSE then you will have no service. But you kind have done this to yourself. But WAREHOUSE "status" can be views via system tables
*"restarted" - well technically, an warehouse can have problems and be restarted, and this can impact query completion, it can be seen in the query performance counters like Warehouse Monitor via Queued (Provisioning), & Queued (Repairing) OR QUERY_HISTORY View QUEUED_PROVISIONING_TIME, & QUEUED_REPAIR_TIME
Related
I would like to add change tracking and change data capture in the same table for SQL Server 2017 (v14.0.3030.27). When I enabled Change tracking it worked, also when I enabled Change data capture, it worked well.
But when I enabled both at the same time in the same table, I'm not getting values of change data capture. couldn't see any documentation saying it so.
My plan is to pull the change using Change tracking and fetch the details from Change Data capture table. Any thoughts?
Given your extra information "But my aim is to poll a small table every minute (if possible seconds)", I would suggest using SqlDependency to be notified when data changes rather than continuously polling.
SqlDependency is ideal for caching scenarios, where your ASP.NET
application or middle-tier service needs to keep certain information
cached in memory. SqlDependency allows you to receive notifications
when the original data in the database changes so that the cache can
be refreshed.
To set up a dependency, you need to associate a SqlDependency object
to one or more SqlCommand objects. To receive notifications, you need
to subscribe to the OnChange event. For more information about the
requirements for creating queries for notifications, see Working with
Query Notifications.
With the caveat:
SqlDependency was designed to be used in ASP.NET or middle-tier
services where there is a relatively small number of servers having
dependencies active against the database. It was not designed for use
in client applications, where hundreds or thousands of client
computers would have SqlDependency objects set up for a single
database server.
We have a requirement for notifying external systems of changes in data in various tables in a SQL Server database. The choice of which data to monitor is somewhat under the control of the user (gets to choose from a list of what we support). The recipients of the notifications may be on a locally connected network (i.e., in the same data center) or they may be remote.
We currently handle this by application code within our data access layer that detects changes and queues notifications on a Service Broker queue which is monitored by a Windows service that performs the actual notification. Not quite real time but close enough.
This has proven to have some maintenance problems so we are looking at using one of the change detection mechanisms that are built into SQL Server. Unfortunately none of the ones I have looked at (I think I looked at them all) seem to fit very well:
Change Data Capture and Change Tracking: Major problem is that they require polling the captured information to determine changes that are to be passed on to recipients. I suspect that will introduce too much overhead.
Notification Services: Essentially uses SQL Server as a web server, which is a horrible waste of licenses. It also requires access through at least two firewalls in the network, which is unacceptable from a security perspective.
Query Notification: Seems the most likely candidate but does not seem to lend itself particularly well to dynamically choosing the data elements to watch. The need to re-register the query after each notification is sent means that we would keep SQL Server busy with managing the registrations
Event Notification: Designed to notify on database or instance level events, not really applicable to data change detection.
About the best idea I have come up with is to use CDC and put insert triggers on the change data tables. The triggers would queue something to a Service Broker queue that would be handled by some other code to perform the notifications. This is essentially what we do now except using a SQL Server feature to do the change detection. I'm not even sure that you can add triggers to those tables but I thought I'd get feedback before spending a lot of time with a POC.
That seems like an awful roundabout way to get the job done. Is there something I have missed that will make the job easier or have I misinterpreted one of these features?
Thanks and I apologize for the length of this question.
Why don't you use update and insert triggers? A trigger can execute clr code, which is explained enter link description here
I am using Hibernate in an Eclipse RAP application. I have database tables mapped to classes with Hibernate and these classes have properties that are fetched lazily (If these weren't fetched lazily then I would probably end up loading the whole database into memory on my first query). I do not synchronize database access so there are multiple Hibernate Sessions for the users and let the DBMS do the transaction isolation. This means different instances of fetched data will belong to different users. There are things that if a user changes those things, then I would like to update those across multiple users. Currently I was thinking about using Hibernate session.refresh(object) in these cases to refresh the data, but I'm unsure how this will impact performance when refreshing multiple objects or if it's the right way to go.
Hope my problem is clear. Is my approch to the problem OK or is it fundamentally flawed or am I missing something? Is there a general solution for this kind of problem?
I would appreciate any comments on this.
The general solution is
to have transactions as short as possible
to link the session lifecycle to the transaction lifecycle (this is the default: the session is closed when the transaction is committed or rolled back)
to use optimistic locking concurrency to avoid two transactions updating the same object at the same time.
If each transaction is very short and transaction A updates some object from O to O', then concurrent transaction B will only see O until it commits or rolls back, and any other transaction started after A will see O', because a new session starts with the transaction.
We maintain an application that does exactly what you are trying to accomplish. Yes, every session.refresh() will hit the database, but since all sessions will refresh the same row at the same time, the DB server will answer all of these queries from memory.
The only thing that you still need to solve is how to propagate the information that something has changed and needs reloading to all the other sessions, possibly even to sessions on a different host.
For our application, we have about 30 users on RCP and 10-100 users on RAP instances that all connect to the very same DB backend (though through pgpool). We use a small network service that every runtime connects to; when a transaction commits, the application tells this change service that "row id X of table T" has changed and this is then propagated to all other "change subscribers", even across JVMs.
But: make sure that session.refresh() is called within the Thread that belongs to that session, possibly the RAP-Display thread. Do not call refresh() from Jobs or other unrelated threads.
As long you don't have a large number of users updating big counts of rows in short time, I guess you won't have to worry about performance.
What's the best strategy to keep all the clients of a database server synchronized?
The scenario involves a database server and a dynamic number of clients that connect to it, viewing and modifying the data.
I need real-time synchronization of the data across all the clients - if data is added, deleted, or updated, I want all the clients to see the changes in real-time without putting too much strain on the database engine by continuous polling for changes in tables with a couple of million rows.
Now I am using a Firebird database server, but I'm willing to adopt the best technology for the job, so I want to know if there is any kind of already existing framework for this kind of scenario, what database engine does it use and what does it involve?
Firebird has a feature called EVENT that you may be able to use to notify clients of changes to the database. The idea is that when data in a table is changed, a trigger posts an event. Firebird takes care of notifying all clients who have registered an interest in the event by name. Once notified, each client is responsible for refreshing its own data by querying the database.
The client can't get info from the event about the new or old values. This is by design, because there's no way to resolve this with transaction isolation. Nor can your client register for events using wildcards. So you have to design your server-to-client notification pretty broadly, and let the client update to see what exactly changed.
See http://www.firebirdsql.org/doc/whitepapers/events_paper.pdf
You don't mention what client platform or language you're using, so I can't advise on the specific API you would use. I suggest you google for instance "firebird event java" or "firebird event php" or similar, based on the language you're using.
Since you say in a comment that you're using WPF, here's a link to a code sample of some .NET application code registering for notification of an event:
http://www.firebirdsql.org/index.php?op=devel&sub=netprovider&id=examples#3
Re your comment: Yes, the Firebird event mechanism is limited in its ability to carry information. This is necessary because any information it might carry could be canceled or rolled back. For instance if a trigger posts an event but then the operation that spawned the trigger violates a constraint, canceling the operation but not the event. So events can only be a kind of "hint" that something of interest may have happened. The other clients need to refresh their data at that time, but they aren't told what to look for. This is at least better than polling.
So you're basically describing a publish/subscribe mechanism -- a message queue. I'm not sure I'd use an RDBMS to implement a message queue. It can be done, but you're basically reinventing the wheel.
Here are a few message queue products that are well-regarded:
Microsoft MSMQ (seems to be part of Windows Professional and Server editions)
RabbitMQ (free open-source)
Apache ActiveMQ (free open-source)
IBM WebSphere MQ (probably overkill in your case)
This means that when one client modifies data in a way that others may need to know about, that client also has to post a message to the message queue. When consumer clients see the message they're interested in, they know to refresh their copy of some data.
SQL Server 2005 and higher support notification based data source caching expiry.
I have an application that consists of a database and several services. One of these services adds information to the database (triggered by a user).
Another service periodically queries the databases for changes and uses the new data as input for processing.
Until now I used a configurable timer that queries the database every 30 seconds or so. I read about Sql 2005 featuring Notification of changes. However, in Sql 2008 this feature is deprecated.
What is the best way of getting notified of changes that occurred in the database directly in code? What are the best practices?
Notification Services was deprecated, but you don't want to use that anyway.
You might consider Service Broker messages in some scenarios; the details depend on your app.
In most cases, you can probably use SqlDependency or SqlCacheDependency. The way they work is that you include a SqlDependency object with your query when you issue it. The query can be a single SELECT or a complex group of commands in a stored procedure.
Sometime later, if another web server or user or web page makes a change to the DB that might cause the results of the previous query to change, then SQL Server will send a notification to all servers that have registered SqlDependency objects. You can either register code to run when those events arrive, or the event can simply clear an entry in the Cache.
Although you need to enable Service Broker to use SqlDependency, you don't need to interact with it explicitly. However, you can also use it as an alternative mechanism; think of it more as a persistent messaging system that guarantees message order and once-only delivery.
The details of how to use these systems are a bit long for a forum post. You can either Google for them, or I also provide examples in my book (Ultra-Fast ASP.NET).
Yes, this blog post explains that Notification Services is now deprecated, and also what the replacements or alternatives are, going forward.
For your purposes - getting notified of changes that occurred in the dataase - it sounds like you want SQL Server Change Tracking. But the notification is a pull model - your app has to do the query on the change table.
I failed to figure out if SqlDependency continues to work with Notification Services deprecated.
There are a number of different ways of tracking changes in the database: either by triggers that maintain temporal structures such as backlogs, tracking logs (aka 'audit tables') or using the change-tracking facilities in SQL 2008 as references in another answer. Irrespective of whatever mechanism you use, you have the problem of notifying your homegrown service of the change. For this, you can use the Service Broker and event-based activation. From what you describe, it seems like having the application wait on an event from the queue.
http://msdn.microsoft.com/en-us/library/ms171581.aspx
If you don't wish to have the service hang around and sleep on the queue, you can investigate into firing the service into life 'on-demand' by using the external activation mechanism in service broker.
You can use the System.Data.SqlClient.SqlDependency (which works with Service Broker on) to subscribe to changes in a table.