I want to use Elastic search for full text searching in mu SQL server.
As i read I can index my SQL server by elastic search.
My questions is about resource usage.
As I understood by indexing I must duplicate my entire SQL server database.
and I must take both elastic and SQL server sync.
Am I wrong?
Thanks
Yes, you are right. I also maintain a sql server + elastic environment.
You need a separate instance of elasticsearch running.
You will have to index / insert all your existing data from sql server to elastic. There are libraries for almost any language to easily insert data.
You will have to sync these two in order to keep the data up to date.
In theory you only need 1 instance / node of elastic running, but it is recommended to have a full cluster, due to failover, etc.
Related
We are in the process of developing a web application that we want to host on Azure, where the storage size of the SQL database database is expected to reach 100 TB.
The issue is the max size of the Azure SQL database is currently 4 TB which too small in our case.
Knowing the fact that we are using DTU pricing model, is there any workaround for this issue?
You need to scale out by using SQL elastic pool. You will be using shard database, in other words creating different Azure SQL databases. Azure SQL elastic pool will make it easier to manage the database and query the data in a simple way using one Azure SQL Database elastic query.
I recommend reading the following documents from Microsoft docs:
sql-database-elastic-scale-introduction
sql-database-elastic-scale-shard-map-management
sql-database-elastic-query-overview
I want to know if it's posible to share SQL Server SESSION CONTEXT variables between different Azure Sql databases using Elastic Queries.
I searched in official documentation but i can't found any information about this feature is available or not.
SESSION CONTEXT exists locally to a single server instance in SQL Server. (It's tied to a session). SQL Azure is built using SQL Server but there are some parts of the mapping that are opaque to customers (they can change based on circumstances such as what Edition you use or what version of the internal software we are using to deliver the service).
Elastic Queries is a feature to let you query from one database (source) to one or more other databases (target(s)). In such a model, you have a SQL Server session to the source database, and the elastic query has a separate connection/session to each other database being touched.
I think the question you are asking is "can I set the session context on the source connection/session and have it flow through to all the target connections when running queries there?" (That's my best guess - let me know if it is different). The answer today is "no" - the session variables do not flow from source to target as part of the elastic query. Also, since today elastic query is read-only, you can't use elastic query to set the session context individually on each target database connection/session as part of the operation.
In the future, we'll consider whether there is something like this we can do, but right now we don't have a committed timeline for something like this.
I hope this explains how things work a bit under the convers.
Sincerely,
Conor Cunningham
Architect, SQL
Hey guys i am learning about MS SQL-Server clustering instances. I am wondering if you could add a cluster on a existing instance of SQL server 2012.
I mean is it the whole point of clustered instances: to create a new instance that supports disaster recovery?
I have no experience in clustering a SQL Server (I'm no DBA) so all I can give is a link (brentozar.com - An Introduction to SQL Server Clusters) with a good read.
The main things to take away from this excellent post, written by Kendra Little, and I quote:
Key Concept: A Windows Failover Cluster uses shared storage– typically, this shared storage is on a SAN. When a SQL Server instance is installed on the cluster, system and user databases are required to be on the shared storage. That allows the cluster to move the SQL instance to any server (or “node”) in the cluster whenever you request, or if one of the nodes is having a problem. There is only one copy of the data, but the network name and SQL Server service for the instance can be made active from any cluster node.
Translation: A failover cluster basically gives you the ability to have all the data for a SQL Server instance installed in something like a share that can be accessed from different servers. It will always have the same instance name, SQL Agent jobs, Linked Servers and Logins wherever you bring it up. You can even make it always use the same IPAddress and port– so no users of the SQL Server have to know where it is at any given time.
The link has a beautiful description of a use-case with 2 simple images but I don't want to incorporate the whole post in here. If the quote doesn't make things clear, surf to the link.
It seems easy to apply an Azure Search index to an SQL Azure database. I undertand that you query the search index using REST APIs and that the index then needs to be maintained/updated. Now, consider a web server running IIS, with an underlying SQL Server database.
What is considered best practice; querying and updating the index from the web server or from SQL Server, e.g. from within a CLR stored procedure? Are there specific design considerations here?
I work on Azure Search team and will try to help.
Querying and updating the index are two different use cases. Presumably, you want to query the index in response to user input in your Web app. (It is also possible that you have a SQL stored procedure with some complex logic that needs full test search, but that seems less likely).
Updating the index can be done in multiple ways. If you can tolerate updating your index at most every 5 minutes, use Azure Search SQL indexer automagically update the index for you - see http://azure.microsoft.com/en-us/documentation/articles/search-howto-connecting-azure-sql-database-to-azure-search-using-indexers-2015-02-28/ for details on how to do it. That article describes creating indexers using REST API, but we now have support for that in .NET SDK as well.
OTOH, if you need hard real-time updates, you can update the search index at the same time you produce data to insert / update your SQL database.
Let me know if you have any follow up questions!
HTH,
Eugene
Based on our requirement we need to have one database per account as we cannot have a single database. So we were thinking about multiple database on the SQL Server and based on my research we can have appx 32,000 database on single database instance but the resource limit will be reached far sooner.
How many is too many databases on SQL Server?
Another point was to use no-sql database like MongoDB. As far as I know we can create database per account with different database name.
I wanted to check reference to resources used by MongoDB, do I need more ram, processor and lots of servers for the architecture.
We are looking to have about 10,000 databases max on a server. Can that work on the a single mongodb server?
If you do have similar architecture currently running, please do share.