I can connect to my database server (Azure hosted) with SQL Server Management Studio, however I am unable to enumerate its databases.
On the Azure portal, I am seeing a similar issue:
The fact is, these databases are up and running. My application continues to function. I am also able to issues queries via SSMS Query window. The main problem here is the inability to enumerate databases on the server. If you have any idea how I might solve this, thank you very much for any advice which you can provide!
This was an issue with the Azure platform. Here is what we received from our hosting provider in response to our inquiry:
SQL and Open-Source Database Service Management Issues - East US - Mitigated (Tracking ID 8K76-LZ8)
Summary of impact: Between approximately 13:30 and 16:30 UTC on 19 May 2020, a subset of customers in East US may have intermittently experienced timeouts and latency issues when processing service management operations - such as create, update, delete - for SQL resources hosted in this region, including Azure SQL Database and open-source databases such as Azure Database for PostgreSQL, Azure Database for MySQL, and Azure Database for MariaDB. Some customers may have also encountered issues or experienced latency when loading database management tools or expanding database resources in SQL Server Management Studio (SSMS). Retries may have been successful.
Preliminary root cause: After an initial investigation, engineers identified that an increased volume of requests had consumed a large number of available private endpoint connections for the SQL control plane in East US, thus leading to failures for subsequent control plane operations in the region.
Mitigation: Once the preliminary root cause was identified, engineering teams terminated connections from the source of the increased load.
Next steps: We apologize for the impact to affected customers. Engineers will continue to investigate the underlying cause and take steps to prevent future occurrences, which includes:
• Further investigating and understanding the source of increased workload on the SQL platform
• Increasing the resiliency of the SQL platform to better account for this type of work volume
[End of Report]
Anyway, I am posting this so that anyone who encounters this issue in the future should know - it may not be your issue. It may be an issue with the Azure hosting platform in which case you will just need to wait until engineers resolve the issue at their end.
We are in the process of setting up new DB server but need some help on whether we should go Virtual hosting route or physical one. Here is the background info:
Database: SQL Server 2016 with SSRS
It will be centrally hosted with around max 200 concurrent users, system will be accessed by users a crossed globe. In future number of concurrent users may rise to 300 users.
Infra team has assured me that they will be setting up dedicated DB server but they want to host virtual server on it cause it is more beneficial from DR point of view.
Development team prefer to have physical server because it makes life easy when things goes wrong and needs investigation
I hope you can either provide me some guidance on it or point me in right direction on this dilemma.
Many thanks in advance.
As Steve Matthews noted you omitted all kinds of crucial details about the app. But in real life the use of Virtual Machines for production apps is very, very widespread both using VMWARE products and HyperV (Microsoft products). While you may lose 3-10% performance vs running directly on a machine there are many, many advantages to running on a virtual machine (Admins can allocate more memory, CPU and other resources easily if they are needed and available).
Another increasing popular approach is to virtualize using Azure (i.e. the 'cloud.') Here too you can add all kinds of resources as your app's needs changes but of course you will be charged for it. When you go with AZURE there are certain parts of the product you cannot run - see this http://searchsqlserver.techtarget.com/feature/Why-you-should-think-twice-about-Windows-Azure-SQL-Database
You may also want to see this from Microsoft:
https://azure.microsoft.com/en-us/documentation/articles/sql-database-general-limitations/
But much of the administration, including backup, can be done by Azure which makes it very attractive to many shops.
Good luck whichever way you go.
Our company has used Access for its database needs. It wants to stick with the current database frontend, but migrate the tables to some cloud based solution. We do not want to host SharePoint or pay a monthly fee for Office 365. I have used MySql as a backend, but we had to install drivers. We will also be gathering data from Google Forms on a regular basis. Can anyone suggest possible options for this combination? Data from Google Forms, Access frontend, and cloud based backend.
Thanks!
When you say you don't want any monthly fees, are you thus expecting to find some database server system for free and without cost that allows external connections? I simply don't think you going to find such a service for free.
Since office 365 starts at $6 per month, then I not sure why you think this is too high of a cost here? So you could certainly consider to up-size your Access back end tables up to office 365 and continue to use your Access front end. And more amazing is with Access 2010 this means you get a "off line" and disconnected mode. This means that your application will continue to run EVEN WITH NO internet connection. The instant you find a wifi then the data sync process starts again. And this sync is not file based, but record based and is really replication built into the product and this setup requires ZERO extra code on your part.
And since your back end is not an access file, then you can now scale out to millions of users – the only real limit is the size of the 365 server farm (a super huge computer farm).
Keep in mind that in addition to simply linking your Access application to these office 365 tables, you can also publish Access web forms to office 365. So in the following video at the halfway point I switch to running the Access application 100% inn a browser:
http://www.youtube.com/watch?v=AU4mH0jPntI
Note the above resulting browser application does not require any ActiveX or Silverlight. And as noted, again this is again based on that massive server farm.
Another cloud approach is to consider SQL Azure. Access 2010 also has baked into the product the ability to use the cloud based edition of SQL server running on the Azure OS.
So, you could consider using SQL Azure, but that going to be about $10 per month.
I think the office 365 deal at $6 per month is the best bet (and you get lync communication which gives you remote desktop support for your customers or perhaps for supporting this application!). I actually think that Lync makes the $6 worth it alone. Toss in most SharePoint features and document sharing (including free web based Word, and Excel), this is hard to beat.
So it not clear here why you are avoiding office 365, but will have to adopt some kind of server setup here and I not aware of ANY system that going to allow free external connection from your desktop client software such as Access.
I think the best solution is 365 for use with Access.
Another low cost solution I used in the past is to consider some VERY low cost web hosting sites that also allow external connections to their database. In fact I did this for a good number of years (I did not even use the web site hosting!). I simply purchased the monthly web site and used the ability to "connect" external to the database server that was part of the web hosting package. This I did for a good number of years, and at VERY low cost. I thus was deploying Access front ends to multiple places and using this cheap-o web hosting account.
However, I am much now dropping this low cost web setup with the arrival of office 365 being even less cost than that cheap-o $9 web package I was using for this.
So, at the end of the day, I don't think there is any free hosting that allows external database connections, but the most low cost approach is office 365 at this point in time.
Cloud based does not mean you have to think about your database backend any differently, you could it you wished stick with MS access, however as Access does not natively support remote connections then you would need to setup a VPN to your cloud server in order to connect to the .mdb, .aacdb file.
Dedicated database servers are always a nice option (MySql, Sql Server Express, Postgre SQL to mention the free ones) but you will always need to ensure you have the necessary drivers installed (shouldn't be the end of the world)
As for Google forms, I don't really have much experience with those but I imagine Google would have made every effort to ensure they can be implemented relatively easily with existing database products.
We are in the process of rethinking our development environment. At the moment, we all have Elitebook notebooks which are not as fast as we'd like. We're thinking of virtualizing our development environment to a central VM server.
Our developers work in Visual Studio and use SQL Server as a database. We also have a few SharePoint developers who need a 64bit Win2k8 machine for SharePoint 2010. These are already virtual machines with their own local SQL Server installation.
Every developer's machine or VM has SQL Server installed. This requires resources from all boxes, and is challenging when working with a team on a project. Therefore we're looking into the possibility of centralizing the resources into a single DB server. That box would have to run multiple SQL Server instances (each Sharepoint developer needs a separate one to begin with). We also have the need for an older SQL Server 2005 and SQL Server 2000 installation for backwards compatibility. Besides the SQL Server box, the plan consists of a VM session for each developer with the development tools installed. So a developer could just RDP into the development environment, have his own image and make use of the centralized DB server. Test servers will also be virtualized in the same environment.
I'm looking for some tips and best practices on this matter. For instance:
What's the number of SQL Server instances a normal box can take? And if we upscale virtualized cores / memory; is that enough to add new instances? I don't expect heavy usage in dev.
What's the downside on centralizing the SQL Server instances as opposed to keeping a local instance on every development box?
How should this be integrated in a DTAP strategy?
Just some thoughts:
The number of instances is hardware dependent; I'm not sure if there's a mathematical formula to help calculate how many instances you can run on a VMbox, but it sounds like you're going to need a beast of a machine to run multiple instances per developer PLUS development tools (I can barely get SSMS and Visual Studio to play nice on my laptop). Better hardware = more cost.
Developing on a terminal system may have some advantages, but I can think of a few disadvantages as well:
network latency. If your developers work remotely at all, they'll
need a fast internet connection in order to do anything. If the dev
environments stay on their laptops, they could work disconnected.
instance interdependence. Although SQL Server instances are
completely seperate, sometimes you gotta reboot the server. You'll
need to coordinate that with all of your developers (may not be a big
deal if there's only a handful of you).
redundancy/maintenance. If one developer's machine is down, you
lose a person-day (is that even a word?) of productivity; if the
server goes down, your company is paying for a holiday for everyone
:)
Have you priced out what it would cost to upgrade your individual workstations so that you could compare costs on building out a centralized infrastructure?
Again, these are just some challenges that I think you should consider; there are probably ways to offset them, but they may factor in to your decision.
I would centralize the database servers as best as you can. Having all the dev's work off of centralized database instances should make migrating changes between environments MUCH easier. That alone is worth the effort.
For the SharePoint development environment, I'd highly recommend investing in a few books to make sure you go down the right path. You should be able to have all the dev's working off the same development instances for that too. Here is a good book on the subject, I'm sure there are others: SharePoint 2010 Development
As far as making the developer machines VM's too, I would seek the input of the development staff first and foremost and seriously consider Stuart's pros & cons.
Listening to Scott Hanselman's interview with the Stack Overflow team (part 1 and 2), he was adamant that the SQL server and application server should be on separate machines. Is this just to make sure that if one server is compromised, both systems aren't accessible? Do the security concerns outweigh the complexity of two servers (extra cost, dedicated network connection between the two, more maintenance, etc.), especially for a small application, where neither piece is using too much CPU or memory? Even with two servers, with one server compromised, an attacker could still do serious damage, either by deleting the database, or messing with the application code.
Why would this be such a big deal if performance isn't an issue?
Security. Your web server lives in a DMZ, accessible to the public internet and taking untrusted input from anonymous users. If your web server gets compromised, and you've followed least privilege rules in connecting to your DB, the maximum exposure is what your app can do through the database API. If you have a business tier in between, you have one more step between your attacker and your data. If, on the other hand, your database is on the same server, the attacker now has root access to your data and server.
Scalability. Keeping your web server stateless allows you to scale your web servers horizontally pretty much effortlessly. It is very difficult to horizontally scale a database server.
Performance. 2 boxes = 2 times the CPU, 2 times the RAM, and 2 times the spindles for disk access.
All that being said, I can certainly see reasonable cases that none of those points really matter.
It doesn't really matter (you can quite happily run your site with web/database on the same machine), it's just the easiest step in scaling..
It's exactly what StackOverflow did - starting with single machine running IIS/SQL Server, then when it started getting heavily loaded, a second server was bought and the SQL server was moved onto that.
If performance is not an issue, do not waste money buying/maintaining two servers.
On the other hand, referring to a different blogging Scott (Watermasyck, of Telligent) - they found that most users could speed up the websites (using Telligent's Community Server), by putting the database on the same machine as the web site. However, in their customer's case, usually the db & web server are the only applications on that machine, and the website isn't straining the machine that much. Then, the efficiency of not having to send data across the network more that made up for the increased strain.
Tom is correct on this. Some other reasons are that it isn't cost effective and that there are additional security risks.
Webservers have different hardware requirements than database servers. Database servers fare better with a lot of memory and a really fast disk array while web servers only require enough memory to cache files and frequent DB requests (depending on your setup). Regarding cost effectiveness, the two servers won't necessarily be less expensive, however performance/cost ratio should be higher since you don't have to different applications competing for resources. For this reason, you're probably going to have to spend a lot more for one server which caters to both and offers equivalent performance to 2 specialized ones.
The security concern is that if the single machine is compromised, both webserver and database are vulnerable. With two servers, you have some breathing room as the 2nd server will still be secure (for a while at least).
Also, there are some scalability benefits since you may only have to maintain a few database servers that are used by a bunch of different web applications. This way you have less work to do applying upgrades or patches and doing performance tuning. I believe that there are server management tools for making these tasks easier though (in the single machine case).
I would think the big factor would be performance. Both the web server/app code and SQL Server would cache commonly requested data in memory and you're killing your cache performance by running them in the same memory space.
Security is a major concern. Ideally your database server should be sitting behind a firewall with only the ports required to perform data access opened. Your web application should be connecting to the database server with a SQL account that has just enough rights for the application to function and no more. For example you should remove rights that permit dropping of objects and most certainly you shouldn't be connecting using accounts such as 'sa'.
In the event that you lose the web server to a hijack (i.e. a full blown privilege escalation to administrator rights), the worst case scenario is that your application's database may be compromised but not the whole database server (as would be the case if the database server and web server were the same machine). If you've encrypted your database connection strings and the hacker isn't savvy enough to decrypt them then all you've lost is the web server.
One factor that hasn't been mentioned yet is load balancing. If you start off thinking of the web server and the database as separate machines, you optimize for fewer network round trips and also it gets easier to add a second web server or a second database engine as needs increase.
I agree with Daniel Earwicker - the security question is pretty much flawed.
If you have a single box setup with a webserver and only the database for that webserver on it, if that webserver is compromised you lose both the webserver and only the database for that specific application.
This is exactly the same as what happens if you lose the webserver on a 2-server setup. You lose the web server, and just the database for that specific application.
The argument that 'the rest of the DB server's integrity is maintained' where you have a 2-server setup is irrelevant, because in the first scenario, every other database server relating to every other application (if there are any) remain unaffected as well - being, as they are, hosted elsewhere.
Similarly, to the question posed by Kev 'what about all the other databases residing on the DB server? All you've lost is one database.'
if you were hosting an application and database on one server, you would only host databases on that server which related to that application. Therefore, you would not lose any additional databases in a single server setup when compared to a multiple server setup.
By contrast, in a 2 server setup, where the attacker had access to the Web Server, and by proxy, limited rights (in the best case scenario) to the database server, they could put the databases of every other application at risk by carrying out slow, memory intensive queries or maximising the available storage space on the database server. By separating the applications out into their own concerns, very much like virtualisation, you also isolate them for security purposes in a positive way.
I can speak from first hand experience that it is often a good idea to place the web server and database on different machines. If you have an application that is resource intensive, it can easily cause the CPU cycles on the machine to peak, essentially bringing the machine to a halt. However, if your application has limited use of the database, it would probably be no big deal to have them share a server.
Wow, No one brings up the fact that if you actually buy SQL server at 5k bucks, you might want to use it for more than your web application. If your using express, maybe you don't care. I see SQL servers run Databases for 20 to 30 applicaitions, so putting it on the webserver would not be smart.
Secondly, depends on whom the server is for. I do work for financial companies and the govt. So we use a crazy pain in the arse approach of using only sprocs and limiting ports from webserver to SQL. So if the web app gets hacked. The only thing the hacker can do is call sprocs as the user account on the webserver is locked down to only see/call sprocs on the DB. So now the hacker has to figure out how to get into the DB. If its on the web server well its kind of easy to get to.
It depends on the application and the purpose. When high availability and performance is not critical, it's not bad to not to separate the DB and web server. Especially considering the performance gains - if the appliation makes a large amount of database queries, a considerable amount of network load can be removed by keeping it all on the same system, keeping the response times low.
I listened to that podcast, and it was amusing, but the security argument made no sense to me. If you've compromised server A, and that server can access data on server B, then you instantly have access to the data on server B.
I think its because the two machines usually would need to be optimized in different ways. Other than that I have no idea, we run all our applications with the server-database on the same machine - granted we're not public facing - but we've had no problems.
I can't imagine that too many people care about one machine being compromised over both since the web application will usually have nearly unrestricted access to at the very least the data if not the schema inside the database.
Interested in what others might say.
Database licences are not cheep and are often charged per CPU, therefore by separating out your web-servers you can reduce the cost of your database licences.
E.g if you have 1 server doing both web and database that contains 8 CPUs you will have to pay for an 8 cpu licence. However if you have two servers each with 4 CPUs and runs the database on one server you will only have to pay for a 4 cpu licences
An additional concern is that databases like to take up all the available memory and hold it in reserve for when it wants to use it. You can force it to limit the memory but this can considerably slow data access.
Something not mentioned here, and the reason I am facing, is 0 downtime deployments. Currently I have DB/webserver on same machine and that makes updates a pain. If you they are on a seprate machine, you can perform A/B releases.
I.e.:
The DNS currently points to WebServerA
Apply sofware updates to WebServerB
Change DNS to point to WebServerB
Work on WebServerA at leisure for the next round of updates.
This works before the state is stored in the DB, on a separate server.
Arguing that there is a real performance gain to be had by running a database server on a web server is a flawed argument.
Since Database servers take query strings and return result sets, the data actually flowing from data server to web server is relatively small, but the horsepower required to process the query and generate the result set is relatively large. Optimizing performance around the data transfer time therefore is optimizing around the wrong thing.
Regarding security, there are advantages to having the data server on a different box than the web server. Having such a setup is not the be all and end all of security, but it is a step in the right direction.
Regarding scalability, it is easy and relatively cheap to add web servers and put them into cluster to handle increased traffic. It is not so easy and cheap to add data servers and cluster them. Also, web servers and data servers have different hardware needs, so multiple boxes help out with scalability.
If you are starting small and have only one box, then a good way would go would be to use virtual machines. Running the web server and data server in different VMs on one host gives you all the gains of separate boxes at the cost of one large box price.
Operating system is another consideration. While your database may require larger memory spaces and therefore UNIX, your web server - or more specifically your app server since you mention only two tiers - may be a .Net-based, and therefore require Windows.
Ok! Here is the thing, it is more Secure to have your DB Server installed on another Machine and your Application on the Web Server. You then connect your application to the DB with a Web Link. Thanks it.