I am having problems with creating an in memory table, using H2 database, and accessing it outside of the JVM it is created and running in.
The documentation structures the url as jdbc:h2:tcp://<host>/mem:<databasename>
I've tried many combinations, but simply cannot get the remote connection to work. Is this feature working, can anyone give me the details of how they used this.
None of the solutions mentioned so far worked for me. Remote part just couldn't connect.
According to H2's official documentation:
To access an in-memory database from another process or from another computer, you need to start a TCP server in the same process as the in-memory database was created. The other processes then need to access the database over TCP/IP or TLS, using a database URL such as: jdbc:h2:tcp://localhost/mem:db1.
I marked the crucial part of the text in bold.
And I found a working solution at this guy's blog:
The first process is going to create the DB, with the following URL:
jdbc:h2:mem:db1
and itβs going to need to start a tcp Server:
org.h2.tools.Server server = org.h2.tools.Server.createTcpServer().start();
The other processes can then access your DB by using the following URL:
"jdbc:h2:tcp://localhost/mem:db1"
And that is it! Worked like a charm!
You might look at In-Memory Databases. For a network connection, you need a host and database name. It looks like you want one of these:
jdbc:h2:tcp://localhost/mem:db1
jdbc:h2:tcp://127.0.0.1/mem:db1
Complete examples may be found here, here and here; related examples are examined here.
Having just faced this problem I found I needed to append DB_CLOSE_DELAY=-1 to the JDBC URL for the tcp connection. So my URLs were:
In Memory : jdbc:h2:mem:dbname
TCP Connection : jdbc:h2:tcp://localhost:9092/dbname;DB_CLOSE_DELAY=-1
From the h2 docs:
By default, closing the last connection to a database closes the
database. For an in-memory database, this means the content is lost.
To keep the database open, add ;DB_CLOSE_DELAY=-1 to the database
URL.
Not including DB_CLOSE_DELAY=-1 means that I cannot connect to the correct database via TCP. The connection is made, but it uses a different version to the one created in-memory (validated by using the IFEXISTS=true parameter)
In SpringBoot: https://www.baeldung.com/spring-boot-access-h2-database-multiple-apps
#Bean(initMethod = "start", destroyMethod = "stop")
public Server inMemoryH2DatabaseaServer() throws SQLException {
return Server.createTcpServer(
"-tcp", "-tcpAllowOthers", "-tcpPort", "9090");
}
Related
I have a Python package that I am able to run successfully on an Azure Data Science Virtual Machine. However, when I push it to Azure as a Function, I cannot successfully make a database connection. I was getting an error that the ODBC Driver 13 for SQL Server was not supported, so I changed the driver to ODBC Driver 17 for SQL Server and now I am NOT getting an error, but no data is being returned for a query that I know should return data.
Is there any other reason that data would not be returned? Firewall issues? do I need to add a binding? Do I need to separate out the connection string to feed each part (e.g., Driver, UID, PWD) into pyodbc.connect() separately? Right now I am feeding it in like this:
setting = os.environ("CONNECTIONSTRING")
conn = pyodbc.connect(setting)
This query works fine returning data when I run it on the VM using this code, just not as a Function.
(Note, this is different from my previous post regarding reading the Azure App Setting. That problem has been solved).
There are many parts where this could be breaking.
I'd suggest start by having a Profiler or Extended Events trace on your SQL Server to verify whether a connection is even being established. If not then you need to work through the the various points of connectivity to find out where it breaks. The identity, firewall, NSGs etc might all come into play here.
Once you see a connection then you can play with permissions to ensure that your query then returns your data.
Without a full picture of your infrastructure and settings it is hard to pin it down further.
Turns out it was not a database connectivity issue like I thought it was; it was a code error.
Platform: Google Data Studio
Data Source: MySQL
Connection was working before,
meaning no issues with credentials.
All of a sudden, getting the below error:
All IPs have been whitelisted from the google data studio list of ips.
The only thing that comes to mind is a limitation of GDS to process data.
The data source table has around 200K+ rows.
Not sure what is the limitation for GDS with MySQL.
There's no indication anywhere.
Anyone out there can help to solve this or maybe provide some info would be appreciated.
Thanks
If you use a firewall, be sure to double check the Google ip adresses. They may have added new ips (in my case, the last one was missing).
Check them here !
After doing so, I had to change the Host name of the connection to the database for a url alias (www.yourserver.com <- url pointing on your server), and change it back to the IP to make it work.
Sounds like a the connector cannot establish a new connection.
Cloud SQL Connector:
At the time of writing this, the connector seems unable to establish a new connection once the existing one has timed out and modifying the JDBC url to include query parameters gives you an error when authenticating.
This is probably due to the connector appending it's own parameters.
(Seems to be a possible bug here when a connection no longer exists)
MySQL Connector (with IP Address):
This connector allows you to add query parameters to the JDBC url. Enable SSL and append useSSL=true to the url.
e.g.jdbc:mysql://<ip>/<database>?useSSL=true
This worked as expected and establishes new connections when required.
Example Source Setup
Suffering from this issue too, my experience is that using the MySQL connector instead of the Cloud SQL Connector provides better stability in combination with setting wait_timeout to a value above 12 hours.
This issue has been reported on the official Google Data Studio bug tracker. Please vote them up if you are also suffering from this !
π 130205306 MySQL connection does not exist Apr 9, 2019 04:36PM
π 118470083 Data source password not stored for MySQL sources. Oct 26, 2018 01:24PM
(I am a sql noob and I just can not figure this out on my own)
For some time now I have been trying to establish a connection to a SQL database in codename one but to no avail. First I tried connecting to a MariaDB database from one.com. All that's needed for the connection is
Database db = Display.getInstance().openOrCreate("databaseName");
if I am not mistaken, but I am guessing this implies that I have somehow already established a connection to the database. This is not the case however so it creates a new .sql file, right? I can recall that you can connect to a database in the services tab in Netbeans. I chose the MySQL(Connector/ J Driver) which should work with MariaDB, or should it? I entered all my data and i says that it can not establish connection to the database.
the error i get
So I thought I might as well try using localhost. I used XAMPP to host a database and connected in the netbeans services tab.
connected?
Now testing was needed to see if this works. I started the SQL journey with this https://www.codenameone.com/manual/files-storage-networking.html#_sql and integrated the part after "You can probably integrate this code into your app as a debugging tool". I changed database name to "mybase" (it's existance can be confirmed in picture 2). Ran the app, opened the dialog, entered "select ID from customers" and got: java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: customers) It does not get past the first call to "executeQuery". The customers table definitely exists so what am I missing to establish connection?
I really need instructions to connect to the localhost database and ideally also to the one hosted by my webhost provider.
Thanks,
Jona
The Database class is to access the SQLite DB on the mobile device. To connect to external databases, you'd have to do something different, such as a ConnectionRequest or Socket I think.
I have a web application using an embedded (in-memory) H2 database. It seems like the H2 DB is set up correctly on Tomcat startup (no errors in server log), but when accessing the DB (through the application) it seems to be "empty" (no application tables nor data available, even though the log states that they have been created/inserted).
How can I check the H2 DB is really set up or not? I've been trying to connect to the DB using a DB tool (e.g. H2 console, DB Visualizer) but I am not sure about the proper DB connection string or username/pw as it is not explicitly defined in the project.
By raising the log level in the server log, I could at least retrieve this information:
Creating new JDBC Driver Connection to [jdbc:h2:mem:myDataSource;DB_CLOSE_DELAY=-1]
Not sure though whether I am really connected or not because I could pass any user/pw combination and can still "connect"? Probably it's not the right way, because I can only retrieve schemas INFORMATION_SCHEMA and PUBLIC on the DB?
You cannot use a memory database, even a named one, because as soon as the last SQL connection using it is closed, the database will be purged. If you open it again, the only thing you'll see is information_schema tables and views.
I suggest you use embedded mode instead: the database will be persisted, and you can later on open it in another process, a DB viewer, or even in tcp mode if you started H2 server.
As you're using Tomcat, I just wrote today an answer to a similar issue (Embedding an H2 Database within the WEB-INF Directory).
Might be helpful to you: https://stackoverflow.com/a/30638808/3956551
I have 3 servers set up for SQL mirroring and automatic failover using a witness server. This works as expected.
Now my application that connects to the database, seems to have a problem when a failover occurs - I need to manually intervene and change connection strings for it to connect again.
The best solution I've found so far involves using Failover Partner parameter of the connection string, however it's neither intuitive nor complete: Data Source="Mirror";Failover Partner="Principal" found here.
From the example in the blog above (scenario #3) when the first failover occurs, and principal (failover partner) is unavailable, data source is used instead (which is the new principal). If it fails again (and I only tried within a limited period), it then comes up with an error message. This happens because the connection string is cached, so until this is refreshed, it will keep coming out with an error (it seems connection string refreshes ~5 mins after it encounters an error). If after failover I swap data source and failover partner, I will have one more silent failover again.
Is there a way to achieve fully automatic failover for applications that use mirroring databases too (without ever seeing the error)?
I can see potential workarounds using custom scripts that would poll currently active database node name and adjust connection string accordingly, however it seems like an overkill at the moment.
Read the blog post here
http://blogs.msdn.com/b/spike/archive/2010/12/15/running-a-database-mirror-setup-with-the-sqlbrowser-service-off-may-produce-unexpected-results.aspx
It explains what is happening, the failover partner is actually being read from the sql server not from your config. Run the query in that post to find out what is actually being used as the failover server. It will probably be a machine name that is not discoverable from where your client is running.
You can clear the application pool in the case a failover has happened. Not very nice I know ;-)
// ClearAllPools resets (or empties) the connection pool.
// If there are connections in use at the time of the call,
// they are marked appropriately and will be discarded
// (instead of being returned to the pool) when Close is called on them.
System.Data.SqlClient.SqlConnection.ClearAllPools();
We use it when we change an underlying server via SQL Server alias, to enforce a "refresh" of the server name.
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.clearallpools.aspx
The solution is to turn connection pooling off Pooling="false"
Whilst this has minimal impact on small applications, I haven't tested it with applications that receive hundreds of requests per minute (or more) and not sure what the implications are. Anyone care to comment?
Try this connectionString:
connectionString="Data Source=[MSSQLPrincipalServerIP,MSSQLPORT];Failover Partner=[MSSQLMirrorServerIP,MSSQLPORT];Initial Catalog=DatabaseName;Persist Security Info=True;User Id=userName; Password=userPassword.; Connection Timeout=15;"
If you are using .net development, you can try to use ObjAdoDBLib or PigSQLSrvLib and PigSQLSrvCoreLib, and the code will become simple.
Example code:
New object
ObjAdoDBLib
Me.ConnSQLSrv = New ConnSQLSrv(Me.DBSrv, Me.MirrDBSrv, Me.CurrDB, Me.DBUser, Me.DBPwd, Me.ProviderSQLSrv)
PigSQLSrvLib or PigSQLSrvCoreLib
Me.ConnSQLSrv = New ConnSQLSrv(Me.DBSrv, Me.MirrDBSrv, Me.CurrDB, Me.DBUser, Me.DBPwd)
Execute this method to automatically connect to the online database after the mirror database fails over.
Me.ConnSQLSrv.OpenOrKeepActive
For more information, see the relevant links.
https://www.nuget.org/packages/ObjAdoDBLib/
https://www.nuget.org/packages/PigSQLSrvLib/
https://www.nuget.org/packages/PigSQLSrvCoreLib/