I use GoLang. I have two different DB servers:
Postgresql
MS SQL.
For both of them I make a connection each:
db1, err := sql.Open("postgres", psqlconn)
db2, err := sql.Open("sqlserver", u.String())
The connections are working fine.
Now I want to connect data of the first DB with the second DB: using JOIN.
Example:
rows, err := db1.Query('select *
from db1.dbname1.tabel
left join db2.dbname2.tabel on db2.dbname2.tabel.id = db1.dbname1.tabel.id;')
But this does not work :( How can I JOIN the two tables together? I am not getting anywhere. Or is there another way to do this?
Thanks for your support!!
Matthias
ORMs like gorm are basically an object relational layer, that provide some APIs that can translate your language level entities to SQL queries and vice versa. They basically translate your query in a single SQL query, now your postgresql database receives a query, which is a join of table a (which happens to exist) and another table b which is in another database! Joining is done in database level, not the ORM level. So you cannot achieve this.
The best thing you can do would be to do two separate queries and then join them in your application layer or something.
You should be able to link the databases together using a foreign data wrapper from PostgreSQL to MSSQLSERVER (https://github.com/tds-fdw/tds_fdw) or in the reverse direction using whatever it is that MSSQLSERVER call their equivalent feature. Then you would send the combined query to just one of the databases for it to execute.
Or you could fetch both datasets into GO and join them there, using hash tables and for loops or something. If the datasets are too large to fit into memory, that would be pretty annoying.
Related
I am investigating the possibility to split one DB into multiple. We decided to move some tables into another database, but we have queries with join on these tables. I found a few solutions about how to achieve that:
Azure SQL Database elastic query
EXTERNAL DATA SOURCE
But I don`t know what the difference between them and what to choose.
Thanks for any help!
Azure SQL Database Elastic Queries and External data sources are two names for the same concept.
My suggestion is to avoid cross database queries and avoid splitting one database into multiples because query performance involving external data sources won't be the same no matter what strategy you choose to query those external tables.
If you still want to stick with the plan of splitting the database into multiple databases, then know that cross database queries show good performance when the remote tables are not big. When remote tables are big, this article shows you how to perform joins remotely using table variables and improve performance. This other article shows you also how to push parameterized operations to remote databases and improve performance.
if you are thinking to split your DB into multiple SQL server DB with the different host then you can prefer Linked server which has flexible to join across SQL servers
What is the best solution when I would like to use an Access front-end application with some linked table (via ODBC) from MSSQL Server?
The difficulty of this for me is that I have to use complex queries with many multiple joins (and functions called from queries).
It is very-very slow because of the joins between the two DB (and there is a lot of data in some tables, the 2 GB Access mdb limit is the reason of the MSSQL DB upgrade).
Pass-through query doesn't help because of the joined Access tables.
With OPENDATASOURCE('Microsoft.ACE.OLEDB.12.0'... it is still slow in SQL Server too. I tried ODBC linked view with WHERE clause from MSSQL, but it
seems as slow as the full table.
I have to move all of joined Access tables to the MSSQL DB and convert all queries to Pass-Through? Is there any other solution?
I have to move all of joined Access tables to the MSSQL DB
Yes, definitely.
and convert all queries to Pass-Through?
Not necessarily, only those that are still slow.
"Normal" INNER JOIN queries, using only linked tables from one server database, are handled by Access and the ODBC driver in a way that everything is processed on the server. They should be (more or less) as fast as when run on the server (or as Pass-Through query).
Only "complex" queries, especially involving multiple INNER and OUTER JOINs, won't work like that. You'll notice that they are still very slow when running on linked tables. These need to be changed to Pass-Through queries.
Edit: I just noticed
functions called from queries
You can't call VBA functions from PT queries, and they will again kill performance when called from Access queries running on linked MSSQL tables (because they have to be processed locally).
You'll need to learn to create views in MSSQL, probably also user defined functions and/or stored procedures.
In the long run, you'll find that views are actually easier to manage than PT queries.
My original question was about whether to keep separate ASPNETDB.MDF from the application database or merge all the tables in one database. Checking the previous questions/answers, I learned that it depends on whether the membership data would be shared across several applications.
Now, my question is this. In case I decide to keep ASPNETDB.MDF separate from the application DB, how can I query 2 tables located in 2 different databases?
Thanks for helping.
If you have two databases/schemas on the same database server, you can query across databases with the following syntax:
select *
from database1.dbo.table1 t1 join database2.dbo.table2 t2 on
t1.field1 = t2.field2
If they are on physically separate servers, you can still do a cross-database query, but you need to link the servers first:
http://msdn.microsoft.com/en-us/library/aa213778(v=sql.80).aspx
You can check your SQL Server version by:
SELECT SERVERPROPERTY('Edition')
If you are using "SQL Azure" you will not be able to use a table from different database. You will get this error:
Reference to database and/or server name in 'DataBase.Schema.Table' is not supported in this version of SQL Server.
Even if you try to Query a different database from your file like this:
USE ANOTHER_DATABASE;
You will get this error:
USE statement is not supported to switch between databases. Use a new connection to connect to a different database.
Ive run into the issue where I need to query 2 separate databases(same instance) in one query.
I am used to doing this with mysql, but Im not sure how to do it with DB2.
In mySQL it would be something like:
SELECT user_info.*, game.*
FROM user_info, second_db.game_stats as game
WHERE user_info.uid = game.uid
So the question is how i translate a query like that into DB2 syntax?
Equivalent of this
Is there a reason why you have the tables in a separate database? MySQL doesn't support the concept of schemas, because in MySQL a "schema" is the same thing as a "database". In DB2, a schema is simply a collection of named objects that lets you group them together.
In DB2, a single database is much closer to an entire MySQL server, as each DB2 database can have multiple schemas. With multiple schemas inside the same database, your query can run more or less unchanged from how it is written.
However, if you really have 2 separate DB2 databases (and, for some reason, don't want to migrate to a single database with multiple schemas): You can do this by defining a nickname in your first database.
This requires a somewhat convoluted process of defining a wrapper (CREATE WRAPPER), a server (CREATE SERVER), user mapping(s) (CREATE USER MAPPING) and finally the nickname (CREATE NICKNAME). It is generally easiest to do these tasks using the Control Center GUI because it will walk you through the process of defining each of these.
If any database spreads across multiple servers (ex. Microsoft Sql Server), how can we do join or filter operations. In my scenario, if suppose:
A single table spreads across multiple servers how can we filter rows based on user input?
If master table is there on one db server and transaction table is at another db server, how can we do join operations?
Please let me know how can we achieve this and where can I get more details about this?
I think you're confused about SQL clustering - it doesn't allow you to split tables across multiple servers any differently than you would put different tables in different databases on the same server. Clustering is used for hot failover and redundancy.
However, I think I see what you're asking - if you want to split a database up between different physical servers, the easiest thing to do might be to have a VIEW that unifies those tables together in one place, and then you can query and filter that. SQL Server is smart enough (as long as there are indexes and statistics in place to make the decision from) to send the query where it needs to go if you select something from the unifying view.
For example, say you have two servers - SERVER1 and SERVER2 that both have a database - DATABASE - and each server has a table - TABLE - that has half the data in it (between the two servers, you have every row). Just create a view somewhere - either server, or somewhere else entirely - that looks like this, and then add Linked Servers for SERVER1 and SERVER2 that allow SQL Server to get the data from the remote location:
CREATE VIEW SomeView
AS
SELECT *
FROM SERVER1.DATABASE..TABLE
UNION
ALL
SELECT *
FROM SERVER2.DATABASE..TABLE
That way, you have one place to query and you'll always grab the data from whatever server it's on, instead of querying each server by itself. You can do this even if you don't want to split up individual tables - just create a view for each table you want to move, and have the view check whatever server the table is actually located on.
If I've missed your actual question, please leave a comment and some clarification and I'll be happy to add some more detail.