I'm beginner in OLAP manipulation in ORACLE. What is difference of OLAP between SQL Server and ORACLE?
Information I've understood about OLAP:
In ORACLE, OLAP is represented logically in data warehouse by relational table with join. We can use T-SQL to query data in OLAP, ORACLE adds ROLLUP, GROUPING, ... in T-SQL to enforce OLAP operations.
In SQL Server:
Like ORACLE, OLAP can be represented logically by relational tables with join, we can use T-SQL with ROLLUP, GROUPING SET, GROUP BY, ... like ORACLE to query data.
OLAP in SQL Server can be stored physically in SSAS, in this case, MS offers MDX to query data in OLAP.
So to learn OLAP in ORACLE, I can use the first option of SQL Server as an alternative (For the moment I have some difficult about material with ORACLE installation)
Thank for any suggestion or confirmation about my information.
Related
Here's the basic idea of what I want to do in SSIS:
I have a large query against a production Oracle database, and I need the following where clause that brings in a long list of ids from SQL Server. From there, the results are sent elsewhere.
select ...
from Oracle_table(s) --multi-join
where id in ([select distinct id from SQL_SERVER_table])
Alternatively, I could write the query this way:
select ...
from Oracle_table(s) --multi-join
...
join SQL_SERVER_table sst on sst.ID = Oracle_table.ID
Here are my limitations:
The Oracle query is large and cannot be run without the where id in (... clause
This means I cannot run the Oracle query, then join it against the ids in another step. I tried this, and the DBA's killed the temp table after it became 3 TB in size.
I have 160k id's
This means it is not practical to iterate through the id's one by one. In the past, I have run against ~1000 IDs, using a comma-separated list. It runs relatively fast - a few minutes.
The main query is in Oracle, but the ids are in SQL Server
I do not have the ability to write to Oracle
I've found many questions like this.
None of the answers I have found have a solution to my limitations.
Similar question:
Query a database based on result of query from another database
To prevent loading all rows from the Oracle table. The only way is to apply the filter in the Oracle database engine. I don't think this can be achieved using SSIS since you have more than 160000 ids in the SQL Server table, which cannot be efficiently loaded and passed to the Oracle SQL command:
Using Lookups and Merge Join will require loading all data from the Oracle database
Retrieving data from SQL Server, building a comma-separated string, and passing it to the Oracle SQL command cannot be done with too many IDs (160K).
The same issue using a Script Task.
Creating a Linked Server in SQL Server and Joining both tables will load all data from the Oracle database.
To solve your problem, you should search for a way to create a link to the SQL Server database from the Oracle engine.
Oracle Heterogenous Services
I don't have much experience in Oracle databases. Still, after a small research, I found something in Oracle equivalent to "Linked Servers" in SQL Server called "heterogeneous connectivity".
The query syntax should look like this:
select *
from Oracle_table
where id in (select distinct id from SQL_SERVER_table#sqlserverdsn)
You can refer to the following step-by-step guides to read more on how to connect to SQL Server tables from Oracle:
What is Oracle equivalent for Linked Server and can you join with SQL Server?
Making a Connection from Oracle to SQL Server - 1
Making a Connection from Oracle to SQL Server - 2
Heterogeneous Database connections - Oracle to SQL Server
Importing Data from SQL Server to a staging table in Oracle
Another approach is to use a Data Flow Task that imports IDs from SQL Server to a staging table in Oracle. Then use the staging table in your Oracle query. It would be better to create an index on the staging table. (If you do not have permission to write to the Oracle database, try to get permission to a separate staging database.)
Example of exporting data from SQL Server to Oracle:
Export SQL Server Data to Oracle using SSIS
Minimizing the data load from the Oracle table
If none of the solutions above solves your issue. You can try minimizing the data loaded from the Oracle database as much as possible.
As an example, you can try to get the Minimum and Maximum IDs from the SQL Server table, store both values within two variables. Then, you can use both variables in the SQL Command that loads the data from the Oracle table, like the following:
SELECT * FROM Oracle_Table WHERE ID > #MinID and ID < #MaxID
This will remove a bunch of useless data in your operation. In case your ID column is a string, you can use other measures to filter data, such as the string length, the first character.
I was playing around with Oracle SQL developer data modeler (SDDM) and created a data model with primary/foreign keys. When I convert it to get DDLs, I see the oracle syntax which is not going to work with Snowflake. So my question is, can you use Oracle SDDM to create data models for snowflake ?
The short answer is almost certainly yes. Snowflake is very forgiving in terms of syntax, and for example accepts the data type VARCHAR2.
You can even create tables with primary and foreign keys - although these will not be actually enforced by Snowflake.
You may find the following links helpful:
CONNECTING TO SNOWFLAKE WITH ORACLE SQL DEVELOPER DATA MODELER (SDDM)
HOW-TO: CUSTOMIZING ORACLE SQL DEVELOPER DATA MODELER (SDDM) TO SUPPORT SNOWFLAKE VARIANT
How to configure Oracle SQL Developer data modeller on Snowflake
I have a requirement where I need to get data from different database i.e. cosmos and sql.
How can I join both the table and get the data?
Below is the data that needs to be fetched. The common column in both is DossierGloabalId which can be use to join both databases tables.
Name--SQL
TaxableYear--SQL
Period--COSMOS
DossierType--SQL
VATType--COSMOS
LastUpdated--SQL
There is no way to do a join between SQL Server and Cosmos DB, as they are two completely different database engines.
Further: While Cosmos DB's SQL API does have a SQL-based query language, it's not a relational database, and is not compatible with SQL Server.
You'll need to do separate sets of queries: one against Cosmos DB, and one against SQL Server.
If you're asking to perform a relational join on data stored in Azure SQL with data stored in CosmosDB, there is no out-of-the-box support for doing so. You are going to have to query records in Azure SQL, query the corresponding records in CosmosDB, and then join them together in your own application code. There are a bunch of approaches to how you go about this in your own application code, and it's highly dependent on your own application.
We currently have a Microsoft SQL Server instance (oltp) we use as our transactional and reporting database. We want to pull out and create a separate database for reporting.
We are currently vetting Redshift and Snowflake. We came up with a question today which is why can't we create a new SQL Server instance for reporting which has the star schema and just use that (instead of redshift or snowflake)? We don't have many tables over a million rows. So maybe using a columnar data warehouse is over kill for us.
Does any know the pros and cons of using Microsoft SQL Server as a reporting database (data warehouse) with a star schema?
We also have a requirement to handle real time or near real time updates.
You can use SQL Server as a data warehouse repository. As long as you have a well designed star schema there is no reason not to use it for that purpose.
I need to migrate about 700 Oracle partitioned tables (RANGE and LIST partitioning) to SQL Server.
Turns out the SSMA (SQL Server Migration Assistant) does not handle Oracle partitioned tables (this is the official answer I got from Microsoft).
Any tool / script / other suggestion to automate this process?
Thanks!
They are correct:
Tried to do this for a project last year for work and found out the same thing:
Tried doing a little research on google to see if things have changed but found out the following:
Migration of Oracle Partitioned Tables is not supported by SSMA. Partitioned tables are migrated as a Non-partitioned simple tables.
Partitioning of the these Tables in SQL server is required to be done manually as per the physical database architecture planning and logical drives of the server system.
Any partition maintenance (adding or dropping or truncating the partitions) related code need to be re-rewritten in SQL Server."