I have an empty table into which I am trying to import data from a flat file. The only option I seem to have is to create the table, which isn't needed. Why can't I select the other options?
In the screen before the one on your screenshot click where the destination table name is to select an existing table.
Related
I am new to AzureSynapse and am technically a Data Scientist whos doing a Data Engineering task. Please help!
I have some xlsx files containing raw data that I need to import into an SQL database table. The issue is that the raw data does not have a uniqueidentifer column and I need to add one before inserting the data into my SQL database.
I have been able to successfully add all the rows to the table by adding a new column on the Copy Data command and setting it to be #guid(). However, this sets the guid of every row to the same value (not unique for each row).
GUID mapping:
DB Result:
If I do not add this mapping, the pipeline throws an error stating that it cannot import a NULL Id into the column Id. Which makes sense as this column does not accept NULL values.
Is there a way to have AzureSynapse analystics read in a raw xlsx file and then import it into my DB with a unique identifier for each row? If so, how can I accomplish this?
Many many thanks for any support.
Giving dynamic content to a column in this way would generate the same value for entire column.
Instead, you can generate a new guid for each row using a for each activity.
You can retrieve the data from your source excel file using a lookup activity (my source only has name column). Give the output array of lookup activity to for each activity.
#activity('Lookup1').output.value
Inside for each, since you already have a linked service, create a script activity. In this script activity, you can create a query with dynamic content to insert values into the destination table. The following is the query I built using dynamic content.
insert into demo values ('#{guid()}','#{item().name}')
This allows you to iterate through source rows, insert each row individually while generating new guid every time
You can follow the above procedure to build a query to insert each row with unique identifier value. The following is an image where I used copy data to insert first 2 rows (same as yours) and inserted the next 2 rows using the above procedure.
NOTE: I have taken Azure SQL database for demo, but that does not affect the procedure.
I have loaded data into HANA using a CSV file and now I have added a new column to the table by
ALTER TABLE <tablename> ADD (<columnname> <datatype>);
Now I want to import data into this specific column using a CSV file. I could not find any methods to import to a specific column. Any help is appreciated. Thanks in advance.
The CSV import feature doesn't allow for partial loading and "column filling".
What you could do is to load the new data (together with the key columns of course) to a staging table and then update the target table columns from there.
Be aware that the CSV import is not meant to be an ETL solution. For that, the smart data integration features are there.
I was referencing this question How to export all data from table to an insertable sql format?
while looking for a way to create an insert statement for a single row from a table without having to manually write it since the table has many columns. In my case I simply followed the steps listed then performed a ctrl-f search in the resulting script for the record I wanted then copied and pasted that single line to another query window but this would be terrible if I had hundreds of millions of rows. Is there a way to get the same functionality but tell the script generator I only want rows where id = value? Is there a better way to do this using only the out of the box Microsoft tools?
There is no way to do this, but you can do it by using a temp table
Create a new table by inset into and select those records which you want to insert.
Create the script and change the table name by using find and replace.
finally drop that temporary table.
While setting up the database replication, the destination table that is getting created is having the same schema name of the source table. I want to have a different destination schema. But how do i get it?
Your question is addressed directly in this forum post.
Snippet:
...change destination_table to the new
table name in sp_addarticle, or in the
wizard, when you get to article
dialog, highlight the table, click the
Article Properties drop down, and
select set properties of highlighted
table article, and in the destination
object name enter the name of the
article.
I have a large (multi-GB) data file exported from an Oracle table. I want to import this data into another Oracle instance, but I want the table name to be different from the original table. Is this possible? How?
Both importing and exporting systems are Oracle 11g. The table includes a BLOB column, if this makes any difference.
Thanks!
UPDATES:
The idea here was to update a table while keeping the downtime on the system that's using it to a minimum. The solution (based on Vincent Malgrat's answer and APC's update) is:
Assuming our table name is A
Make a temp schema TEMP_SCHEMA
Import our data into TEMP_SCHEMA.A
CREATE REAL_SCHEMA.B AS SELECT * FROM TEMP_SCHEMA.A
DROP TABLE REAL_SCHEMA.A Rename REAL_SCHEMA.A to REAL_SCHEMA.A_OLD
Rename REAL_SCHEMA.B to REAL_SCHEMA.A
DROP REAL_SCHEMA.A_OLD
This way, the downtime is only during steps 4 and 5, both should be independent of data size. I'll post an update here if this does not work :-)
If you are using the old EXP and IMP utilities you cannot do this. The only option is to import into a table of the same name (although you could change the schema which owns the table.
However, you say you are on 11g. Why not use the DataPump utility introduced in 10g, which replaces Import and Export. Because in 11g that utility offers the REMAP_TABLE option which does exactly what you want.
edit
Having read the comments the OP added to another response while I was writing this, I don't think the REMAP_TABLE option will work in their case. It only renames new objects. If a table with the original name exists in the target schema the import fails with ORA-39151. Sorry.
edit bis
Given the solution the OP finally chose (drop existing table, replace with new table) there is a solution with Data Pump, which is to use the TABLE_EXISTS_ACTION={TRUNCATE | REPLACE} clause. Choosing REPLACE drops the table whereas TRUNCATE merely, er, truncates it. In either case we have to worry about referential integrity constraints, but that is also an issue with the chosen solution.
I post this addendum not for the OP but for the benefit of other seekers who find this page some time in the future.
I suppose you want to import the table in a schema in which the name is already being used. I don't think you can change the table name during the import. However, you can change the schema with the FROMUSER and TOUSER option. This will let you import the table in another (temporary) schema.
When it is done copy the table to the target schema with a CREATE TABLE AS SELECT. The time it will take to copy the table will be negligible compared to the import so this won't waste too much time. You will need two times the disk space though during the operation.
Update
As suggested by Gary a cleverer method would be to create a view or synonym in the temporary schema that references the new table in the target schema. You won't need to copy the data after the import as it will go through directly to the target table.
Use the option REMAP_TABLE=EXISITNG_TABLE_NAME:NEW_TABLE_NAME in impdp. This works in 11gR2.
Just import it into a table with the same name, then rename the table.
Create a view as select * from ... the table you want to import into, with the view matching the name of the table in the export. Ignore errors on import.