I am new to postgresql and I am trying to import my csv data into the database I created. I created a table in my database with a few columns. I would like to import my data to the table while filtering the columns to the ones I created in the database.
An example of my data source:
X
Y
Z
One
Two
Three
For example, my columns in the table created: X and Z, so I would like only to import X and Z.
I hope I am being clear and sorry for any inconveniences.
Thanks
You can create a temporary table with all the columns from csv.
Then you can copy the file into the temporary table
copy temp_table (x, y, z) from 'csv_file_path'
After this you can insert into your table from the temporary table
insert into your_table (x, z) select x, z from temp_table
Related
How to add a new column to an existed partitioned table in dolphindb?
I create a partitioned table and insert some data in it, the code is:
ID=1..6
x=1..6\5
t1=table(ID, x)
db=database("dfs://rangedb", RANGE, 1 4 7)
pt = db.createPartitionedTable(t1, `pt, `ID)
pt.append!(t1);
Now, I want to add two columns into this table. For an example: the columns are ["price", "qty"], and the data types are [DOUBLE, INT]. What should I do?
It can be added through the addColumn function, an example is as follows:
pt = load Table( "dfs://rangedb" , "pt" )
add Column(pt,[ "price" , "qty" ],[DOUBLE, INT]) ;
We want import data from Oracle to SQL server using SSIS
I was able to transfer data from Oracle to one table (Staging)in SQL. then I need to transform data and I found that I need to run stored procedure to transform the data from Staging to Actual production data. But I wonder How we can do it.
EDIT #1
Source table has four Columns with one field containing date but its datatype is string
Destination table has also four Columns but two column will not be stored as it is there is mapping between source column and destination Column
This mapping is stored in two table for both two column Like Table one stores SourceFeatureID, DestincationFeatureID similarly second table stores SourcePID, DestincationPID
Data is updated periodically so we need from destination data when it was updated last and get remaining where SourceDate > LastUpdated_destination_date
Update 1: Components that you can use to achieve your goal within a Data Flow Task
Source and Destination
OLEDB Source: Read from staging table, you can use an SQL command to return only data with SourceDate > Destination Date
SELECT * FROM StaggingTable T1 WHERE CAST(SourceDate as Datetime) > (SELECT MAX(DestDate) FROM DestinationTable)
OLEDB Destination: Insert data to production database
Join with other table
Lookup transformation: The Lookup transformation performs lookups by joining data in input columns with columns in a reference dataset. You use the lookup to access additional information in a related table that is based on values in common columns.
Merge Join: The Merge Join transformation provides an output that is generated by joining two sorted datasets using a FULL, LEFT, or INNER join
Convert columns data types
Data Conversion transformation: The Data Conversion transformation converts the data in an input column to a different data type and then copies it to a new output column
Derived Column transformation: The Derived Column transformation creates new column values by applying expressions to transformation input columns. An expression can contain any combination of variables, functions, operators, and columns from the transformation input. The result can be added as a new column or inserted into an existing column as a replacement value. The Derived Column transformation can define multiple derived columns, and any variable or input columns can appear in multiple expressions.
References
Lookup Transformation
Merge Join Transformation
Data Conversion Transformation
Derived Column Transformation
Initial answer
I found that I need to run stored procedure to transform the data from Staging to Actual production data
This is not true, you can perform data transfer using DataFlow Task.
There are many links that you can find detailed solutions:
SSIS. How to copy data of one table into different tables?
Create a Project and Basic Package with SSIS
Fill SQL database from a CSV File (even if the source is CSV but it is very helpful)
Executing stored procedure using SSIS
Anyway, to execute a stored procedure from SSIS you can use an Execute SQL Task
Additional informations:
Execute SQL Task
How to Execute Stored Procedure in SSIS Execute SQL Task in SSIS
I'm not going to go through your comments. I'm just going to post an example of loading StagingTable into TargetTable, with an example of a date conversion, and an example of using a mapping table.
This code creates the stored proc
CREATE PROC MyProc
AS
BEGIN
-- First delete any data that exists
-- in the target table that is already there
DELETE TargetTable
WHERE EXISTS (
SELECT * FROM StagingTable
WHERE StagingTable.SomeKeyColumn = TargetTable.SomeKeyColumn
)
-- Insert some data into the target table
INSERT INTO TargetTable(Col1,Col2,Col3)
-- This is the data we are inserting
SELECT
ST.SoureCol1, -- This goes into Col1
-- This column is converted to a date then loaded into Col2
TRY_CONVERT(DATE, ST.SourceCol2,112),
-- This is a column that has been mapped from another table
-- That will be loaded into Col3
MT.MappedColumn
FROM StagingTable ST
-- This might need to be an outer join. Who knows
INNER JOIN MappingTable MT
-- we are mapping using a column called MapCol
ON ST.MapCol = MT.MapCol
END
This code runs the stored proc that you just created. You put this into an execute SQL task after your data flow in the SSIS package:
EXEC MyProc
With regards to date conversion, see here for the numbered styles:
CAST and CONVERT (Transact-SQL)
I extract all the rows from a database table into a table in Matlab,
I make changes in some cells. Now I want to save the content of the Matlab table back into the database table. How can I do it?
The database is SqlServer. The table is a Table in Matlab, which is an exact extraction of a table in Sqlserver database. I am using Matlab version 2014b and datbase toolbox.
SQL = select pr from prices
conn = database('mydb', '', '', 'com.microsoft.sqlserver.jdbc.SQLServerDriver','jdbc:sqlserver://myserver; database=mydb;integratedSecurity=true;');
setdbprefs('DataReturnFormat','table')
e = exec(conn,SQL)
myprices = e.Data
Then I made changes in myprices, which is a table and mirror the table in the database, and I would like to save it back to the database.
Thanks a lot.
Jen
From the MATLAB table you should be able to extract column names as cell array, similarly for each row of your table, then use fastinsert() as described in this link: http://fr.mathworks.com/help/database/ug/fastinsert.html
How to filter an SQL table on the basis of a VALUE that exist in its CSV column.
In detail, I have a table that has a CSV column containing integer number in comma separated form like
12,234,32,55
121,64,43,65
54,25,112,12
996,612,55,3 etc.
now from this table I want to find out the rows that has the value 12(twelve) in csv Column i.e., the row with csv record
12,234,32,55
54,25,112,12 only
Please Help
Select *
from Tablex
where
(Col like '%,12,%')
or (Col = '12')
or LEFT(Col,3)='12,'
or RIGHT(col,3)=',12'
or better as Love2Learn suggested
Select * From Tablex Where ','+Col+',' Like '%,12,%'
I'm kicking around the idea of using the new geometry datatype to represent a vector in the database. The multipoint geometry data type would work well for this, but my question is, is there a way to populate a two column table with the x,y values in a multipoint, where each point is a row in the table, and the X and Y point values go in column1 and column2 respectively?
I'm assuming you have read the Getting Started with the geometry Data Type.
http://msdn.microsoft.com/en-us/library/bb895270.aspx
Beyond that, I can't help with more.
Figured it out:
select
mp.id
,mp.vector.STPointN(nums.number).STX
,mp.vector.STPointN(nums.number).STY
,nums.number
from tblWithMultiPoints mp --table with an int id, and a multipoint called vector
,#NUMBERS nums --temp table with 1 - max num of points in any multipoint
where nums.number <= mp.vector.STNumPoints()