I want to load sql query(Number of column change every time)data in Excel workbook that create column header dynamically.
First i have create four variable
Table Customer_NA have some entries
And create Ado.net connection and configure server-name and database name
Drag Script task and and assign all 4 variable in ReadOnlyVariable.
Edit Script task and write code to generate excel sheet column dynamically
public void Main()
{
// TODO: Add your code here
string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
try
{
//Declare Variables
string ExcelFileName = Dts.Variables["User::ExcelFileName"].Value.ToString();
string FolderPath = Dts.Variables["User::FolderPath"].Value.ToString();
string TableName = Dts.Variables["User::TableName"].Value.ToString();
string SheetName = Dts.Variables["User::SheetName"].Value.ToString();
ExcelFileName = ExcelFileName + "_" + datetime;
OleDbConnection Excel_OLE_Con = new OleDbConnection();
OleDbCommand Excel_OLE_Cmd = new OleDbCommand();
//Construct ConnectionString for Excel
string connstring = "Provider=Microsoft.ACE.OLEDB.16.0;" + "Data Source=" + FolderPath + ExcelFileName
+ ";" + "Extended Properties=\"Excel 12.0 Xml;HDR=YES;\"";
//drop Excel file if exists
File.Delete(FolderPath + "\\" + ExcelFileName + ".xlsx");
//USE ADO.NET Connection from SSIS Package to get data from table
SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)(Dts.Connections["Ado_Conn"].AcquireConnection(Dts.Transaction) as SqlConnection);
//Load Data into DataTable from SQL ServerTable
// Assumes that connection is a valid SqlConnection object.
string queryString =
"SELECT * from " + TableName;
SqlDataAdapter adapter = new SqlDataAdapter(queryString, myADONETConnection);
DataSet ds = new DataSet();
adapter.Fill(ds);
//Get Header Columns
string TableColumns = "";
// Get the Column List from Data Table so can create Excel Sheet with Header
foreach (DataTable table in ds.Tables)
{
foreach (DataColumn column in table.Columns)
{
TableColumns += column + "],[";
}
}
// Replace most right comma from Columnlist
TableColumns = ("[" + TableColumns.Replace(",", " Text,").TrimEnd(','));
TableColumns = TableColumns.Remove(TableColumns.Length - 2);
//MessageBox.Show(TableColumns);
//Use OLE DB Connection and Create Excel Sheet
Excel_OLE_Con.ConnectionString = connstring;
Excel_OLE_Con.Open();
Excel_OLE_Cmd.Connection = Excel_OLE_Con;
Excel_OLE_Cmd.CommandText = "Create table " + SheetName + " (" + TableColumns + ")";
Excel_OLE_Cmd.ExecuteNonQuery();
//Write Data to Excel Sheet from DataTable dynamically
foreach (DataTable table in ds.Tables)
{
String sqlCommandInsert = "";
String sqlCommandValue = "";
foreach (DataColumn dataColumn in table.Columns)
{
sqlCommandValue += dataColumn + "],[";
}
sqlCommandValue = "[" + sqlCommandValue.TrimEnd(',');
sqlCommandValue = sqlCommandValue.Remove(sqlCommandValue.Length - 2);
sqlCommandInsert = "INSERT into " + SheetName + "(" + sqlCommandValue.TrimEnd(',') + ") VALUES(";
int columnCount = table.Columns.Count;
foreach (DataRow row in table.Rows)
{
string columnvalues = "";
for (int i = 0; i < columnCount; i++)
{
int index = table.Rows.IndexOf(row);
columnvalues += "'" + table.Rows[index].ItemArray[i] + "',";
}
columnvalues = columnvalues.TrimEnd(',');
var command = sqlCommandInsert + columnvalues + ")";
Excel_OLE_Cmd.CommandText = command;
Excel_OLE_Cmd.ExecuteNonQuery();
}
}
Excel_OLE_Con.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception exception)
{
// Create Log File for Errors
using (StreamWriter sw = File.CreateText(Dts.Variables["User::FolderPath"].Value.ToString() + "\\" +
Dts.Variables["User::ExcelFileName"].Value.ToString() + datetime + ".log"))
{
sw.WriteLine(exception.ToString());
Dts.TaskResult = (int)ScriptResults.Failure;
}
}
}
Same Ado_Conn connection name i have write in c# script still facing error
after successfully buid script and run package and got error
Please help me out
I hope i Explain the scnerio
The alert shows a generic error message, therefore it is not possible to pinpoint the cause of the error.
Consider to add this line into a CATCH block:
Dts.Events.FireError(0, "Script Task Example", exception.Message + "\r" + exception.StackTrace, String.Empty, 0);
Also, perhaps it makes sense, for now, temporarily disable writing logs into a custom log file since this operation by itself can cause errors and they will not be captured by try..catch...
So the edited version will look this way:
catch (Exception exception)
{
Dts.Events.FireError(0, "Script Task", exception.Message + "\r" + exception.StackTrace, String.Empty, 0);
// Create Log File for Errors
// using (StreamWriter sw = File.CreateText(Dts.Variables["User::FolderPath"].Value.ToString() + "\\" +
// Dts.Variables["User::ExcelFileName"].Value.ToString() + datetime + ".log"))
// {
// sw.WriteLine(exception.ToString());
// Dts.TaskResult = (int)ScriptResults.Failure;
// }
}
The real reason of the exception will be routed to SSIS logs and can be tracked via SSDT output window or native SSIS Catalog logging in case if the package is deployed to a server
Related
Good day !
OK so I've made researches to find a solution to my problem, and tried everything and still...
I have to upload records from Excel to SQL Server. First I created a SQL Connection Manager, with the dynamic variable #[User::FilePath] as an expression, and it was working perfectly... until this morning !
Error message:
Error: 0xC0202009 at CriticalList, Connection manager "Excel Connection Manager": SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft Access Database Engine" Hresult: 0x80004005 Description: "External table is not in the expected format.".
Error: 0xC020801C at Load Input file into Staging table, Excel Source 1: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Error: 0xC0047017 at Load Input file into Staging table, SSIS.Pipeline: Excel Source failed validation and returned error code 0xC020801C.
Here is how I setup my connection:
Since, I tried to :
Set the ConnectionString as an expression instead of the FilePath as follow:
“Provider=Microsoft.ACE.OLEDB.12.0; User ID=;Data Source=” + #[User::FilePath] + “;Extended Properties="EXCEL 12.0;HDR=YES;IMEX=1";”
Create an OLE DB Connection instead (Provider=Microsoft.ACE.OLEDB.12.0), but I didn't find the way to pass the FilePath.
Please note that the account that I am using has access to all of the directories where the files are located. Can somebody help me with this please ? My deadline is in two weeks, and I am stuck with this :-(
Thanks a lot in advance for your help.
Mylene
I updated the package to use bulk insert (ADO Connection) in a script task. It' working perfectly.
Here is the CSharp code:
public void Main()
{
string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
try
{
//Declare Variables
string ExcelFileName = Dts.Variables["$Package::ExcelFileName"].Value.ToString();
string FolderPath = Dts.Variables["$Package::FolderPath"].Value.ToString();
string TableName = Dts.Variables["$Package::SQLTableName"].Value.ToString();
string SchemaName = Dts.Variables["$Package::SQLTableSchema"].Value.ToString();
string SheetName = Dts.Variables["$Package::SheetName"].Value.ToString();
ExcelFileName = ExcelFileName + "_" + datetime;
string lastChar = FolderPath.Substring(FolderPath.Length - 1);
//Validate format of FolderPath
if (lastChar != "\\")
{
FolderPath = FolderPath + "\\";
}
OleDbConnection Excel_OLE_Con = new OleDbConnection();
OleDbCommand Excel_OLE_Cmd = new OleDbCommand();
//Construct ConnectionString for Excel
string connstring = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source=" + FolderPath + ExcelFileName
+ ";" + "Extended Properties=\"Excel 12.0 Xml;HDR=YES;\"";
//drop Excel file if exists
File.Delete(FolderPath + "\\" + ExcelFileName + ".xlsx");
//USE ADO.NET Connection from SSIS Package to get data from table
SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)(Dts.Connections["ADO_DBConnection"].AcquireConnection(Dts.Transaction) as SqlConnection);
//Load Data into DataTable from SQL ServerTable
// Assumes that connection is a valid SqlConnection object.
string queryString =
"SELECT * from " + SchemaName + "." + TableName;
SqlDataAdapter adapter = new SqlDataAdapter(queryString, myADONETConnection);
DataSet ds = new DataSet();
adapter.Fill(ds);
//Get Header Columns
string TableColumns = "";
// Get the Column List from Data Table so can create Excel Sheet with Header
foreach (DataTable table in ds.Tables)
{
foreach (DataColumn column in table.Columns)
{
TableColumns += column + "],[";
}
}
// Replace most right comma from Columnlist
TableColumns = ("[" + TableColumns.Replace(",", " Text,").TrimEnd(','));
TableColumns = TableColumns.Remove(TableColumns.Length - 2);
//Use OLE DB Connection and Create Excel Sheet
Excel_OLE_Con.ConnectionString = connstring;
Excel_OLE_Con.Open();
Excel_OLE_Cmd.Connection = Excel_OLE_Con;
Excel_OLE_Cmd.CommandText = "Create table " + SheetName + " (" + TableColumns + ")";
Excel_OLE_Cmd.ExecuteNonQuery();
//Write Data to Excel Sheet from DataTable dynamically
foreach (DataTable table in ds.Tables)
{
String sqlCommandInsert = "";
String sqlCommandValue = "";
foreach (DataColumn dataColumn in table.Columns)
{
sqlCommandValue += dataColumn + "],[";
}
sqlCommandValue = "[" + sqlCommandValue.TrimEnd(',');
sqlCommandValue = sqlCommandValue.Remove(sqlCommandValue.Length - 2);
sqlCommandInsert = "INSERT into " + SheetName + "(" + sqlCommandValue.TrimEnd(',') + ") VALUES(";
int columnCount = table.Columns.Count;
foreach (DataRow row in table.Rows)
{
string columnvalues = "";
for (int i = 0; i < columnCount; i++)
{
int index = table.Rows.IndexOf(row);
columnvalues += "'" + table.Rows[index].ItemArray[i] + "',";
}
columnvalues = columnvalues.TrimEnd(',');
var command = sqlCommandInsert + columnvalues + ")";
Excel_OLE_Cmd.CommandText = command;
Excel_OLE_Cmd.ExecuteNonQuery();
}
}
Excel_OLE_Con.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception exception)
{
// Create Log File for Errors
using (StreamWriter sw = File.CreateText(Dts.Variables["$Package::FolderPath"].Value.ToString() + "\\" + Dts.Variables["$Package::ExcelFileName"].Value.ToString() + datetime + ".log"))
{
sw.WriteLine(exception.ToString());
Dts.TaskResult = (int)ScriptResults.Failure;
}
}
}
}
}
I want to know, how to implement the text qualifier while importing using script task. Below is my code. I have been only able to use delimiter and not text qualifier. So, I am also getting double quotes loaded in my table.
public void Main()
{
// TODO: Add your code here
string SourceFolderPath = Dts.Variables["User::SourceFolder"].Value.ToString();
string FileExtension = Dts.Variables["User::FileExtension"].Value.ToString();
string FileDelimiter = Dts.Variables["User::FileDelimiter"].Value.ToString();
string TableName = Dts.Variables["User::DestinationTable"].Value.ToString();
SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)
(Dts.Connections["PEGASUS.AdventureWorks1"].AcquireConnection(Dts.Transaction) as SqlConnection);
string[] fileEntries = Directory.GetFiles(SourceFolderPath, "*" + FileExtension);
foreach (string fileName in fileEntries)
{
int counter = 0;
string line;
string ColumnList="";
MessageBox.Show(fileName);
System.IO.StreamReader SourceFile = new System.IO.StreamReader(fileName);
while ((line = SourceFile.ReadLine()) != null)
{
if (counter == 0)
{
ColumnList = "[" +(line.Replace(FileDelimiter, "],[").Replace("\"", ""))+ "]";
MessageBox.Show(ColumnList.ToString());
}
else
{
MessageBox.Show("pass 2");
string query = "Insert into " + TableName + " (" + ColumnList + ") ";
query += "VALUES('" + line.Replace(FileDelimiter, "','") + "')";
MessageBox.Show("pass 3");
//MessageBox.Show(query.ToString());
SqlCommand cmd = new SqlCommand(query, myADONETConnection);
cmd.ExecuteNonQuery();
MessageBox.Show("pass 4");
}
counter++;
}
SourceFile.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
}
above are the input and expected output.
I have the following code
string connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + txtFileLocation.Text;
using (DbConnection conn = new OleDbConnection(connectionString))
{
conn.Open();
DbCommand command = conn.CreateCommand();
string idfield = txtIDField.Text;
string picfield = txtPictureField.Text;
command.CommandText = "select " + idfield + ", " + picfield + " from " + selectedTable;
command.CommandType = CommandType.Text;
DbDataReader reader = command.ExecuteReader();
gridResults.DataSource = reader;
conn.Close();
}
The database is an access database as u might have seen from the connection string, also the database I was provided with store their images in the database, as Image attachment.
Whenever I load the grid it shows random stuff :(
eg.
I am not familiar with the wiring up of a database so this might be an easy question, how do I enable my picture list in my form?
When I try to upload an Excel sheet of data into SQL Server I get the following error first time it uploads well when I try to upload it shows the following error
Before uploading I am deleting files in the directory
string[] filePaths1 = Directory.GetFiles(Server.MapPath(#"Excel\"));
foreach (string filePath1 in filePaths)
File.Delete(filePath1);
The process cannot access the file
'C:\inetpub\vhosts\xyz.com\httpdocs\newofficework\Excel\super.xlsx'
because it is being used by another process.
You seem to be performing multiple actions here (1) Delete and then (2) Upload
How are you Uploading the files? If you are using the SaveAs method, then please ensure that you Dispose the same.
E.g. (fuExcel is my FileUpload control)
fuExcel.SaveAs(sUniqueFilePath); // string variable containing file path
fuExcel.Dispose();
The Page having a FileUpload control and the Upload button, on selecting the Excel file user needs to click on Upload button to store the data to Server. Here we are treating the uploaded file as database hence we need to create OLEDB connection to this file, from this connection will be created and the data is fetched to C# as DataTable. '[Sheet1$]' is the Name of the Worksheet where requires data is present.
string SourceConstr = #"Provider=Microsoft.ACE.OLEDB.12.0;Data Source='" + + "';Extended Properties= 'Excel 8.0;HDR=Yes;IMEX=1'";
OleDbConnection con = new OleDbConnection(SourceConstr);
string query = "Select * from [Sheet1$]";
OleDbDataAdapter data = new OleDbDataAdapter(query, con);
data.Fill(dtExcel);
From this code connection will be created for the Excel file and the data in Sheet1 will be dumped to the dtExcel of type DataTable.
Note:'$' must be there after the worksheet name.
fileName = FileUpload1.ResolveClientUrl(FileUpload1.PostedFile.FileName);
int count = 0;
DataClassesDataContext conLinq = new DataClassesDataContext("Data Source=server name;Initial Catalog=Database Name;Integrated Security=true");
try
{
DataTable dtExcel = new DataTable();
string SourceConstr = #"Provider=Microsoft.ACE.OLEDB.12.0;Data Source='" + fileName + "';Extended Properties= 'Excel 8.0;HDR=Yes;IMEX=1'";
OleDbConnection con = new OleDbConnection(SourceConstr);
string query = "Select * from [Sheet1$]";
OleDbDataAdapter data = new OleDbDataAdapter(query, con);
data.Fill(dtExcel);
for (int i = 0; i < dtExcel.Rows.Count; i++)
{
try
{
count += conLinq.ExecuteCommand("insert into table name values(" + dtExcel.Rows[i][0] + "," + dtExcel.Rows[i][1] + ",'" + dtExcel.Rows[i][2] + "',"+dtExcel.Rows[i][3]+")");
}
catch (Exception ex)
{
continue;
}
}
if (count == dtExcel.Rows.Count)
{
<--Success Message-->
}
else
{
<--Failure Message-->
}
}
catch (Exception ex)
{
throw ex;
}
finally
{
conLinq.Dispose();
}
Add these two name space in your class file
using System.Data.OleDb;
using System.Data.SqlClient;
Use following code
public void importDataFromExcel(string excelFilePath)
{
//Declare Variables - Edit these based on your particular situation
string sSQLTable = "tDataMigrationTable";
// make sure your sheet name is correct, here sheet name is Sheet1, so you can change your sheet name if have different
string myExcelDataQuery = "Select StudentName,RollNo,Course from [Sheet1$]";
try
{
//Create our connection strings
string sExcelConnectionString = #"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + excelFilePath + ";Extended Properties=" + "\"Excel 8.0;HDR=YES;\"";
string sSqlConnectionString = "SERVER=MyDatabaseServerName;USER ID=DBUserId;PASSWORD=DBUserPassword;DATABASE=DatabaseName;CONNECTION RESET=FALSE";
//Execute a query to erase any previous data from our destination table
string sClearSQL = "DELETE FROM " + sSQLTable;
SqlConnection SqlConn = new SqlConnection(sSqlConnectionString);
SqlCommand SqlCmd = new SqlCommand(sClearSQL, SqlConn);
SqlConn.Open();
SqlCmd.ExecuteNonQuery();
SqlConn.Close();
//Series of commands to bulk copy data from the excel file into our SQL table
OleDbConnection OleDbConn = new OleDbConnection(sExcelConnectionString);
OleDbCommand OleDbCmd = new OleDbCommand(myExcelDataQuery, OleDbConn);
OleDbConn.Open();
OleDbDataReader dr = OleDbCmd.ExecuteReader();
SqlBulkCopy bulkCopy = new SqlBulkCopy(sSqlConnectionString);
bulkCopy.DestinationTableName = sSQLTable;
while (dr.Read())
{
bulkCopy.WriteToServer(dr);
}
OleDbConn.Close();
}
catch (Exception ex)
{
//handle exception
}
}
I have the following code :
ADOX.Catalog cat = new ADOX.Catalog();
string pathToNewAccessDatabase = "Data Source=D:\\Data\\NewMDB.mdb";
cat.Create("Provider=Microsoft.Jet.OLEDB.4.0;" + pathToNewAccessDatabase + ";Jet OLEDB:Engine Type=5");
System.Data.OleDb.OleDbConnection AccessConn = new System.Data.OleDb.OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0; " + pathToNewAccessDatabase);
AccessConn.Open();
System.Data.OleDb.OleDbCommand AccessCommand = new System.Data.OleDb.OleDbCommand("SELECT * INTO [ReportFile] FROM [Data Source=server_path; Initial Catalog=database_name; User Id=user_name; Password=pass_word;Trusted_Connection=False].[dbo.DataSourceTable]", AccessConn);
AccessCommand.ExecuteNonQuery();
AccessConn.Close();
I want to select from SQL SERVER into the ACCESS database.
Also, if the password contains the [ character, how do I escape that ?
I suggest establishing an SQLConnection first to the SQL server and query your desired data into a DataTable.
using (SqlConnection conn = new SqlConnection("yourConnectionString"))
{
using (SqlCommand comm = new SqlCommand("Select columns from targetTable", conn))
{
SqlDataReader reader = comm.ExecuteReader();
DataTable tbl = new DataTable();
tbl.Load(reader);
}
}
After you have the data in your datatatable, create the query for your insert command from it by looping through the data.
string insertCommandString = string.Empty;
for (int row = 0; row < tbl.Rows.Count; row++)
{
insertCommandString = "Insert into yourTableName(yourColumnNames) values(";
for (int column = 0; column < tbl.Columns.Count; column++)
{
if(tbl.Columns[column].DataType == typeof(String))
{
insertCommandString += "'" + tbl.Rows[row][column].ToString() + "'";
}
else
{
insertCommandString += tbl.Rows[row][column].ToString();
}
if (column < tbl.Columns.Count - 1)
{
insertCommandString += ",";
}
}
insertCommandString += ")";
System.Data.OleDb.OleDbCommand AccessCommand = new System.Data.OleDb.OleDbCommand(insertCommandString, AccessConn);
AccessCommand.ExecuteNonQuery();
}
You should not escape any character from password.