I am using VFPOLEDB driver to read DBF files and I keep getting this error and I am not sure why and how to fix the problem:
The provider could not determine the Decimal value. For example, the row was just created, the default for the Decimal column was not available, and the consumer had not yet set a new Decimal value.
Here is the code. I call this routine to return a DataSet of the DBF file and display the data in a DataGridView.
public DataSet GetDBFData(FileInfo fi, string tbl)
{
using (OleDbConnection conn = new OleDbConnection(
#"Provider=VFPOLEDB.1;Data Source=" + fi.DirectoryName + ";"))
{
conn.Open();
string command = "SELECT * FROM " + tbl;
OleDbDataAdapter da = new OleDbDataAdapter(command, conn);
DataSet ds = new DataSet();
da.Fill(ds);
return ds;
}
}
I found the solution here:
Error reading certain numeric values with VFPOLEDB driver
SELECT CAST(FieldName As NUMERIC(11, 3)) From TableName
I finally solved the problem by getting the table schema and then casting all of non-character fields to varchar in the select statement. Good enough for previewing the contents of the table.
It is a known issue.
Especially, if You need to select all columns, it is much more comfortable:
Select * from some_table
One working solution is to use another provider, for example Microsoft.Jet.OLEDB.4.0.
Example connection string can be found here: http://docs.30c.org/conn/dbf-foxpro.html
If you add a row from your gridview, it doesn't necessarily use a default value, but rather NULLs, so you may need to pre-set your defaults, or set the schema to NOT Allow Nulls.
You could automate through the columns after the query is done and force defaults based on the columns data types, such as
foreach (DataColumn oDC in YourDataSet.Tables[0].Columns)
{
if (oDC.DataType.ToString().Contains("String"))
oDC.DefaultValue = "";
else if (oDC.DataType.ToString().Contains("Int32"))
oDC.DefaultValue = 0;
else if (oDC.DataType.ToString().Contains("DateTime"))
oDC.DefaultValue = DateTime.MinValue;
}
these are just 3 default types, but there could be others like boolean, decimal, float, whatever, just add into the if/else and put whatever "default" values. It MAY help where otherwise "NULL" values are getting injected in when adding new rows.
Related
Using VS 2013 (VB) and SQL server 2016.
I have linQ query that returns two columns from a database. The query is as follows.
Dim val = (From value In db.ngc_flowTypes
Where value.defaultValue IsNot Nothing
Select value.flowName, value.defaultValue)
The data it returns is a as follows.
I want to iterate through each row of the results and pass the values to certain variables. A ForEach statement doesnt seem to work as it just runs through once. I am sure this must be easy but I ont quite understand it. Am I getting the data returned in the best way via my query? Can I transpose the data to a data table in VB? so I can work with it easier?
The end result I want is string for each flow name with its corresponding default value (along with some other text). So something like this.
dim strsubmission as string = flowName + " has a value of " + defaultValue
Use ToDictionary.
Dim val = (From value In db.ngc_flowTypes
Where value.defaultValue IsNot Nothing
Select value).ToDictionary(Function(key) key.flowName,
Function(value) value.defaultValue)
This will actually execute the SQL of the linq on the database (approx. Select * From ngc_flowTypes Where defaultValue Is Not NULL), traverse each record into a key/value pair (flowName, defaultValue) and put it into a in-memory dictionary variable (val).
After that you can do whatever you like with the dictionary.
For Each flowName In val.Keys
Console.WriteLine("{0} has a value of {1}", flowName, val(flowName))
Next
Edit:
This will only work as long flowName is unique in table ngc_flowTypes
I am creating a winform application in c#.and using sql database.
I have one table, employee_master, which has columns like Id, name, address and phone no. Id is auto increment and all other datatypes are varchar.
I am using this code to get the next auto increment value:
string s = "select max(id) as Id from Employee_Master";
SqlCommand cmd = new SqlCommand(s, obj.con);
SqlDataReader dr = cmd.ExecuteReader();
dr.Read();
int i = Convert.ToInt16(dr["Id"].ToString());
txtId.Text = (i + 1).ToString();
I am displaying on a textBox.
But when last row from table is deleted, still I get that value which is recently deleted in textbox
How should I get the next autoincrement value?
To get the next auto-increment value from SQLServer :
This will fetch the present auto-increment value.
SELECT IDENT_CURRENT('table_name');
Next auto-increment value.
SELECT IDENT_CURRENT('table_name')+1;
------> This will work even if you add a row and then delete it because IDENT_CURRENT returns the last identity value generated for a specific table in any session and any scope.
try this:
SELECT IDENT_CURRENT('tbl_name') + IDENT_INCR('tbl_name');
If you are using Microsoft SQL Server. Use this statement to get current identity value of table. Then add your seed value which you have specified at time of designing table if you want to get next id.
SELECT IDENT_CURRENT(<TableName>)
As for me, the best answer is:
dbcc checkident(table_name)
You will see two values (probably same)
current identity value , current column value
When you delete a row from the table the next number will stay the same as it doesnt decrement in any way.
So if you have 100 rows and you deleted row 100. You would have 99 rows but the next number is still going to be 101.
select isnull((max(AddressID)+1),1) from AddressDetails
the max(id) will get you maximum number in the list pf employee_master
e.g. id = 10, 20, 100 so max will get you 100
But when you delete the record it must have been not 100
So you still get 100 back
One important reason for me to say this might be the issue because you are not using order by id in your query
For MS SQL 2005 and greater:
Select Cast(IsNULL(last_value,seed_value) As Int) + Cast(increment_value As Int) As NextID
From sys.identity_columns
WHERE NAME = <Table_Name>
Just a thought, if what you wanted was the last auto-number that you inserted on an already open connection try using:
SELECT ##IDENTITY FROM...
from that connection. That's the best way to keep track of what has just happened on a given connection and avoids race conditions w/ other connections. Getting the maximum identity is not generally feasible.
SqlConnection con = new SqlConnection("Data Source=.\SQLEXPRESS;Initial Catalog=databasename;User ID=sa;Password=123");
con.Open();
SqlCommand cmd = new SqlCommand("SELECT TOP(1) UID FROM InvoiceDetails ORDER BY 1 DESC", con);
SqlDataReader reader = cmd.ExecuteReader();
//won't need a while since it will only retrieve one row
while (reader.Read())
{
string data = reader["UID"].ToString();
//txtuniqueno.Text = data;
//here is your data
//cal();
//txtuniqueno.Text = data.ToString();
int i = Int32.Parse(data);
i++;
txtuid.Text = i.ToString();
}
I am using SQL Server 2016.
I have a stored procedure GET_RECORDS that takes input parameters for filter and outputs a CURSOR parameter
I want to get this cursor in my SSIS package
I had created data flow task, OleDb source and variables for parameter values. Then mapped parameters
Params mapping screen
but when I wanted to save the component - I got an error
error screen
I tried to add clause WITH RESULT SETS with some dummy columns, but my procedure doesn't return any result set
What am I doing wrong?
Any advices will be helpful.
Thank you.
With regards, Yuriy.
The source component is trying to determine what columns and types will be returned. Because you are using dynamic SQL the metadata can change each time you run it.
With result sets allows you to define the data being returned but should only be used if you are guaranteed to have those results every time you execute.
EDIT:
I create a connection and run the command so that it populates a data table. Then I put the column headers into a string array. There are plenty of examples out there.
Then I use the following function to create a destination table. Finally I create a datareader and pass that to the .Net SqlBulkCopy. Hope this helps.
private void CreateTable(string TableName, string[] Fields)
{
if (TableExists(TableName) && Overwrite)
{
SqlCommand = new SqlCommand($"Drop Table [{TableName}]", SqlConnection);
SqlCommand.ExecuteNonQuery();
}
string Sql = $"Create Table [{TableName}] (";
int ColumnNumber = 1;
foreach (string Field in Fields)
{
string FieldValue = Field;
if (! HasHeaders)
{
FieldValue = "Column" + ColumnNumber;
ColumnNumber++;
}
Sql += $"[{FieldValue}] Varchar(8000),";
}
Sql = Sql + "ImportFileID Int, ID Int Identity(1,1) Not Null, Constraint [PK_" + TableName + "] Primary Key Clustered ([ID] Asc))";
SqlCommand = new SqlCommand(Sql, SqlConnection);
SqlCommand.ExecuteNonQuery();
}
Use ado.net source instead of oledb source, define a simple select and get the columns you wish to return. Now you can define expresión in the dataflow properties.
Search ado.net source dynamic sql
:)
try to return the records and use foreach in ETL instead of cursor
https://www.simple-talk.com/sql/ssis/implementing-foreach-looping-logic-in-ssis/
I think you can do it from a simple way, but I don't know what you are you doing, exactly...
I need to programmatically (ADO.Net) copy records from a table in one database to a table in another database on a different server.
This is very similar to "How can I copy data records between two instances of an SQLServer database" except that I am not allowed to create a link to the destination server so the accepted answer to that question won't work for me.
You can use the SqlBulkCopy class
The SqlBulkCopy class suggested by santiiii is very efficient but it creates a non-logged operation. I had to do this once but my target database participated in replication, so I needed the operation to be fully logged. What I essentially ended up doing was selecting a dataset from the source database .
Select * from SourceDatabaseTable where (some clause to get the right records)
Then creating an empty dataset from the destination table with this statement
Select * from DestinationDatabaseTable where 1<>1
Then I had two datasets. The first with the records I wanted to copy and the second that is empty. Next I just did a nested foreach loop to copy the records from one dataset to the other. Here is the Pseudocode for the core copy function:
foreach(datarow sourcedr in sourcetable)
{
datarow destdr = destdatatable.createrow();
foreach(datacolumn in sourcedatatable)
{
destdr[datacolumn]=Sourcedr[datacolum];
}
}
Lastly, I just used a data adapter to submit the changes on the destination database.
Here's how I did it. Thanks to the other respondants for the inspiration. The code that builds the mappings is not necessary if the schemas of the two tables are identical.
public void CopyTables(string sourceConnectionString, string destConnectionString)
{
string sql = "Select * From SourceTable";
using (SqlConnection sourceConn = new SqlConnection(sourceConnectionString))
using (SqlCommand sourceCmd = new SqlCommand(sql, sourceConn)) {
sourceConn.Open();
using (SqlDataReader reader = sourceCmd.ExecuteReader())
using (SqlBulkCopy copier = new SqlBulkCopy(destConnectionString)) {
copier.DestinationTableName = "DestinationTable";
copier.BulkCopyTimeout = 300;
DataTable schema = reader.GetSchemaTable();
copier.ColumnMappings.Clear();
foreach (DataRow row in schema.Rows) {
copier.ColumnMappings.Add(row["ColumnName"].ToString(), row["ColumnName"].ToString());
}
copier.WriteToServer(reader);
}
}
}
}
I get this error when I do a bulk insert with select * from [table_name], and another table name:
the locale id '0' of the source column 'PAT_NUM_ADT' and the locale id '1033'
of the destination column 'PAT_ID_OLD' do not match
I tried resetting my db collation but this did not help.
Has anyone seen this error?
If you are copying less than a full set of fields from one table to another, whether that table is on another domain across the world, or is collocated in the same database, you just have to select them in order. SqlBulkCopyColumnMappings do not work. Yes, I tried. I used all four possible constructors, and I used them both as SqlBulkCopyMapping objects and just by providing the same information to the Add method of SqlBulkCopy.ColumnMappings.Add.
My columns are named the same. If you're using a different name as well as a different order, you may well find that you have to actually rename the columns. Good luck.
I just had this error message when bulk copying some data. While it might not have been the exact same problem you were having, I was getting the same error.
Specifically, I was doing the following:
SELECT NULL AS ColumnName ...
And the destination was a nullable varchar(3).
In this case, all I needed to do was update my select statement as follows:
SELECT CONVERT(VARCHAR(3),NULL) AS ColumnName...
This worked perfectly and the error message went away!
It is right that when we use SqlBulkCopy, some time it gives error, the best way to map the columns when you are using SqlBulkCopy.
My Previous Code :
SqlConnectionStringBuilder cb = new SqlConnectionStringBuilder("Data Source=ServerName;User Id=userid;Password=****;Initial Catalog=Deepak; Pooling=true; Max pool size=200; Min pool size=0");
SqlConnection con = new SqlConnection(cb.ConnectionString);
SqlCommand cmd = new SqlCommand("select Name,Class,Section,RollNo from Student", con);
con.Open();
SqlDataReader rdr = cmd.ExecuteReader();
SqlBulkCopy sbc = new SqlBulkCopy("Data Source=DestinationServer;User Id=destinationserveruserid;Password=******;Initial Catalog=DeepakTransfer; Pooling=true; Max pool size=200; Min pool size=0");
sbc.DestinationTableName = "StudentTrans";
sbc.WriteToServer(rdr);
sbc.Close();
rdr.Close();
con.Close();
The Code Was giving me the Error as :
The locale id '0' of the source column 'RollNo' and the locale id '1033' of the destination column 'Section' do not match.
Now After Column Mapping my Code Is Running Successfully.
My Modified Code is :
SqlConnectionStringBuilder cb = new SqlConnectionStringBuilder("Data Source=ServerName;User Id=userid;Password=****;Initial Catalog=Deepak;");
SqlConnection con = new SqlConnection(cb.ConnectionString);
SqlCommand cmd = new SqlCommand("select Name,Class,Section,RollNo from Student", con);
con.Open();
SqlDataReader rdr = cmd.ExecuteReader();
SqlBulkCopy sbc = new SqlBulkCopy("Data Source=DestinationServer;User Id=destinationserveruserid;Password=******;Initial Catalog=DeepakTransfer;");
sbc.DestinationTableName = "StudentTrans";
sbc.ColumnMappings.Add("Name", "Name");
sbc.ColumnMappings.Add("Class", "Class");
sbc.ColumnMappings.Add("Section", "Section");
sbc.ColumnMappings.Add("RollNo", "RollNo");
sbc.WriteToServer(rdr);
sbc.Close();
rdr.Close();
con.Close();
This code is running Successfully.
The answer by sal
If you are copying less than a full
set of fields from one table to
another, whether that table is on
another domain across the world, or is
collocated in the same database, you
just have to select them in order.
SqlBulkCopyColumnMappings do not work.
is according to my work absolutely right! Thanks for posting it. Everything has to be the same - data types, etc. Each time it finds a mismatch it throws the mysterious Locale Id error - funny yet frustrating as h###.
I was getting same error and it turned out I was copying from VARCHAR column in the DataTable to INT.
After I changed the data type it worked flawlessly. I successfully copied subset of fields, specifying proper field mappings (mappings worked by both field name or sequence number).
So make sure your data types are correct.
I would check what your default locale settings are. Also, you'll need to check the locale of both tables using sp_help to verify they are the same. If they aren't you'll need to convert it to the correct locale
When you change the Collation of a database the table columns keep the old collation so you need to drop the tables and create them again.
A great way to debug this is to take the sql query being used in your SqlBulkCopy and run it in management studio as a select-into, for instance, change select * from [table_name] to select * into newTable from [table_name], then look at the nullability and data types of 'newTable' versus 'table_name'. If there are any differences then you are likely to end up with this misleading error. Adjust the query or target table until they match, and then your command will work.
Many thanks to Deepak Dwivedi for help. Here is little more hack with COLLATE DATABASE_DEFAULT which finally solved problem for me:
SqlConnectionStringBuilder cb = new SqlConnectionStringBuilder("Data Source=ServerName;User Id=userid;Password=****;Initial Catalog=Deepak;");
SqlConnection con = new SqlConnection(cb.ConnectionString);
SqlCommand cmd = new SqlCommand("select Name COLLATE DATABASE_DEFAULT Name ,Class COLLATE DATABASE_DEFAULT Class ,Section COLLATE DATABASE_DEFAULT Section ,RollNo COLLATE DATABASE_DEFAULT RollNo from Student", con);
con.Open();
SqlDataReader rdr = cmd.ExecuteReader();
SqlBulkCopy sbc = new SqlBulkCopy("Data Source=DestinationServer;User Id=destinationserveruserid;Password=******;Initial Catalog=DeepakTransfer;");
sbc.DestinationTableName = "StudentTrans";
sbc.ColumnMappings.Add("Name", "Name");
sbc.ColumnMappings.Add("Class", "Class");
sbc.ColumnMappings.Add("Section", "Section");
sbc.ColumnMappings.Add("RollNo", "RollNo");
sbc.WriteToServer(rdr);
sbc.Close();
rdr.Close();
con.Close();