I have using previous version of npgsql (2.0.7), it works fine.
Now I have upgrade to npgsql 3.0.5
and with new rewrite of copy method of 3.0.5
I have to change code for geometry
I try to using
while
{
var line = new NpgsqlLine(122149.006850, 483672.683450, 122156.366150);
writer.Write<NpgsqlLine>(line, NpgsqlDbType.Line)
}
writer.Close();
at debug mode : in the loop is ok, but when writer.Close()
Error!! with this message
XX000: Invalid endian flag value encountered.
Need help on this, any suggestion are highly appreciate.
Thanks in advance.
Hope it helps you as I was having the same problem.
In my case, it has been resolved by taking away my "using" directive :
instead of:
using (var writer = conn.BeginBinaryImport("sql copy from stdin command"){ ///writer.StartRow(); writer.Write(...); writer.Write();}
Take away using :
var writer = conn.BeginBinaryImport("sql copy from stdin command");
then in while statement:
while(condition){writer.StartRow(); writer.Write(...); writer.Write();}
I think the problem comes from writer.Dispose() managed by using directive : In fact, without using directive, if you call your writer.Dispose(), the same Npgsql exception is raised.
Good luck!
Above does not work - in 4.05 you have to do
dbimporter.Complete()
but this works for all geometry types loading into a geometry column in postgres
using
Dim conn As New NpgsqlConnection
Dim ct As String = "COPY " + datatab + "( " + datacolumns + ") FROM STDIN (FORMAT BINARY)"
Dim dbimporter = conn.BeginBinaryImport(ct)
when you need to import geometry data use this, you will need sqlserverdatatypes dll
dim wkt1 as string ="POINT(7 7)" - any wkt geometry representation
Dim udtText1 As New System.Data.SqlTypes.SqlChars(wkt1)
Dim sqlGeometry11 As Microsoft.SqlServer.Types.SqlGeometry = Microsoft.SqlServer.Types.SqlGeometry.STGeomFromText(udtText1, srid)
Dim ms1 As New MemoryStream()
Dim bw1 As New BinaryWriter(ms1)
Dim WKB1() As Byte = sqlGeometry11.STAsBinary().Buffer
bw1.Write(WKB1)
dbimporter.Write(WKB1, NpgsqlTypes.NpgsqlDbType.Bytea)
Related
Consider a text file stored in an online location that looks like this:
;aiu;
[MyEditor45]
Name = MyEditor 4.5
URL = http://www.myeditor.com/download/myeditor.msi
Size = 3023788
Description = This is the latest version of MyEditor
Feature = Support for other file types
Feature1 = Support for different encodings
BugFix = Fix bug with file open
BugFix1 = Fix crash when opening large files
BugFix2 = Fix bug with search in file feature
FilePath = %ProgramFiles%\MyEditor\MyEditor.exe
Version = 4.5
Which details information about a possible update to an application which a user could download. I want to load this into a stream reader, parse it and then build up a list of Features, BugFixes etc to display to the end user in a wpf list box.
I have the following piece of code that essentially gets my text file (first extracting its location from a local ini file and loads it into a streamReader. This at least works although I know that there is no error checking at present, I just want to establish the most efficient way to parse this first. One of these files is unlikely to ever exceed more than about 250 - 400 lines of text.
Dim UpdateUrl As String = GetUrl()
Dim client As New WebClient()
Using myStreamReader As New StreamReader(client.OpenRead($"{UpdateUrl}"))
While Not myStreamReader.EndOfStream
Dim line As String = myStreamReader.ReadLine
If line.Contains("=") Then
Dim p As String() = line.Split(New Char() {"="c})
If p(0).Contains("BugFix") Then
MessageBox.Show($" {p(1)}")
End If
End If
End While
End Using
Specifically I'm looking To collate the information about Features, BugFixes and Enhancements. Whilst I could construct what would in effect be a rather messy if statement I feel sure that there must be a more efficient way to do this , possibly involving linq. I'd welcome any suggestions.
I have added the wpf tag on the off chance that someone reading this with more experience of displaying information in wpf listboxes than I have might just spot a way to effectively define the info I'm after in such a way that it could then be easily displayed in a wpf list box in three sections (Features, Enhancements and BugFixes).
Dom, Here is an answer in C#. I will try to convert it to VB.Net momentarily. First, since the file is small, read all of it into a list of strings. Then select the strings that contain an "=" and parse them into data items that can be used. This code will return a set of data items that you can then display as you like. If you have LinqPad, you can test thecode below, or I have the code here: dotnetfiddle
Here is the VB.Net version: VB.Net dotnetfiddle
Imports System
Imports System.Collections.Generic
Imports System.Linq
Public Class Program
Public Sub Main()
Dim fileContent As List(Of String) = GetFileContent()
Dim dataItems = fileContent.Where(Function(c) c.Contains("=")).[Select](Function(c) GetDataItem(c))
dataItems.Dump()
End Sub
Public Function GetFileContent() As List(Of String)
Dim contentList As New List(Of String)()
contentList.Add("sb.app; aiu;")
contentList.Add("")
contentList.Add("[MyEditor45]")
contentList.Add("Name = MyEditor 4.5")
contentList.Add("URL = http://www.myeditor.com/download/myeditor.msi")
contentList.Add("Size = 3023788")
contentList.Add("Description = This is the latest version of MyEditor")
contentList.Add("Feature = Support for other file types")
contentList.Add("Feature1 = Support for different encodings")
contentList.Add("BugFix = Fix bug with file open")
contentList.Add("BugFix1 = Fix crash when opening large files")
contentList.Add("BugFix2 = Fix bug with search in file feature")
contentList.Add("FilePath = % ProgramFiles %\MyEditor\MyEditor.exe")
contentList.Add("Version = 4.5")
Return contentList
End Function
Public Function GetDataItem(value As String) As DataItem
Dim parts = value.Split("=", 2, StringSplitOptions.None)
Dim dataItem = New DataItem()
dataItem.DataType = parts(0).Trim()
dataItem.Data = parts(1).Trim()
Return dataItem
End Function
End Class
Public Class DataItem
Public DataType As String
Public Data As String
End Class
Or, in C#:
void Main()
{
List<string> fileContent = GetFileContent();
var dataItems = fileContent.Where(c => c.Contains("="))
.Select(c => GetDataItem(c));
dataItems.Dump();
}
public List<string> GetFileContent()
{
List<string> contentList = new List<string>();
contentList.Add("sb.app; aiu;");
contentList.Add("");
contentList.Add("[MyEditor45]");
contentList.Add("Name = MyEditor 4.5");
contentList.Add("URL = http://www.myeditor.com/download/myeditor.msi");
contentList.Add("Size = 3023788");
contentList.Add("Description = This is the latest version of MyEditor");
contentList.Add("Feature = Support for other file types");
contentList.Add("Feature1 = Support for different encodings");
contentList.Add("BugFix = Fix bug with file open");
contentList.Add("BugFix1 = Fix crash when opening large files");
contentList.Add("BugFix2 = Fix bug with search in file feature");
contentList.Add("FilePath = % ProgramFiles %\\MyEditor\\MyEditor.exe");
contentList.Add("Version = 4.5");
return contentList;
}
public DataItem GetDataItem(string value)
{
var parts = value.Split('=');
var dataItem = new DataItem()
{
DataType = parts[0],
Data = parts[1]
};
return dataItem;
}
public class DataItem
{
public string DataType;
public string Data;
}
The given answer only focuses on the first part, converting the data to a structure that can be shaped for display. But I think you main question is how to do the actual shaping.
I used a somewhat different way to collect the file data, using Microsoft.VisualBasic.FileIO.TextFieldParser because I think that makes coding just al little bit easier:
Iterator Function GetTwoItemLines(fileName As String, delimiter As String) _
As IEnumerable(Of Tuple(Of String, String))
Using tfp = New TextFieldParser(fileName)
tfp.TextFieldType = FieldType.Delimited
tfp.Delimiters = {delimiter}
tfp.HasFieldsEnclosedInQuotes = False
tfp.TrimWhiteSpace = False
While Not tfp.EndOfData
Dim arr = tfp.ReadFields()
If arr.Length >= 2 Then
Yield Tuple.Create(arr(0).Trim(), String.Join(delimiter, arr.Skip(1)).Trim())
End If
End While
End Using
End Function
Effectively the same thing happens as in your code, but taking into account Andrew's keen caution about data loss: a line is split by = characters, but the second field of a line consists of all parts after the first part with the delimiter re-inserted: String.Join(delimiter, arr.Skip(1)).Trim().
You can use this function as follows:
Dim fileContent = GetTwoItemLines(file, "=")
For display, I think the best approach (most efficient in terms of lines of code) is to group the lines by their first items, removing the numeric part at the end:
Dim grouping = fileContent.GroupBy(Function(c) c.Item1.TrimEnd("0123456789".ToCharArray())) _
.Where(Function(k) k.Key = "Feature" OrElse k.Key = "BugFix" OrElse k.Key = "Enhancement")
Here's a Linqpad dump (in which I took the liberty to change one item a bit to demonstrate the correct dealing with multiple = characters:
You could do it with Regular Expressions:
Imports System.Text.RegularExpressions
Private Function InfoReader(ByVal sourceText As String) As List(Of Dictionary(Of String, String()))
'1) make array of fragments for each product info
Dim products = Regex.Split(sourceText, "(?=\[\s*\w+\s*])")
'2) declare variables needed ahead
Dim productProperties As Dictionary(Of String, String)
Dim propertyNames As String()
Dim productGroupedProperties As Dictionary(Of String, String())
Dim result As New List(Of Dictionary(Of String, String()))
'2) iterate along fragments
For Each product In products
'3) work only in significant fragments ([Product]...)
If Regex.IsMatch(product, "\A\[\s*\w+\s*]") Then
'4) make array of property lines and extract dictionary of property/description
productProperties = Regex.Split(product, "(?=^\w+\s*=)", RegexOptions.Multiline).Where(
Function(s) s.Contains("="c)
).ToDictionary(
Function(s) Regex.Match(s, "^\w+(?=\s*=)").Value,
Function(s) Regex.Match(s, "(?<==\s+).*(?=\s+)").Value)
'5) extract distinct property names, ignoring numbered repetitions
propertyNames = productProperties.Keys.Select(Function(s) s.TrimEnd("0123456789".ToCharArray)).Distinct.ToArray
'6) make dictionary of distinctProperty/Array(Of String){description, description1, ...}
productGroupedProperties = propertyNames.ToDictionary(
Function(s) s,
Function(s) productProperties.Where(
Function(kvp) kvp.Key.StartsWith(s)
).Select(
Function(kvp) kvp.Value).ToArray)
'7) enlist dictionary to result
result.Add(productGroupedProperties)
End If
Next
Return result
End Function
help, combobox just keep adding items, i tried using removeallitems but after that i cant put anything on the first combobox
public class Function {
public void combofillsect(JComboBox section, String year){
Connection conn = null;
PreparedStatement pst = null;
ResultSet rs = null;
String query;
try{
query = "Select Section from asd where Year=?";
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/test","root","");
pst = conn.prepareStatement(query);
pst.setString(1, year);
rs = pst.executeQuery();
while(rs.next()){
section.addItem(rs.getString("Section"));
}
}catch(Exception e){
JOptionPane.showMessageDialog(null, e);
section.addItem(e.toString());
};
}
Function funct= new Function();
{funct.combofillsect(jComboBox1,String.valueOf(jComboBox2.getSelectedItem())); }
why cant I post image?
Are you programming in C# ? If it's the case then you can use the function Clear like that : yourComboBox.Items.Clear() to delete all the current items. I don't know if it will solve your problem but your technique of getting the data from your database seems weird to me, if you used a dataset you could have done dataset.Tables(0).Rows.Count() to get the number of entries and then set the exit condition of your loop like this -> counter < dataset.Tables(0).Rows.Count(), and set a counter++ at the end of your while (maybe that's why you say your combobox won't stop filling, but I don't know what do the next() function).
I don't know the C# code but there is my VB.NET function :
Public Function getAll() As DataSet
ConnectionDB()
Dim cmd As SqlClient.SqlCommand
cmd = New SqlClient.SqlCommand("SELECT * FROM table", Connect)//Connect is a System.Data.SqlClient.SqlConnection, or my connection string
Dim adapter As New Data.SqlClient.SqlDataAdapter
Dim dataset As New DataSet
adapter.SelectCommand = cmd
adapter.Fill(dataset)
adapter.Dispose()
cmd.Dispose()
Connect.Close()
Return dataset
End Function
I don't know if I helped you but I didn't really understood what your problem was and you didn't even mentioned the language you use ^^ Good luck
Edit : and if you can't post images, that's because you don't have yet 10 points of reputation, you can get informations about reputation here : https://stackoverflow.com/help/whats-reputation, but you still can post the link of a picture it will allow users to click on it
I'm struggling with what seems to be an innocent block of code. When I type in "FILL", the intellisence recognizes the method and gives me a hint with 5 overloads and one of them (3rd) actually says it accepts dataset and a string as parameters. This is what I am passing and yet the compiler seems confused by something as it puts the red squiggly line right underneath the last statement in the code below (adapter.Fill(ds, "tbl");). Sometimes, I erase it and re-type it again and then it says "Error 1 The best overloaded method match for 'System.Data.Common.DbDataAdapter.Fill(int, int, params System.Data.DataTable[])' has some invalid arguments...." As if it's totally confused by which overloaded method of FILL I'm trying to use.
protected void ListBox1_SelectedIndexChanged(object sender, EventArgs e)
{
string strSQL = "SELECT * from myTableName";
using (SqlConnection cn = new SqlConnection("MyConnectionStrng"))
{
using (SqlDataAdapter adapter = new SqlDataAdapter(strSQL, cn))
{
DataSet ds = new DataSet();
adapter.Fill(ds, "tbl");
}
}
}
Anyone has an idea how to fix this?
TIA,
-Tony.
I am writing an app to read in a very large tif file (15000x10000), then chop it up into 256x256 tiles which get saved as jpegs. I am trying to use the WPF windows.media.imaging objects to do the chopping. The example below runs fine and will extract a number of tiles for me (it just gets multiple copies of the same tile for this example) but the app uses up memory and that memory never gets released. Even forcing a CG.Collect to test this still doesn't free up the memory.
Dim croppedImage As CroppedBitmap
Dim strImagePath As String = "C:\Huge.tif"
Dim imageSource As BitmapSource = TiffBitmapDecoder.Create(New Uri(strImagePath), BitmapCreateOptions.IgnoreImageCache, BitmapCacheOption.None).Frames(0) 'CreateImage(imageBytes, 0, 0)
Dim enc As JpegBitmapEncoder
Dim stream As FileStream
For i As Integer = 0 To 5000
croppedImage = New CroppedBitmap()
croppedImage.BeginInit()
croppedImage.Source = imageSource
croppedImage.SourceRect = New Int32Rect(0, 0, 256, 256)
croppedImage.EndInit()
enc = New JpegBitmapEncoder()
enc.QualityLevel = 70
enc.Frames.Add(BitmapFrame.Create(croppedImage))
stream = New FileStream("C:\output\" & i & ".jpg", FileMode.Create, FileAccess.Write)
enc.Save(stream)
stream.Close()
enc = Nothing
stream = Nothing
croppedImage.Source = Nothing
croppedImage = Nothing
Next
imageSource = Nothing
Am I missing something fundamental here? How can I ensure that these resources are released correctly?
Thanks
More Information:
The answers provided below definitely help. Thanks for that. I have a another issue to add to this now. I am trying to watermark each tile before it is save by adding the following code:
Dim targetVisual = New DrawingVisual()
Dim targetContext = targetVisual.RenderOpen()
targetContext.DrawImage(croppedImage, New Rect(0, 0, tileWidth, tileHeight))
targetContext.DrawImage(watermarkSource, New Rect(0, 0, 256, 256))
Dim target = New RenderTargetBitmap(tileWidth, tileHeight, 96, 96, PixelFormats.[Default])
targetContext.Close()
target.Render(targetVisual)
Dim targetFrame = BitmapFrame.Create(target)
This is starting to use some serious memory. Running through the large tif uses over 1200MB of memory as reported by task manager. It looks like this memory gets released eventually, but I am slightly concerned that something is not right with the code and is there anyway to stop it consuming all this memory in the first place. Perhaps this is simply down to the issue that Franci discussed?
Andrew
Your large object is definitely promoted to gen 2, after the ~25000 object allocations in the loop. Gen 2 objects are collected only on full collections, which are done rarely, thus your object might sit in memory for a while.
You can try forcing a full collection using GC.Collect(). You can also use the other GC methods to check for the allocated memory before you force the full allocation, then wait for it to finish and then check the allocated memory again to determine if indeed the large object was collected.
Note that even though a full collection might occur, the memory has already been included in the process working set, and it might take a bit for the OS to react to freeing that memory and remove it from the working set. This is important to keep in mind if you are trying to judge if the memory was properly collect by looking at the process working set in Task Manager.
I recreated your code locally and did a small experimentation, as a result I found that if you make the stream local to the for loop and use the enc.Save inside Using stream As New FileStream(System.IO.Path.Combine(outputDir, i.ToString() & ".jpg"), FileMode.Create, FileAccess.Write) the total private bytes stays approximately at 60 MB (against 80+ MB when not used inside using).
My complete code for your reference:
Dim fileName As String = System.IO.Path.Combine(Environment.CurrentDirectory, "Image1.tif")
Dim outputDir As String = System.IO.Path.Combine(Environment.CurrentDirectory, "Out")
Dim imageSource As BitmapSource = TiffBitmapDecoder.Create(New Uri(fileName), BitmapCreateOptions.IgnoreImageCache, BitmapCacheOption.None).Frames(0)
Dim enc As JpegBitmapEncoder
Dim croppedImage As CroppedBitmap
For i As Integer = 0 To 4999
croppedImage = New CroppedBitmap()
croppedImage.BeginInit()
croppedImage.Source = imageSource
croppedImage.SourceRect = New Int32Rect(0, 0, 256, 256)
croppedImage.EndInit()
enc = New JpegBitmapEncoder()
enc.QualityLevel = 70
enc.Frames.Add(BitmapFrame.Create(croppedImage))
Using stream As New FileStream(System.IO.Path.Combine(outputDir, i.ToString() & ".jpg"), FileMode.Create, FileAccess.Write)
enc.Save(stream)
End Using
enc = Nothing
croppedImage.Source = Nothing
croppedImage = Nothing
Next
imageSource = Nothing
try creating your TiffBitmaDecoder like so:
Dim imageSource As BitmapSource = TiffBitmapDecoder.Create(New Uri(fileName), BitmapCreateOptions.IgnoreImageCache, BitmapCacheOption.OnLoad).Frames(0)
Im reading a xlsx file as a db to do some work
i noticed that its reading some feilds in as int and date even though i just want it all to come in as text . is there anyway to override this feature?
Code below
(feel free to point out anything i could be doing better with my code as well)
private DataSet ExceltoDT(OpenFileDialog dialog)
{
try
{
string connst = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + dialog.FileName + ";Extended Properties=\"Excel 12.0 Xml;HDR=NO\";";
string sheet = "Sheet1$";
string strSQL = "SELECT * FROM [" + sheet + "]";
//string Table = "081710";
OleDbConnection xlsdb = new OleDbConnection(connst);
xlsdb.Open();
OleDbDataAdapter adp = new OleDbDataAdapter(strSQL, xlsdb);
DataSet ds2 = new DataSet();
adp.Fill(ds2);
adp.Dispose();
xlsdb.Close();
xlsdb.Dispose();
return ds2;
}
catch (StackOverflowException stack_ex2)
{
MessageBox.Show("(2007 Excel file) Stack Overflowed!" + "\n" + stack_ex2.Message);
return null;
}
catch (OleDbException ex_oledb2)
{
MessageBox.Show("An OleDb Error Thrown!" + "\n" + ex_oledb2.Message);
return null;
}
}
Add a ' (apostrophe) in front of every cell value. That will tell Excel "Treat this as text even when it looks like a number/date/whatever".
Not what you want? Then don't use the DB connector because it's badly broken. You'll notice that when you have a column with cells that are mixed. In that case, the DB driver will look at the first 8 rows and set the type to the majority of types it finds and return NULL for anything in that column that doesn't fit. You can fix that by hacking your registry.
Instead use the OLE API to open the Workbook and then start from there, reading row by row, converting the data as you need (this long list of posts should contain about every possible way to access Excel from C# plus all the bugs and problems you can encounter).