I have ASP.NET MVC application and I'm using a SqlDataAdapter for handling sql server methods. The method below returns nothing when an exception occurs, so I don't have any information when there is an error with a sql server command.
public int ExecuteNonQuery(string pSql, List<SqlParameter> prm)
{
SqlCommand cmd = new SqlCommand(pSql, sqlConnection);
int result = 0;
if (sqlConnection.State == ConnectionState.Closed)
{
sqlConnection.Open();
}
try
{
cmd.Parameters.AddRange(prm.ToArray());
result = cmd.ExecuteNonQuery();
return result;
}
catch (Exception ex)
{
return 0;
}
finally
{
sqlConnection.Close();
}
}
How can I asynchronously log exceptions without disrupting the user experience?Additionally, I want to log it to database and if there is a timeout exception then logging this timeout can get timeout again when logging.
There is no SqlDataAdapter code in your sample. That being said, there are three DataAdapter events that you can use to respond to changes made to data at the data source.
RowUpdating,
RowUpdated, &
FillError
There is also a:
Status property to determine if an error has occurred during the
operation and, if desired, to control the actions against the current
and resulting rows.
You can asynchronously log exceptions in those events when there is an error.
Examples from MSDN:
protected static void OnRowUpdated(
object sender, SqlRowUpdatedEventArgs args)
{
if (args.Status == UpdateStatus.ErrorsOccurred)
{
args.Row.RowError = args.Errors.Message;
args.Status = UpdateStatus.SkipCurrentRow;
Task.Run(() => LogError());
}
}
protected static void FillError(object sender, FillErrorEventArgs args)
{
if (args.Errors.GetType() == typeof(System.OverflowException))
{
// Code to handle precision loss.
//Add a row to table using the values from the first two columns.
DataRow myRow = args.DataTable.Rows.Add(new object[]
{args.Values[0], args.Values[1], DBNull.Value});
//Set the RowError containing the value for the third column.
args.RowError =
"OverflowException Encountered. Value from data source: " +
args.Values[2];
args.Continue = true;
Task.Run(() => LogError());
}
}
private void LogError()
{
// logging code
}
Related
Having a spout which on each tick goes to Postgre database and reads an additional row. The spout code looks as follows:
class RawDataLevelSpout extends BaseRichSpout implements Serializable {
private int counter;
SpoutOutputCollector collector;
#Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("col1", "col2"));
}
#Override
public void open(Map map, TopologyContext context, SpoutOutputCollector spoutOutputCollector) {
collector = spoutOutputCollector;
}
private Connection initializeDatabaseConnection() {
try {
Class.forName("org.postgresql.Driver");
Connection connection = null;
connection = DriverManager.getConnection(
DATABASE_URI,"root", "root");
return connection;
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return null;
}
#Override
public void close() {
}
#Override
public void nextTuple() {
List<String> values = new ArrayList<>();
PreparedStatement statement = null;
try {
Connection connection = initializeDatabaseConnection();
statement = connection.prepareStatement("SELECT * FROM table1 ORDER BY col1 LIMIT 1 OFFSET ?");
statement.setInt(1, counter++);
ResultSet resultSet = statement.executeQuery();
resultSet.next();
ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
int totalColumns = resultSetMetaData.getColumnCount();
for (int i = 1; i <= totalColumns; i++) {
String value = resultSet.getString(i);
values.add(value);
}
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
collector.emit(new Values(values.stream().toArray(String[]::new)));
}
}
What is the standard way how to approach connection pooling in Spouts in apache storm? Furthermore, is it possible to somehow synchronize the coutner variable accross multiple running instances within the cluster topology?
Regarding connection pooling, you could pool connections via static variable if you wanted, but since you aren't guaranteed to have all spout instances running in the same JVM, I don't think there's any point.
No, there is no way to synchronize the counter. The spout instances may be running on different JVMs, and you don't want them all blocking while the spouts agree what the counter value is. I don't think your spout implementation makes sense though. If you wanted to just read one row at a time, why would you not just run a single spout instance instead of trying to synchronize multiple spouts?
You seem to be trying to use your relational database as a queue system, which is probably a bad fit. Consider e.g. Kafka instead. I think you should be able to use either one of https://www.confluent.io/product/connectors/ or http://debezium.io/ to stream data from your Postgres to Kafka.
I have an winform application which consumes windows service, i user ChannelFactory
to connect to service, problem is when i call service method using channel the memory usage increase and after
method execute memory not go down(even after form close), i call GC.Collect but no change
channel Create class
public class Channel1
{
List<ChannelFactory> chanelList = new List<ChannelFactory>();
ISales salesObj;
public ISales Sales
{
get
{
if (salesObj == null)
{
ChannelFactory<ISales> saleschannel = new ChannelFactory<ISales>("SalesEndPoint");
chanelList.Add(saleschannel);
salesObj = saleschannel.CreateChannel();
}
return salesObj;
}
}
public void CloseAllChannels()
{
foreach (ChannelFactory chFac in chanelList)
{
chFac.Abort();
((IDisposable)chFac).Dispose();
}
salesObj = null;
}
}
base class
public class Base:Form
{
public Channel1 channelService = new Channel1();
public Channel1 CHANNEL
{
get
{
return channelService;
}
}
}
winform class
Form1:Base
private void btnView_Click(object sender, EventArgs e)
{
DataTable _dt = new DataTable();
try
{
gvAccounts.AutoGenerateColumns = false;
_dt = CHANNEL.Sales.GetDatatable();
gvAccounts.DataSource = _dt;
}
catch (Exception ex)
{
MessageBox.Show("Error Occurred while processing...\n" + ex.Message, "Warning", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
finally
{
CHANNEL.CloseAllChannels();
_dt.Dispose();
//GC.Collect();
}
}
You're on the right track in terms of using ChannelFactory<T>, but your implementation is a bit off.
ChannelFactory<T> creates a factory for generating channels of type T. This is a relatively expensive operation (as compared to just creating a channel from the existing factory), and is generally done once per life of the application (usually at start). You can then use that factory instance to create as many channels as your application needs.
Generally, once I've created the factory and cached it, when I need to make a call to the service I get a channel from the factory, make the call, and then close/abort the channel.
Using your posted code as a starting point, I would do something like this:
public class Channel1
{
ChannelFactory<ISales> salesChannel;
public ISales Sales
{
get
{
if (salesChannel == null)
{
salesChannel = new ChannelFactory<ISales>("SalesEndPoint");
}
return salesChannel.CreateChannel();
}
}
}
Note that I've replaced the salesObj with salesChannel (the factory). This will create the factory the first time it's called, and create a new channel from the factory every time.
Unless you have a particular requirement to do so, I wouldn't keep track of the different channels, especially if follow the open/do method/close approach.
In your form, it'd look something like this:
private void btnView_Click(object sender, EventArgs e)
{
DataTable _dt = new DataTable();
try
{
gvAccounts.AutoGenerateColumns = false;
ISales client = CHANNEL.Sales
_dt = client.GetDatatable();
gvAccounts.DataSource = _dt;
((ICommunicationObject)client).Close();
}
catch (Exception ex)
{
((ICommunicationObject)client).Abort();
MessageBox.Show("Error Occurred while processing...\n" + ex.Message, "Warning", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
}
The code above gets a new ISales channel from the factory in CHANNEL, executes the call, and then closes the channel. If an exception happens, the channel is aborted in the catch block.
I would avoid using Dispose() out of the box on the channels, as the implementation in the framework is flawed and will throw an error if the channel is in a faulted state. If you really want to use Dispose() and force the garbage collection, you can - but you'll have to work around the WCF dispose issue. Google will give you a number of workarounds (google WCF Using for a start).
I have a code like the following (I have stripped some code for readability)
private void RicercaArticoloStripped(object sender, DoWorkEventArgs e)
{
try
{
using (var context = new ControlloSchedeLocalEntities())
{
var prodotti = context.VProdotti.Where(i => i.WACMAT == textBoxCodiceArticolo.Text);
if (prodotti.Count() > 0)
{
this.Dispatcher.Invoke(System.Windows.Threading.DispatcherPriority.Normal, new Action(delegate()
{
textBlockDescrizioneArticolo.Text = prodotti.FirstOrDefault().WADESC;
}));
}
}
}
catch (Exception ex)
{
MessageBox.Show("Error:\n\n" + ex.Message + "\r\nStack: " + ex.ToString());
}
}
private void textBoxCodiceArticolo_KeyUpStripped(object sender, KeyEventArgs e)
{
if (e.Key == Key.Enter)
{
worker = new BackgroundWorker();
worker.WorkerReportsProgress = true;
worker.DoWork += new DoWorkEventHandler(RicercaArticoloStripped);
worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(worker_RicercaArticoloWorkerCompleted);
worker.ProgressChanged += new ProgressChangedEventHandler(worker_ProgressChanged);
object[] parameters = new object[] { textBoxCodiceArticolo.Text, textBlockDescrizioneArticolo.Text };
worker.RunWorkerAsync(parameters);
}
}
So, as for my understanding of BackgroundWorker, when I instantiate the:
var prodotti = context.VProdotti.Where(i => i.WACMAT == textBoxCodiceArticolo.Text);
I am doing it FROM the working thread, not the UI thread.
But I get an exception on the line immediately below, when I try to access the value:
if (prodotti.Count() > 0)
I get the (in)famous error:
"The calling thread cannot access this object because a different thread owns it"
Why?
As you already said you must use Dispatcher.BeginInvoke/Invoke to perform operations from the control's owner thread or you will get "The calling thread cannot access this object because a different thread owns it" exception. Thats why you got this exception;
And here is why you got this exception on the line below (when prodotti.Count() was called):
When you create prodotti variable it's just a IEnumerable<T> object. So he actually calculates only when you call for prodotti.Count() method and thats why you got exception on this line.
IEnumerable actually is generator, that means that he will produce new set of objects every time he used.
To test this you can calculate prodotti as shown bellow:
var prodotti = context.VProdotti.Where(i => i.WACMAT == textBoxCodiceArticolo.Text).ToList();
In this case you will get exception immediately because .ToList() forces all calculations.
Check this article for generators and enumerators: http://www.codeproject.com/Articles/155462/IEnumerable-Lazy-and-Dangerous
Updated:
really, when you use IEnumerable with reference types you can get the same objects as previously. Read this answer for more: https://stackoverflow.com/a/14361094/1467309
The property Text of TextBox gets the value of the DependencyProperty Text, this can only be done from the UI-thread. You are accessing textBoxCodiceArticolo.Text from your Worker thread.
I've just started using Dapper for a project, having mostly used ORMs like NHibernate and EF for the past few years.
Typically in our web applications we implement session per request, beginning a transaction at the start of the request and committing it at the end.
Should we do something similar when working directly with SqlConnection / System.Transactions?
How does StackOverflow do it?
Solution
Taking the advice of both #gbn and #Sam Safron I'm not using transactions. In my case I'm only doing read queries so it seems there is no real requirement to use transactions (contrary to what I've been told about implicit transactions).
I create a lightweight session interface so that I can use a connection per request. This is quite beneficial to me as with Dapper I often need to create a few different queries to build up an object and would rather share the same connection.
The work of scoping the connection per request and disposing it is done by my IoC container (StructureMap):
public interface ISession : IDisposable {
IDbConnection Connection { get; }
}
public class DbSession : ISession {
private static readonly object #lock = new object();
private readonly ILogger logger;
private readonly string connectionString;
private IDbConnection cn;
public DbSession(string connectionString, ILogger logger) {
this.connectionString = connectionString;
this.logger = logger;
}
public IDbConnection Connection { get { return GetConnection(); } }
private IDbConnection GetConnection() {
if (cn == null) {
lock (#lock) {
if (cn == null) {
logger.Debug("Creating Connection");
cn = new SqlConnection(connectionString);
cn.Open();
logger.Debug("Opened Connection");
}
}
}
return cn;
}
public void Dispose() {
if (cn != null) {
logger.Debug("Disposing connection (current state '{0}')", cn.State);
cn.Dispose();
}
}
}
This is what we do:
We define a static called DB on an object called Current
public static DBContext DB
{
var result = GetContextItem<T>(itemKey);
if (result == null)
{
result = InstantiateDB();
SetContextItem(itemKey, result);
}
return result;
}
public static T GetContextItem<T>(string itemKey, bool strict = true)
{
#if DEBUG // HttpContext is null for unit test calls, which are only done in DEBUG
if (Context == null)
{
var result = CallContext.GetData(itemKey);
return result != null ? (T)result : default(T);
}
else
{
#endif
var ctx = HttpContext.Current;
if (ctx == null)
{
if (strict) throw new InvalidOperationException("GetContextItem without a context");
return default(T);
}
else
{
var result = ctx.Items[itemKey];
return result != null ? (T)result : default(T);
}
#if DEBUG
}
#endif
}
public static void SetContextItem(string itemKey, object item)
{
#if DEBUG // HttpContext is null for unit test calls, which are only done in DEBUG
if (Context == null)
{
CallContext.SetData(itemKey, item);
}
else
{
#endif
HttpContext.Current.Items[itemKey] = item;
#if DEBUG
}
#endif
}
In our case InstantiateDB returns an L2S context, however in your case it could be an open SQLConnection or whatever.
On our application object we ensure that our connection is closed at the end of the request.
protected void Application_EndRequest(object sender, EventArgs e)
{
Current.DisposeDB(); // closes connection, clears context
}
Then anywhere in your code where you need access to the db you simple call Current.DB and stuff automatically works. This is also unit test friendly due to all the #if DEBUG stuff.
We do not start any transactions per session, if we did and had updates at the beginning of our session, we would get serious locking issues, as the locks would not be released till the end.
You'd only start a SQL Server Transaction when you need to with something like TransactionScope when you call the database with a "write" call.
See a random example in this recent question: Why is a nested transaction committed even if TransactionScope.Complete() is never called?
You would not open a connection and start a transaction per http request. Only on demand. I'm having difficulty understanding why some folk advocate opening a database transaction per session: sheer idiocy when you look at what a database transaction is
Note: I'm not against the pattern per se. I am against unnecessary, too long, client-side database transactions that invoke MSDTC
I'm trying to use hibernate #Entity with java.sql.Blob to store some binary data. Storing doesn't throw any exceptions (however, I'm not sure if it really stores the bytes), but reading does. Here is my test:
#Test
public void shouldStoreBlob() {
InputStream readFile = getClass().getResourceAsStream("myfile");
Blob blob = dao.createBlob(readFile, readFile.available());
Ent ent = new Ent();
ent.setBlob(blob);
em.persist(ent);
long id = ent.getId();
Ent fromDb = em.find(Ent.class, id);
//Exception is thrown from getBinaryStream()
byte[] fromDbBytes = IOUtils.toByteArray(fromDb.getBlob().getBinaryStream());
}
So it throws an exception:
java.sql.SQLException: could not reset reader
at org.hibernate.engine.jdbc.BlobProxy.getStream(BlobProxy.java:86)
at org.hibernate.engine.jdbc.BlobProxy.invoke(BlobProxy.java:108)
at $Proxy81.getBinaryStream(Unknown Source)
...
Why? Shouldn't it read bytes form DB here? And what can I do for it to work?
Try to refresh entity:
em.refresh(fromDb);
Stream will be reopened. I suspect that find(...) is closing the blob stream.
It is not at all clear how you are using JPA here, but certainly you do not need to deal with Blob data type directly if you are using JPA.
You just need to declare a field in the entity in question of #Lob somewhat like this:
#Lob
#Basic(fetch = LAZY)
#Column(name = "image")
private byte[] image;
Then, when you retrieve your entity, the bytes will be read back again in the field and you will be able to put them in a stream and do whatever you want with them.
Of course you will need a getter and setter methods in your entity to do the byte conversion. In the example above it would be somewhat like:
private Image getImage() {
Image result = null;
if (this.image != null && this.image.length > 0) {
result = new ImageIcon(this.image).getImage();
}
return result;
}
And the setter somewhat like this
private void setImage(Image source) {
BufferedImage buffered = new BufferedImage(source.getWidth(null), source.getHeight(null), BufferedImage.TYPE_INT_RGB);
Graphics2D g = buffered.createGraphics();
g.drawImage(source, 0, 0, null);
g.dispose();
ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
ImageIO.write(buffered, "JPEG", stream);
this.image = stream.toByteArray();
}
catch (IOException e) {
assert (false); // should never happen
}
}
}
You need to set a breakpoint on method org.hibernate.engine.jdbc.BlobProxy#getStream on line stream.reset() and examine a reason of IOException:
private InputStream getStream() throws SQLException {
try {
if (needsReset) {
stream.reset(); // <---- Set breakpoint here
}
}
catch ( IOException ioe) {
throw new SQLException("could not reset reader");
}
needsReset = true;
return stream;
}
In my case the reason of IOException was in usage of org.apache.commons.io.input.AutoCloseInputStream as a source for Blob:
InputStream content = new AutoCloseInputStream(stream);
...
Ent ent = new Ent();
...
Blob blob = Hibernate.getLobCreator(getSession()).createBlob(content, file.getFileSize())
ent.setBlob(blob);
em.persist(ent);
While flushing a Session hibernate closes Inpustream content (or rather org.postgresql.jdbc2.AbstractJdbc2Statement#setBlob closes Inpustream in my case). And when AutoCloseInputStream is closed - it rases an IOException in method reset()
update
In your case you use a FileInputStream - this stream also throws an exception on reset method.
There is a problem in test case. You create blob and read it from database inside one transaction. When you create Ent, Postgres jdbc driver closes InputStream while flushing a session. When you load Ent (em.find(Ent.class, id)) - you get the same BlobProxy object, that stores already closed InputStream.
Try this:
TransactionTemplate tt;
#Test
public void shouldStoreBlob() {
final long id = tt.execute(new TransactionCallback<long>()
{
#Override
public long doInTransaction(TransactionStatus status)
{
try
{
InputStream readFile = getClass().getResourceAsStream("myfile");
Blob blob = dao.createBlob(readFile, readFile.available());
Ent ent = new Ent();
ent.setBlob(blob);
em.persist(ent);
return ent.getId();
}
catch (Exception e)
{
return 0;
}
}
});
byte[] fromStorage = tt.execute(new TransactionCallback<byte[]>()
{
#Override
public byte[] doInTransaction(TransactionStatus status)
{
Ent fromDb = em.find(Ent.class, id);
try
{
return IOUtils.toByteArray(fromDb.getBlob().getBinaryStream());
}
catch (IOException e)
{
return new byte[] {};
}
}
});
}
My current and only solution is closing the write session and opening new Hibernate session to get back the streamed data. It works. However I do not know what is the difference. I called inputStream.close(), but that was not enough.
Another way:
I tried to call free() method of blob after session.save(attachment) call too, but it throws another exception:
Exception in thread "main" java.lang.AbstractMethodError: org.hibernate.lob.SerializableBlob.free()V
at my.hibernatetest.HibernateTestBLOB.storeStreamInDatabase(HibernateTestBLOB.java:142)
at my.hibernatetest.HibernateTestBLOB.main(HibernateTestBLOB.java:60)
I am using PostgreSQL 8.4 + postgresql-8.4-702.jdbc4.jar, Hibernate 3.3.1.GA
Is the method IOUtils.toByteArray closing the input stream?