I apologize if I am missing a few things here, I am still learning .NET.
I have a project I have been working on where I am restoring a database as well as transaction logs. After each transaction log I have a PRINT statement to output that the transaction log has been processed.
I have been using the OnInfoMessage function to get the messages out of what is done in SQL Server, but was looking to ONLY extract the PRINT command out of it and display it in a textbox / label / RichTextBox as sort of a status box that updates as it goes.
My procedure in SQL Server is as follows
....
WHILE ##FETCH_STATUS=0
BEGIN
SET #sql = 'RESTORE LOG FCS FROM DISK = ''C:\Databases\'+#fileName +''' WITH NORECOVERY';
EXEC (#sql);
PRINT '===' +#sql;
FETCH NEXT FROM cFile INTO #fileName
END
CLOSE cFile
DEALLOCATE cFile
In VB.net I am able to get ALL messages to display in a log file.
Private Shared Sub OnInfoMessage(ByVal sender As Object, ByVal e As System.Data.SqlClient.SqlInfoMessageEventArgs)
Using LogFile As IO.StreamWriter = New IO.StreamWriter("C:\SDBT\worklog.ft", True)
LogFile.WriteLine(e.Message)
End Using
End Sub
This in turn gives me the following output...
Processed 10392 pages for database 'FCS', file 'FCS' on file 1.
Processed 2 pages for database 'FCS', file 'FCS_log' on file 1.
RESTORE DATABASE successfully processed 10394 pages in 2.652 seconds (30.619 MB/sec).
Processed 0 pages for database 'FCS', file 'FCS' on file 1.
Processed 6 pages for database 'FCS', file 'FCS_log' on file 1.
RESTORE LOG successfully processed 6 pages in 0.061 seconds (0.720 MB/sec).
===RESTORE LOG FCS FROM DISK = 'C:\SDBT_TestDB\log_00001.trn' WITH NORECOVERY
Processed 0 pages for database 'FCS', file 'FCS' on file 1.
Processed 2 pages for database 'FCS', file 'FCS_log' on file 1.
RESTORE LOG successfully processed 2 pages in 0.058 seconds (0.252 MB/sec).
===RESTORE LOG FCS FROM DISK = 'C:\SDBT_TestDB\log_00002.trn' WITH NORECOVERY
Ideally I would like to only retrieve the lines preceded with "===" and output those to the user.
Is it possible to get the #sql variable value from SQL Server to VB.net or is there something built into .NET that I am missing?
Thanks in advance!
In your OnInfoMessage sub, check each line of text in e.Message to see if it starts with === and only write those lines.
For Each line In e.Message.Split(vbNewLine)
If(line.StartsWith("===")) Then
LogFile.WriteLine(line)
End If
Next
Related
I am developing a service application (VB.NET) which pulls information from a source and imports it to a SQL Server database
The process can involve one or more “batches” of information at a time (the number and size of batches in any given “run” is arbitrary based on a queue maintained elsewhere)
Each batch is assigned an identifier (BatchID) so that the set of records in the staging table which belong to that batch can be easily identified
The ETL process for each batch is sequential in nature; the raw data is bulk inserted to a staging table and then a series of stored procedures perform updates on a number of columns until the data is ready for import
These stored procedures are called in sequence by the service and are generally simple UPDATE commands
Each SP takes the BatchID as an input parameter and specifies this as the criteria for inclusion in each UPDATE, á la :
UPDATE dbo.stgTable
SET FieldOne = (CASE
WHEN S.[FieldOne] IS NULL
THEN T1.FieldOne
ELSE
S.[FieldOne]
END
)
, FieldTwo = (CASE
WHEN S.[FieldTwo] IS NULL
THEN T2.FieldTwo
ELSE
S.[FieldTwo]
END
)
FROM dbo.stgTable AS S
LEFT JOIN dbo.someTable T1 ON S.[SomeField] = T1.[SomeField]
LEFT JOIN dbo.someOtherTable T2 ON S.[SomeOtherField] = T2.[SomeOtherField]
WHERE S.BatchID = #BatchID
Some of the SP’s also refer to functions (both scalar and table-valued) and all incorporate a TRY / CATCH structure so I can tell from the output parameters if a particular SP has failed
The final SP is a MERGE operation to move the enriched data from the staging table into the production table (again, specific to the provided BatchID)
I would like to thread this process in the service so that a large batch doesn’t hold up smaller batches in the same run
I figured there should be no issue with this as no thread could ever attempt to process records in the staging table that could be targeted by another thread (no race conditions)
However, I’ve noticed that, when I do thread the process, arbitrary steps on arbitrary batches seem to fail (but no error is recorded from the output of the SP)
The failures are inconsistent; e.g. sometimes batches 2, 3 & 5 will fail (on SP’s 3, 5 & 7 respectively), other times it will be different batches, each at different steps in the sequence
When I import the batches sequentially, they all import perfectly fine – always!
I can’t figure out if this is an issue on the service side (VB.NET) – e.g. is each thread opening an independent connection to the DB or could they be sharing the same one (I’ve set it up that each one should be independent…)
Or if the issue is on the SQL Server side – e.g. is it not feasible for concurrent SP calls to manipulate data on the same table, even though, as described above, no thread/batch will ever touch records belonging to another thread/batch
(On this point – I tried using CTE’s to create subsets of data from the staging table based on the BatchID and apply the UPDATE’s to those instead but the exact same behaviour occurred)
WITH CTE AS (
SELECT *
FROM dbo.stgTable
WHERE BatchID = #BatchID
)
UPDATE CTE...
Or maybe the problem is that multiple SP’s are calling the same function at the same time and that is why one or more of them are failing (I don’t see why that would be a problem though?)
Any suggestions would be very gratefully received – I’ve been playing around with this all week and I can’t for the life of me determine precisely what the problem might be!
Update to include sample service code
This is the code in the service class where the threading is initiated
For Each ItemInScope In ScopedItems
With ItemInScope
_batches(_batchCount) = New Batch(.Parameter1, .Parameter2, .ParameterX)
With _batches(_batchCount)
If .Initiate() Then
_doneEvents(_batchCount) = New ManualResetEvent(False)
Dim _batchWriter = New BatchWriter(_batches(_batchCount), _doneEvents(_batchCount))
ThreadPool.QueueUserWorkItem(AddressOf _batchWriter.ThreadPoolCallBack, _batchCount)
Else
_doneEvents(_batchCount) = New ManualResetEvent(True)
End If
End With
End With
_batchCount += 1
Next
WaitHandle.WaitAll(_doneEvents)
Here is the BatchWriter class
Public Class BatchWriter
Private _batch As Batch
Private _doneEvent As ManualResetEvent
Public Sub New(ByRef batch As Batch, ByVal doneEvent As ManualResetEvent)
_batch = batch
_doneEvent = doneEvent
End Sub
Public Sub ThreadPoolCallBack(ByVal threadContext As Object)
Dim threadIndex As Integer = CType(threadContext, Integer)
With _batch
If .PrepareBatch() Then
If .WriteTextOutput() Then
.ProcessBatch()
End If
End If
End With
_doneEvent.Set()
End Sub
End Class
The PrepareBatch and WriteTextOutput functions of the Batch class are entirely contained within the service application - it is only the ProcessBatch function where the service starts to interact with the database (via Entity Framework)
Here is that function
Public Sub ProcessScan()
' Confirm that a file is ready for import
If My.Computer.FileSystem.FileExists(_filePath) Then
Dim dbModel As New DatabaseModel
With dbModel
' Pass the batch to the staging table in the database
If .StageBatch(_batchID, _filePath) Then
' First update (results recorded for event log)
If .UpdateOne(_batchID) Then
_stepOneUpdates = .RetUpdates.Value
' Second update (results recorded for event log)
If .UpdateTwo(_batchID) Then
_stepTwoUpdates = .RetUpdates.Value
' Third update (results recorded for event log)
If .UpdateThree(_batchID) Then
_stepThreeUpdates = .RetUpdates.Value
....
End Sub
Im making a program with visual basic 2010 and using sqlserver compact as database. I have two folders named "Year2015" and "Year2016". The folders are in the same location where the program is. Both folders has a database named "MyData.sdf" in themselves. Both of the "MyData.sdf" has the same tables etc. Im trying to do something like that:
When user select "Year2015", program starts to run with the data of "MyData.sdf" that is in the folder "Year2015" and when user select "Year2016", program starts to run with the data of "MyData.sdf" that is in the folder "Year2016". I mean that i want to change the datasource address programmaticly. Searched net for that. There are some explanations but no codes i could find. If this is a bad question sorry for that.
Dave Pinal is a genius at this stuff, and I happened to read his blog on this very topic:
ALTER DATABASE TestDB
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE;
GO
-- Detach DB
EXEC MASTER.dbo.sp_detach_db #dbname = N'TestDB'
GO
-- Move MDF File from Loc1 to Loc 2
-- Re-Attached DB
CREATE DATABASE [TestDB] ON
( FILENAME = N'F:\loc2\TestDB.mdf' ),
( FILENAME = N'F:\loc2\TestDB_log.ldf' )
FOR ATTACH
GO
Note: even his comment sections are great, too!
*SOURCE SCRIPT CAME FROM PINAL.
http://blog.sqlauthority.com/2012/10/28/sql-server-move-database-files-mdf-and-ldf-to-another-location/
I finally create my own code for this issue. I want to share it for the people who uses VB2010 and SQL Server Compact and want to change the data source for the active form. The code is:
Dim sConnectionString As String
sConnectionString = "Data Source=" & My.Computer.FileSystem.CurrentDirectory & "\Year2015\MyData.sdf"
TableAdapterManager.Connection.ConnectionString = sConnectionString
This will change your data source of active form. The other forms continues to use the default source
Hi, everybody.
I am new to PL/SQL and Oracle Databases.
I need to read/write file that exists on server so i'm using utl_file.fopen('/home/tmp/','text.txt','R') but Oracle shows error 'invalid directory path'.
Main problem is that i have only user privileges, so i cant use commands like create directory user_dir as '/home/temp/' or view utl_file_dir with just show parameter utl_file_dir;
I used this code to view utl_file_dir:
SQL> set serveroutput on;
SQL> Declare
2 Intval number;
3 Strval varchar2 (500);
4 Begin
5 If (dbms_utility.get_parameter_value('utl_file_dir', intval,strval)=0)
6 Then dbms_output.put_line('value ='||intval);
7 Else dbms_output.put_line('value = '||strval);
8 End if;
9 End;
10 /
and output was 'value = 0'.
I google'd much but didnt find any solution of this problem, so i'm asking help here.
To read file i used this code:
declare
f utl_file.file_type;
s varchar2(200);
begin
f := utl_file.fopen('/home/tmp/','text.txt','R');
loop
utl_file.get_line(f,s);
dbms_output.put_line(s);
end loop;
exception
when NO_DATA_FOUND then
utl_file.fclose(f);
end;
If you do not have permission to create the directory object (and assuming that the directory object does not already exist), you'll need to send a request to your DBA (or someone else that has the appropriate privileges) in order to create a directory for you and to grant you access to that directory.
utl_file_dir is an obsolete parameter that is much less flexible than directory objects and requires a reboot of the database to change-- unless you're using Oracle 8.1.x or you are dealing with a legacy process that was written back in the 8.1.x days and hasn't been updated to use directories, you ought to ignore it.
I have to store multiple files in a Database. In order to do that I am using jQuery Multiple File Uplaod control written by blueimp. For the server language I use asp.net. It uses a handler – ashx which accepts every particular file and stores it in the DB.
In my DB I have a stored procedure which returns the document id of the currently uploaded file and then stores it into the DB.
This is a code fragment from it which shows that getting this id is a 3 step procedure:
SELECT TOP 1 #docIdOut = tblFile.docId --
,#creator=creator,#created=created FROM [tblFile] WHERE
friendlyname LIKE #friendlyname AND #categoryId = categoryId AND
#objectId = objectId AND #objectTypeId = objectTypeId ORDER BY
tblFile.version DESC
IF #docIdOut = 0 BEGIN --get the next docId SELECT TOP 1
#docIdOut = docId + 1 FROM [tblFile] ORDER BY docId DESC END
IF #docIdOut = 0 BEGIN --set to 1 SET #docIdOut = 1 END
If more than one calls to that stored procedure are executed then will be a problem due to inconsistency of the data, but If I add a transaction then the upload for some files will be cancelled.
https://dl.dropboxusercontent.com/u/13878369/2013-05-20_1731.png
Is there a way to call the stored procedure again with the same parameters when the execution is blocked by transaction?
Another option is to use the blueimp plugin synchronously with the option “sequentialUploads = true”
This option works for all browsers except firefox, and I read that it doesn’t work for Safari as well.
Any suggestions would be helpful. I tried and enabling the option for selecting only one file at a time, which is not the best, but will stop the DB problems (strored procedure) and will save the files correctly.
Thanks,
Best Regards,
Lyuubomir
set singleFileUploads: true, sequentialUploads: false in the main.js.
singleFileUploads: true means each file of a selection is uploaded using an individual request for XHR type uploads. Then you can get information of an individual file and call the store procedure with the information you have just got.
The output of sql commands that is visible to users who interactively run SQL commands from SQL Server Management studio, is different than the output you get back from executing an ADO command or ADO query object.
USE [DBNAME]
BACKUP DATABASE [DBNAME] TO
DISK = 'C:\SqlBackup\Backup.mdf'
The successful completion output is like this:
Processed 465200 pages for database 'DBNAME', file 'filename' on file 2.
Processed 2 pages for database 'DBNAME', file 'filename_log' on file 2.
BACKUP DATABASE successfully processed 465202 pages in 90.595 seconds (40.116 MB/sec).
When I execute either a TADOCommand or TADOQuery with the CommandText or SQL set as above, I do not get any such output. How do I read this "secondary output" from the execution of an SQL command? I'm hoping that perhaps via some raw ADO operations I might be able to execute a command and get back the information above, for success, as well as any errors in performing an Sql backup.
Update: The answer below works better for me than my naive attempt, which did not work, using plain Delphi TADOCommand and TADOConnection classes:
create TADOCommand and TADOConnection.
execute command.
get info-messages back.
The problem I experienced in my own coding attempts, is that my first command is "use dbname" and the only recordset I traversed in my code, was the results of the "use dbname" command, not the second command I was executing. The accepted answer below traverses all recordsets that come back from executing the ADO command, and thus it works much better. Since I'm doing all this in a background thread, I actually think it's better to create the raw Com Objects anyways, and avoid any VCL entanglement in my thread. The code below could be a nice component if anybody is interested, let me know and I might make an open source "SQL Backup for Delphi" component.
Here is an example. I've tested it with D7 and MSSQL2000. And it adds to Memo1 all messages from server:
29 percent backed up.
58 percent backed up.
82 percent backed up.
98 percent backed up.
Processed 408 pages for database 'NorthWind', file 'Northwind' on file 1.
100 percent backed up.
Processed 1 pages for database 'NorthWind', file 'Northwind_log' on file 1.
BACKUP DATABASE successfully processed 409 pages in 0.124 seconds (26.962 MB/sec).
Also if it takes a long time consider to implement a WHILE loop not in the main thread.
uses AdoInt,ComObj;
.....
procedure TForm1.Button1Click(Sender: TObject);
var cmd : _Command;
Conn : _Connection;
RA : OleVariant;
rs :_RecordSet;
n : Integer;
begin
Memo1.Clear;
Conn := CreateComObject(CLASS_Connection) as _Connection;
Conn.ConnectionString := 'Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=NorthWind;Data Source=SQL_Server';
Conn.Open(Conn.ConnectionString,'','',Integer(adConnectUnspecified));
cmd := CreateComObject(CLASS_Command) as _Command;
cmd.CommandType := adCmdText;
cmd.Set_ActiveConnection(Conn);
cmd.CommandText := 'BACKUP DATABASE [NorthWind] TO DISK = N''c:\sql_backup\NorthWind'' WITH INIT , NOUNLOAD , NAME = N''NortWind backup'', NOSKIP , STATS = 10, NOFORMAT;';
rs:=cmd.Execute(RA,0,Integer(adCmdText));
while (rs<>nil) do
begin
for n:=0 to(Conn.Errors.Count-1)do begin
Memo1.Lines.Add(Conn.Errors.Item[n].Description);
end;
rs:=rs.NextRecordset(RA);
end;
cmd.Set_ActiveConnection(nil);
Conn.Close;
cmd := nil;
Conn := nil;
end;
I've found this thread (Russian) for stored procedure and correct it for BACKUP command.