First off I would like to say that I think that my question may be time consuming for people to solve it in a complete sense so I understand that it is totally possible that the complete answer is asking for just way too much, so anything to help me better understand, like: reading material, examples, links, and/or advice would be great and I do very much appreciate any and every comment I receive, good or bad it just makes me and this place alot better, finally, I would like to thank you all so much for everything that you do here, this is truly a place that was build by smart people, and people that care.
MY QUESTION
(using Classic ASP and SQL Server)
I know that it is possible to read a remote XML file and then insert it into a SQL Server database table. I found one example that does that it uses Classic ASP and MS Access but that can be changed to SQL Server with minimal coding the URL is: http://forums.aspfree.com/code-bank-54/classic-asp-import-remote-xml-file-into-database-152966.html
But the problem is that I cannot get it to work on my remote XML file, I tried to edit the classic asp code (found in the link above) for days and days and I just cannot get it to work the way I would like.
So I want to start from the beginning,
The XML file in question is located on: http://www.iftach.org/taxmatrix/charts/2Q2012.xml
I seen a couple of examples on how you can export the entire xml file into the SQL Server database (like do a BULK insert, see URL: http://support.microsoft.com/kb/316005) and also on how to extract some info. from the XML but my request is kind of odd since I want to check for the Country first and then get that Counties Rate only and not the other one, I want to do this for the entire xml file.
For example the xml file is something like this:
(or you can view the full xml file by clicking on the URL above)
<FILE>
<QUARTER>2Q2012</QUARTER>
<RECORD>
<JURISDICTION ID="#15">AB</JURISDICTION>
<COUNTRY>CAN</COUNTRY>
......
......
<RATE RATECHANGE="0" COUNTRY="US">0.3366</RATE>
<RATE RATECHANGE="0" COUNTRY="CAN">0.0900</RATE>
......
......
......
</RECORD>
<RECORD>
<JURISDICTION ID="#15">FL</JURISDICTION>
<COUNTRY>U.S.</COUNTRY>
......
......
<RATE RATECHANGE="0" COUNTRY="US">1.5700</RATE>
<RATE RATECHANGE="0" COUNTRY="CAN">1.3210</RATE>
......
......
......
</RECORD>
</FILE>
and so on....
Now I would like to insert that info into the SQL Server table called FFTR and name the column specific for each JURISDICTION
Like for example the above would be:
Field Name 1 --> "JURISDICTION_AB_CAN"
Field Name 2 --> "JURISDICTION_FL_US"
and so on...
NOTE:
The prefix JURISDICTION_ will always be the same only the two letters will change and the CAN can become US.
Another thing is if the COUNTRY is "CAN" then I would like to use the CAN Rate and if it's U.S. I would like to use the US Rate and insert that info. into the database with the Field named "RATE". (The Rate will always be 4 decimal places) the Rate I want is only under: <FUEL_TYPE>Special Diesel</FUEL_TYPE> I don't need the other Rates.
And the last thing I would like to do is to have the <QUARTER>2Q2012</QUARTER> inserted into a Field named "Quarter"
So the final SQL Server database would look like this (using the 2 records as an example above)
Field Name: JURISDICTION_AB_CAN
Rate: 0.0900
Quarter: 2Q2012
Field Name: JURISDICTION_FL_US
Rate: 1.5700
Quarter: 2Q2012
So what I tried to do is this (see code below) and I got it to show each line but it doesn't even come close to a solution:
<%
Option Explicit
Response.Buffer = True
Dim xml
Set xml = Server.CreateObject("Microsoft.XMLDOM")
xml.async = False
xml.setProperty "ServerHTTPRequest", True
xml.Load ("http://www.iftach.org/TaxMatrix/charts/2Q2012.xml")
Dim paragraph1,paragraph2,paragraph3,paragraph4,paragraph5,paragraph6
paragraph1 = xml.documentElement.childNodes(1).text
paragraph2 = xml.documentElement.childNodes(2).text
paragraph3 = xml.documentElement.childNodes(3).text
paragraph4 = xml.documentElement.childNodes(4).text
paragraph5 = xml.documentElement.childNodes(5).text
paragraph6 = xml.documentElement.childNodes(6).text
Set xml = Nothing
%>
<html>
<head>
<title></title>
</head>
<body>
<p align="center"><% = paragraph1 %></p>
<p align="center"><% = paragraph2 %></p>
<p align="center"><% = paragraph3 %></p>
<p align="center"><% = paragraph4 %></p>
<p align="center"><% = paragraph5 %></p>
<p align="center"><% = paragraph6 %></p>
</body>
</html>
I even think that adding it to a ADODB Recordset would be great and then I would insert it into SQL Server one by one or just loop it all in there, but it only shows me the columns I need the rows in there also. See code below:
<%
Dim objRS
Set objRS = Server.CreateObject("ADODB.Recordset")
objRS.ActiveConnection = "Provider=MSDAOSP; Data Source=MSXML2.DSOControl.3.0;"
objRS.Open(Server.MapPath("2Q2012.xml"))
Response.Write(objRS.Fields(2) & "<br>") ' <-- Returns the Quarter only, that I need for the Quarter Field in the DB
'Response.Write(objRS.Fields(6) & "<br>") ' <-- Returns the entire xml page
Do While Not objRS.EOF
objRS.MoveNext
Loop
%>
<table border="1" width="100%">
<%
dim fld
Response.Write("<tr>")
For Each fld in objRS.Fields
If fld.Name <> "$Text" Then
Response.Write("<td>" & fld.Name & "</td>")
End If
Next
Response.Write("</tr>")
Do While Not objRS.EOF
Response.Write("<tr>")
For Each fld in objRS.Fields
If fld.Name <> "$Text" Then
Response.Write("<td>" & fld.Value & "</td>")
End If
Next
Response.Write("</tr>")
objRS.MoveNext
Loop
%>
</table>
Again, Thank you so much for any advice, links, or any help at all...
have a look here: msxml DOM
you should use the msxml object to read the xml. then you can query the elements of the xml by using the api.
example code for loading the xml:
<%
dim xml : set xml = server.createobject("Msxml2.DOMDocument.6.0")
dim xmlString : xmlString = getXMLHTTPResponse("http://www.iftach.org/TaxMatrix/charts/2Q2012.xml")
xml.loadxml(xmlString)
function getXMLHTTPResponse(url)
dim tout, xmlhttp
set xmlhttp = server.createObject("Msxml2.ServerXMLHTTP.6.0")
with xmlhttp
.setTimeouts 2000, 2000, 2000, 2000
.open "GET", url, false
.send()
if .responseXML.xml = "" then
getXMLHTTPResponse = .responseText
else
getXMLHTTPResponse = .responseXML.xml
end if
end with
end function
%>
Finally!!! I made it work, if anyone needs the solution it is below:
Using the code I posted in the question I added the following code:
Note: I know that this is not the best way to do it since I'm not going after the TAGS/IDs/Names but the xml file will always stay formated the same so I just Looped, Cut out what I needed, and Inserted/Updated into the DB.
<%
' LOOP ALL OF THE FIELDS ONE BY ONE
For x = 0 to xml.documentelement.childnodes.length - 1
set XMLobjchildnodes = xml.documentelement.childnodes
strXMLtxt=xml.documentElement.childNodes(x).text & "<br>"
' SETUP THE UPDATE FIELDS FOR SQL
dim strTest
strTest="objSQL(""IMP_STATE_" & Mid(strXMLtxt,1,2) & """)=" & Mid(strXMLtxt,46,7)
' SKIP THE Fi and 2Q because they are not States/Prov.
if strTest="objSQL(""IMP_STATE_Fi"")=" OR strTest="objSQL(""IMP_STATE_2Q"")=" then
else
' ALSO SKIP hi and U and CN because they also are not States/Prov.
if InStr(strTest,"STATE_ht")>0 OR InStr(strTest,"STATE_U.")>0 OR InStr(strTest,"STATE_CN")>0 then
else
' ADD YOUR SQL CONNECTION INFO. HERE AND THEN INSERT/UPDATE THE SQL DB
end if
Next
%>
Note: By making small changes you can set it to Insert the data into SQL and/or update, should this be a problem for anyone please let me know I will be more then happy to help.
To all that have tried to solve my problem/question, Thank you so much for all of your hard work and effort.
Related
Which one could be a better practice? In my situation, I need to check if a specific data exists in a table. I am iterating through an Excel file and verifying if a code there exists in my table using VB.NET. I have two options to do this (or if there is a better way to do this, I am open for suggestions).
First is to check it one by one, this code is executed per loop:
SQL = "SELECT TOP 1 * FROM Table1 WHERE Code = '" & codeFromExcel & "'"
rs = dbConn.Execute(SQL)
If Not rs.EOF Then
isFound = True
Else
isFound = False
End If
The other one is I load all the codes in a List(Of T)
Dim myList As New List(Of String)()
rs = Nothing
rs = dbConn.Execute("Select Code from Table1")
If Not rs.EOF Then
Do While Not rs.EOF
myList.Add(rs.Fields("Code").Value.ToString)
rs.MoveNext()
Loop
End If
Then check every record if it is in the List(Of T) while iterating in the Excel.
If myList.Contains(codeFromExcel) Then
isFound = True
Else
isFound = False
End If
I've been working with this kind of stuff most of the time and I want to know which one is the most efficient way to use. At the moment I only have a few records in my database. I want my code to be ready and efficient when the time comes that I need to deal with numerous records. Thanks in advance!
Additional info: The data doesn't need to be "fresh" as that table is meant for one-time entry only.
Personally I prefer to open as less connections to data base as possible.
So:
If the table is not very large (some hundred rows) I would go with the "cache" option.
Generally:
I would gather all excel codes in a list. ( excelCodes )
Then I would query something like Select Distinct Code from Table1 Where Code In ( excelCodesList ) and store it in a second list ( foundCodes ).
Then I would compare these lists.
I test it on a table with 6.143.993 rows.
To select just one column (description) to "cache" took 1'29".
On the other hand query like:
select distinct description from ItemDetail where description in ( 'abc','cddd','xxx' )
took 0'58".
UPDATE
An index on Code column might help with performance.
In sqlserver I have a function which generates a complex xml of all products with several tables joined: location, suppliers, orders etc.
No problem in that, it runs in 68 sec and produces around 450MB.
It should only be called occationally during migration to another server, so it doesn't matter it takes some time.
I want to make this available for download over webserver.
I've tried some variations of this in classic asp:
Response.Buffer = false
set rs=conn.execute("select cast(dbo.exportXML() as varchar(max)) as res")
response.write rs("res")
But I just get a standard
An error occurred on the server when processing the URL. Please contact the system administrator.
If you are the system administrator please click here to find out more about this error.
Not my usual custom 500-errorhandler, so I'm not sure how to find the error.
The problem is in response.write rs("res"), if i just do
temp = rs("res")
the script runs, but displays nothing of cause; if I then
response.write temp
I get the same failure.
So the problem is writing such a ling string.
Can I save the file from tsql directly; and run the job periodically from sql agent?
I found that there seems to be a limit on how much data can be written at once using Response.Write. The workaround I used was to break the data into chunks like this:
Dim Data, Done
Done = False
Do While Not Done
Data = RecordSet(0).GetChunk(8192)
If Not Len(Data) = 0 Then
Response.Write Data
Else
Done = True
End If
Loop
Try this:
Response.ContentType = "text/xml"
rs.CursorLocation = 3
rs.Open "select cast(dbo.exportXML() as varchar(max)) as res",conn
'Persist the Recordset in XML format to the ASP Response object.
'The constant value for adPersistXML is 1.
rs.Save Response, 1
Can any one please help me how to Insert the data into database from window form. How to fetch the data to show on window form & same to update the data from database. I am looking for the code that contain sql query with in the code not from the quick select data window. I am very new in powerbuilder.I want to write a code fetch update data from the code any where & show anywhere.
Thanks
I'm not quite sure about your question. Try going to this website http://powerbuilder.hyderabad-colleges.com.
Look for Datawindow control and Datawndow object topics.
There are other ways to manipulate data in Powerbuilder like using Embeded SQL (stored procedure and cursors).
I hope this will help you.
The whole point of the Datawindow is that it does all that work for you.
Retrieve data:
dw_1.Retrieve(arguments)
Update the database:
dw_1.Update()
I'm not understanding the question entirely you must be having trouble with a multi-table update they can be challenging for a new developer.
This will do an update into two tables I did it in a hurry so might be a syntax error or two.
// insert a row
li_row = dw_1.insertrow(0)
dw_1.setitem(li_row, 'col1', 'try reading')
dw_1.setitem(li_row, 'col2', 'the PowerBuilder')
dw_1.setitem(li_row, 'col3', 'manual next time')
// do accept text left out for purposes of brevity
// Update first table and dont bother with another accepttext
// since weve already done one and dont set the updateflags
// so second half of update creates correct sql statement
li_rtn = dw_1.Update(false, false)
if li_rtn = 1 then
dw_1.modify('tbl1_col1.Update = No')
dw_1.modify('tbl1_col2.Update = No')
dw_1.modify('tbl1_col3.Update = No')
dw_1.modify('tbl1_id.Key = No')
dw_1.modify("Datawindow.Table.updateable = 'tbl2'")
dw_1.modify('tbl2_col1.Update = Yes')
dw_1.modify('tbl2_col2_id.Key = Yes')
li_rtn = dw_1.update(false, true)
if li_rtn = 1 then
commit using sqlca;
else
rollback using sqlca;
end if
end if
// cleanup the temp recs
li_rowcount = dw_1.rowcount()
for li_row = li_rowcount to 1 step -1
dw_1.deleterow(li_row)
next
dw_1.Update()
one of the simplest Components in my website just stopped working from one day to the other without any changes in Code.
'Connection Declaration as connection
Set rs = Server.CreateObject ("ADODB.Recordset")
rs.Open "SELECT * FROM tablename ORDER BY id DESC", connection, 1, 3
while not rs.EOF
'writing some Table from the records in DB
'Simplified Code %>
<tr><td><%=rs("id")%></td><td><%=rs("description")&></td></tr>
<%
rs.MoveNext
Wend
in my Database i have verified the extraordinary number of 30 records :(
when above code is executed i see 2 of them
This tells me two things,
first: the tablename is Correct and the connection to the Database is established
second: the table-generation in itself is correct
I also have a smaller Testing-System. there the exact same code on a sample Database produces the expected Results.
Unfortunately i have no means of "instant-access" to my main page for "debugging purposes"
Is there any known Bugs for ADODB Recordsets losing records? Please keep in mind the Code is exactly the same and working "error-free".
A few suggestions.
Use Option Explicit if not already - (I didn't see it in your code) this will display SQL errors, so that may help.
Check that you haven't destroyed RS.
Also, "connection, 1, 3" means 'active connection', 'cursortype', 'locktype'
Your cursortype is 'adOpenKeySet' - 3 or 'adOpenStatic' is better, unless you specifically want a KeySet? try calling the Open this way to force the defaults (which oddly enough are 3 and 1 respectively !) :
RS.Open "SELECT * FROM tablename ORDER BY id DESC",connection
I also usually write RS output loops like this :
If Not RS.BOF Then
' write table tag HTML
Do While Not RS.EOF
' write table row + row data
RS.MoveNext
Loop
' write end table tag HTML
Else
' write "RS is empty!"
End If
This will make it easier to tell if recordset is empty or not.
Is there a way to get the database column DataType length information given a table's EntityType?
Example SQL (SQL Server) that you can run to see precisely what information I am looking for:
select
sys.tables.name as 'Table Name',
sys.columns.name as 'Column Name',
sys.systypes.name as 'DataType',
sys.columns.max_length as 'Max Length',
sys.columns.precision as 'Precision'
from
sys.columns, sys.systypes, sys.tables
where
sys.columns.system_type_id = sys.systypes.xtype
and sys.systypes.name <> 'sysname'
and sys.tables.type = 'U'
and sys.tables.name <> 'sysdiagrams'
and sys.columns.object_id=sys.tables.object_id
order by
sys.tables.name, sys.columns.column_id;
The last 3 columns contain the data that I would like to have access to because I'm generating some documentation. One example reason for the documentation is: Entity Framework will throw an Exception by default if a string is set on a property that can't support it's length. A developer without access to the database metadata has a challenge with the discoverability of length requirements in this case.
Thanks,
Aaron
Unfortunately no.
Even if that information is correctly captured in the SSDL (i.e. the Storage Schema Definition language) there is no public API in EF to go from C-Space (conceptual model) property to S-Space (storage model) column.
If your model is simple you can perhaps infer that information, using the EF metadata workspace and some simple heuristics, but once things get even a little complicated, those heuristics will break down.
Your only option at that point is to write code to interpret MSL (mapping or CS-Space) files, and use that in conjunction with the MetadataWorkspace to go from C-Space to S-Space.
EDIT: as pointed out by KristoferA you often have the attribute on the C-Space property, so you can go to directly to that. Unfortunately that is not always the case, and often it gets out of sync with the database.
I'm pretty sure that Julie Lerman's book covers how to get maxlength, at least a tool to validate against it, by making changes in the POCO creation. Chapter 13, starts around page 356. Example 13-12 covers it, it starts with
string MaxLengthValidation(EdmProperty prop)...
it's copyrighted material so I won't cut/paste it, but I hope you can buy a copy of her book and get the info.
Yes, this is possible: (EF6.1)
<Extension>
Public Function GetColumns(Of TEntity)(Db As IObjectContextAdapter) As List(Of DataColumn)
Dim oMetadata As MetadataWorkspace
Dim oObjects As ObjectItemCollection
Dim oContext As ObjectContext
Dim oColumn As DataColumn
Dim oQuery As Func(Of EdmProperty, Boolean)
Dim oType As EntityType
GetColumns = New List(Of DataColumn)
oContext = Db.ObjectContext
oMetadata = oContext.MetadataWorkspace
oObjects = oMetadata.GetItemCollection(DataSpace.OSpace)
oType = oMetadata.GetItems(Of EntityType)(DataSpace.OSpace).
Single(Function(EntityType As EntityType) oObjects.GetClrType(EntityType) Is GetType(TEntity))
oQuery = Function(EdmProperty As EdmProperty) EdmProperty.DeclaringType.Name = oType.Name
oType.Properties.ToList.ForEach(Sub(Column As EdmProperty)
oColumn = New DataColumn With
{
.AutoIncrement = Column.IsStoreGeneratedIdentity,
.AllowDBNull = Column.Nullable,
.ColumnName = Column.Name,
.DataType = Column.PrimitiveType.ClrEquivalentType,
.Caption = Column.Name
}
If oColumn.DataType Is GetType(String) Then
oColumn.MaxLength = Column.MaxLength.GetValueOrDefault
Else
oColumn.MaxLength = -1
End If
GetColumns.Add(oColumn)
End Sub)
End Function