Reading a table from a webpage using msxml2.xmlhttp - msxml

I have tried to use a webbrowser in my VB program to load a web page and the table on the web page. It works but VERY slowly and a lot of the time it freezes.
I found some sample code that reads the web page without a web browser. It reads the text on the web page and puts it into a Rich Text Box called WebText.
This code works really quick and never locks up but it does not read the text that is in the table on the webpage.
The code I am using calls the function below:
WebText.Text = GetHTMLCode("http://xfinitytv.comcast.net/tv-listings")
Function GetHTMLCode(ByVal strURL) As String
Dim strReturn ' As String
Dim objHTTP ' As MSXML.XMLHTTPRequest
If Len(strURL) = 0 Then Exit Function
objHTTP = CreateObject("MSXML2.XMLHTTP")
objHTTP.open("GET", strURL, False)
objHTTP.send() 'Get it.
strReturn = objHTTP.responseText
objHTTP = Nothing
GetHTMLCode = strReturn
End Function
I am new to programming (hoping to get better with time). If I write something that sounds like I do not know what I am talking about it is probably because I don't. Any help would be appreciate.
Thanks,
Chris

Related

Adding a DocumentWindow to a DocumentTabStrip causes the application to hang indefinitely

I have a Winforms application with a primary form that contains (among other things) a Telerik DocumentTabStrip. These tabs are used to hold user controls or web pages (via a web browser control). It has worked fine for quite a while, but I'm running into an issue now.
I recently switched the web browser control from the built-in .NET web browser based on IE to CefSharp. Since doing so, I've noticed that occasionally when trying to add the DocumentWindow to the DocumentTabStrip, the call will hang indefinitely (in debug) or crash outright (running the app normally). This only appears to happen when opening a DocumentWindow that contains the browser control, not any other user controls. The actual call itself is below.
I'm at a bit of a loss as to how to even begin to debug this, since there's no error that gets received - it just hangs inside the Controls.Add() method indefinitely. Any advice would be appreciated.
Private dts As New Telerik.WinControls.UI.Docking.DocumentTabStrip
Try
dts.InvokeIfRequired(Sub()
Dim docWindow As Telerik.WinControls.UI.Docking.DocumentWindow = Nothing
Dim ctrl As ucBaseControl = Nothing
Dim browser As ucBrowser = Nothing
Dim isBrowser As Boolean = False
docWindow = New Telerik.WinControls.UI.Docking.DocumentWindow
docWindow.BackColor = Color.FromArgb(89, 89, 89)
'Do various stuff to determine the type of control to load (ctrl or browser), then setup the applicable control
If isBrowser Then
'Place the browser into the Document Window.
If Not IsNothing(browser) Then
browser.Dock = DockStyle.Fill
docWindow.Controls.Add(browser)
End If
Else
'Place the ctrl into the Document Window.
ctrl.Dock = DockStyle.Fill
docWindow.Controls.Add(ctrl)
End If
'Add the DocumentWindow to the DocumentTabStrip
' Ensure DockWindow not disposed due to lag in bringing up
If IsNothing(docWindow) OrElse docWindow.IsDisposed Then
Exit Sub
End If
Try
docWindow.Padding = New Padding(0)
dts.TabStripElement.Children(0).Children(1).Padding = New Padding(0)
dts.Controls.Add(docWindow) 'This is where the issue is. It only happens sporadically here.
Catch ex As Exception
'Code to log any exceptions here. In the problem described here, no exception is ever generated, though.
End Try
'Bring the control to the front and focus it, here...
End Sub)
Catch ex As Exception
'Error handling code here'
End Try
I'm assuming InvokeIfRequired is an extension method you've created for Controls. Note that if it relies on Invoke, that is a synchronous call, instead use BeginInvoke (see: What's the difference between Invoke() and BeginInvoke())
No exception was ever thrown because you were suffering from deadlock

Getting filenames from a folder in Sharepoint with VBA

I've done alot of research on this and found a number of help sites but still can't understand why this sometimes doesn't work.
I'm trying to access a sharepoint site (to which there are no restrictions for me) and extract all the files in a folder within that site.
Sometimes my Path works and it does it, other times it does not. I have a feeling it works if I've gone into the sharepoint site on my browser before but can't confirm that (because I just tried it again now and it doesnt work ARGGH). But the same code below has worked in the past.
It's failing on the File System Object function below
Public Function GetFullFileName(strfilepath As String, _
strFileNamePartial As String) As String
Dim objFS As Variant
Dim objFolder As Variant
Dim objFile As Variant
Dim intLengthOfPartialName As Integer
Dim strfilenamefull As String
Set objFS = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFS.GetFolder(strfilepath)
'work out how long the partial file name is
intLengthOfPartialName = Len(strFileNamePartial)
For Each objFile In objFolder.Files
'Test to see if the file matches the partial file name
If Left(LCase(Replace(objFile.Name, " ", "")), intLengthOfPartialName) = LCase(strFileNamePartial) Then
'get the full file name
strfilenamefull = objFile.Name
Exit For
Else
End If
Next objFile
'Return the full file name as the function's value
GetFullFileName = strfilenamefull
End Function
I get a "Run-time error '76': Path not found" when it gets to the GetFolder(strfilepath) code
The strfilepath is just a regular sharepoint site name (e.g. like \teams.uk\gm\FX\SharedDocuments\London\11) November 2013\20 November\Reports)
As mentioned I've tried different variations of the file path including DavWWW but nothing seems to work and I dont know what else to try.
Any advice please?
Thanks
Raiyan
The webclient service must be started to Access SharePoint using the FileSystemObject.
This service is often not started when you first log on. However as you access SharePoint via the normal user interface it gets started. I think this is why your code works inconsistently.
If you have admin rights on the local machine you can start the service manually (or using code). However if you don't have rights then you can try a trick - it's clumsy but works.
Using VBA code open a known SharePoint folder in explorer view then close it. This will trigger webclient to start.

InternetGetCookieEx: not working

I am trying to read an HttpOnly cookie through a WPF, VB.NET application.
The site in question, upon visit, serves 2 cookies: one HttpOnly, one regular.
I visit the site whose cookie I want to read via Internet Explorer. Then Developer tools --> Cache --> View cookie information. ONLY the regular cookie is shown there, not the HttpOnly one (Chrome displays both of them correctly. If I delete it and refresh, the site returns a 500 error so the cookie definitely exists).
I run my code, based on what is explained here: c# Get httponly cookie, which is as follows:
Private Const INTERNET_COOKIE_HTTPONLY As Integer = &H2000
<SuppressUnmanagedCodeSecurity, SecurityCritical, DllImport("wininet.dll", EntryPoint:="InternetGetCookieExW", SetLastError:=True, CharSet:=CharSet.Unicode, ExactSpelling:=True)>
Friend Shared Function InternetGetCookieEx(<[In]> Url As String, <[In]> cookieName As String, <Out> cookieData As StringBuilder, <[In], Out> ByRef pchCookieData As UInteger, flags As UInteger, reserved As IntPtr) As Boolean
End Function
<SecurityCritical()>
Public Shared Function GetCookie(url As String) As String
Dim size As Integer = 0
Dim sb As New StringBuilder
If InternetGetCookieEx(url, vbNullString, Nothing, size, INTERNET_COOKIE_HTTPONLY, IntPtr.Zero) Then '<-- this always returns false
If size <= 0 Then
Return Nothing
End If
sb = New StringBuilder(size + 1)
If Not InternetGetCookieEx(url, vbNullString, sb, size, INTERNET_COOKIE_HTTPONLY, IntPtr.Zero) Then
Return Nothing
End If
End If
Dim lastErrorCode = Marshal.GetLastWin32Error '<-- 259
Return sb.ToString()
End Function
GetCookie("https://www.xyz.com")
I have tried numerous variations of the above, the result is the same: lastErrorCode ALWAYS equals to 259, which in turn translates to ERROR_NO_MORE_ITEMS, meaning that no cookies have been found.
1) The site in question is in the trusted sites zone, so it doesn't work under protected mode.
2) The site is under SSL (I do not know if this has to do with anything).
3) I have desperately searched my hard disk for these cookies' location, to no avail.
4) Both are session cookies (ie no declared expiration date)
5) Windows 8 x64, IE10, VS2012
This is a tiny bit of a project milestone which has given me countless hours of pain, so any help will be greatly appreciated.
I am very willing to change my methodology completely as soon as it will give me this cookie's value, unless it's overkill (ie winpcap / fiddlercore etc.)
You're correct to note that your code would only ever work in the Trusted Zone, due to Q10 here: http://blogs.msdn.com/b/ieinternals/archive/2009/08/20/wininet-ie-cookie-internals-faq.aspx since cookies aren't shared between IE (which runs in Protected Mode or AppContainer) and your application (which runs at Medium IL).
You should pass a valid URL into the function (e.g. you need at least a trailing slash after the hostname); even if an invalid URL works today, it might not in the future.
Also keep in mind that you'll only ever see IE's persistent cookies with this code; Session cookies are isolated per-process, so your application won't see Session cookies from an IE tab.
To all whom it may concern:
Eric was absolutely right: Session cookies are in-process, so they are virtually invisible from the outside world.
One possible solution that works is the following:
Load the site in a webbrowser control.
The code I posted works as expected.

Getting RavenDB Client API to work in Silverlight

I just learned that RavenHQ is an easy-to-use database for .NET and especially Silverlight. I'm trying to insert some Test Data (see Code) but it won’t work. I think I am missing something big because I'm new to RavenDB.
I added the references to the project and created clientaccesspolicy.xml and crossdomain.xml. I get (Firebug) an HTTP 200 Response to "GET clientaccesspolicy.xml" from 1.ravenhq.com but nothing else happens and no data is inserted. Any Ideas would be much appreciated.
Private Sub TestRavenDB()
Try
Dim ds = New DocumentStore()
ds.Url = "https://1.ravenhq.com/databases/AppHarbor_60e82bd1-234f-4178-a59a-b527a1d391bb"
ds.ApiKey = "1f827c44-3e38-4e66-8801-83ba03b01f67"
ds.Initialize()
Dim p As New Person("Donald")
Dim session As IAsyncDocumentSession = ds.OpenAsyncSession()
session.Store(p)
session.SaveChangesAsync()
Catch ex As Exception
MessageBox.Show(ex.Message)
End Try
End Sub
Note that you have SaveChangesAsync there. Before your test return, you need to wait for that to complete.

HttpWebRequest.BeginGetResponse is not Async

I have some code that iterates a few 100 urls and requests the data from the web.
It looks something like this
for each url in urls
Dim hwr = CType(WebRequest.Create(url), HttpWebRequest)
Dim rq = New ReqArgs
rq.Url= url
rq.Request = hwr
Dim res =
hwr.BeginGetResponse(New AsyncCallback(AddressOf FinishWebRequest), rq)
Dim a = 1
next
Does this look ok?
How come the BeginGetresponse line takes about 2-3 seconds to complete before going to dim a=1?.
Actually I debugged and I see that the FinishWebRequest procedure runs completely before the Dim a=1 is reached.
So is this async?
I'm not earning any time by using the async. am I? Or is there a different way to do this?
The point is that the main sub should fire off 300 requests and return control to the UI, then the FinishWebRequest should process them slowly on its own thread and own time , as the requests come in.
How do I do that?
Btw, the main sub is running in a BackgroundWorker, but I checked with out the BackgroundWorker and the problem is the same
It seems that the answer should be here but its just not working for me
I'm WPF 4.0
Appreciate your help and advice. Thanks
yup
the problem was with the POST
i now start the post writing like this
Dim ReqStream = hwr.BeginGetRequestStream(New AsyncCallback(AddressOf FinishRequestStream), rq)
and then my callback is like this
Sub FinishRequestStream(ByVal result As IAsyncResult)
Dim ag = CType(result.AsyncState, ReqArgs)
Dim postStream = ag.Request.EndGetRequestStream(result)
Dim PostBytes = Encoding.UTF8.GetBytes(ag.PostText)
postStream.Write(PostBytes, 0, PostBytes.Length)
postStream.Close()
Dim res = ag.Request.BeginGetResponse(New AsyncCallback(AddressOf FinishResponse), ag)
End Sub
hope this helps someone in the future
Reposting this from another question.
From the documentation on HttpWebRequest.BeginGetResponse Method:
The BeginGetResponse method requires some synchronous setup tasks to complete (DNS resolution, proxy detection, and TCP socket connection, for example) before this method becomes asynchronous. [...]
it might take considerable time (up to several minutes depending on network settings) to complete the initial synchronous setup tasks before an exception for an error is thrown or the method succeeds.
To avoid waiting for the setup, you can use
HttpWebRequest.BeginGetRequestStream Method
but be aware that:
Your application cannot mix synchronous and asynchronous methods for a particular request. If you call the BeginGetRequestStream method, you must use the BeginGetResponse method to retrieve the response.

Resources