HtmlUnit parses "http://www.ean-search.org/sitemap.html" with 404 return - http-status-code-404

I'm trying to get and parse the page "http://www.ean-search.org/sitemap.html", but it always got 404 error and empty page. All the text content area are blank.
I tried many options configuration of HtmlUnit webclient, e.g. .setThrowExceptionOnFailingStatusCode(false), setThrowExceptionOnScriptError(true), setRedirectEnabled(false), setJavaScriptEnabled(true), setThrowExceptionOnScriptError(false).
None of them worked...
Anyone have any suggestion? Thanks.
ps: my webclient code:
myWebClient = new WebClient(BrowserVersion.FIREFOX_3_6);
myWebClient.setIncorrectnessListener(new CustomizedInconnectnessListener());
myWebClient.setTimeout(180000); //3 min, used twice, first for connection, second for retrieval
try {
myWebClient.setUseInsecureSSL(true);
} catch (GeneralSecurityException ex) {
logger.log(Level.SEVERE, "cannot set UseInsecureSSL for BNP webclient",ex);
//ignore it, continue
}
myWebClient.setRedirectEnabled(true);
myWebClient.setCssEnabled(false);
myWebClient.setJavaScriptTimeout(30000); //timeout for executing java script
myWebClient.setThrowExceptionOnScriptError(false);
HtmlPage htmlpage = (HtmlPage) myWebClient.getHtmlPage("http://www.ean-search.org/sitemap.html");
myWebClient.waitForBackgroundJavaScriptStartingBefore(3000);
Thread.sleep(3000);
System.out.println(htmlpage.asXml());

Well, the code you're using looks really bad. I got many errors and warnings... it is not even possible to compile it. Eg myWebClient.getHtmlPage should be myWebClient.getPage.
This code works for me and outputs the content of the page:
WebClient myWebClient = new WebClient(BrowserVersion.FIREFOX_17);
HtmlPage page = myWebClient.getPage("http://www.ean-search.org/sitemap.html");
System.out.println(page.asXml());
Make sure you use the latest HtmlUnit libraries and also pay attention to the compiler when it tells you that something is deprectated. It is not advised to use those method and fields.

Related

WPF MessageBox in App.xaml.cs stops MainWindow initialisation so app never appears but is (apparently) running correctly

There's no error message and no indication why it is not displaying the window. The app initialises App.xaml.cs: App() {} and I can step through the App.xaml file. It gets the startup uri and then... silence. No output in the Output window and no unhandled exception and no window, I can't find where to put a breakpoint to debug as it isn't hitting the start of MainWindow.xaml.cs.
Really confused.
This was working 20m ago.
In that time all I did was add Windows.Office.Interop.Outlook reference. I removed the reference and rebuilt but still the same. Would that cause this problem? Has anyone seen this before? Google isn't helping!
EDIT :
App.xaml.cs:
public App()
{
using (var dbContext = new DBEntities())
{
if (!db.Exists())
{
try
{
db.Database.Create();
MessageBox.Show("Database created"); // this is the problem!!
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
}
}
I've added App.xaml.cs, I found that the problem was using a MessageBox to give info (this is still in development!). I'd been meaning to get rid of it and eventually did and my problem went away. This meant I could find relevent Goolge results:
MSDN query and answer for exactly my problem
I will be adding an 'loading window' in between app load and main window load eventually in which I will be able to feedback information using Bindings etc.
Fixed error by removing the MessageBox.Show(..) call. The selected answer from the MSDN URL given in the question states:
"I performed a test based on your description, the applicationi stop at the method : USER32!GetMessageW+0x33, calling USER32!NtUserGetMessage"
I assume this is what was occurring in my case, although I didn't test it.
What happens if you create a new window and set that as the StartupUri?
You also might want to create a new project and make sure that the namespaces referenced in the App.xaml in your existing app haven't somehow been inadvertently edited.

Create an attachment from MVCPortlet - serveResource

I need to export a .property file when user clicks on a hyperlink. I am using liferay portal 6.1.1. When user clicks on hyperlink, I am making a jquery.get() to the serveResource method which is inside the MyMVCPortlet class. Here the code that always writes the content to response(verified on fiddler) but is not creating a downloadable file.
resourceResponse.reset();
resourceResponse.setContentType("text/plain");
resourceResponse.setProperty("content-disposition", "attachment; filename=test.txt");
OutputStream out = resourceResponse.getPortletOutputStream();
try {
out.write("key=value".getBytes());
}
catch(IOException e){
e.printStackTrace();
}
finally {
out.close();//Also tried out.flush(); - dint help
}
Do I need to set something on resourceResponse after the write is complete?
I tried different options and got exhausted. The same code on plain java servlet works but not on liferay. Is there anything I am missing? Thanks in advance!
Try to use one of PortletResponseUtil.sendFile() instead of manual operations on portletOutputStream. Eg.
PortletResponseUtil.sendFile(resourceRequest, resourceResponse, "test.txt", "key=value".getBytes())

Download file with JSF

everyone!
I have a trouble. I tried to save excel file in jsf web application.
I generated file by my utils and trying to get "save" window, but I failed.
Here is my code:
<div>
<h:commandButton value="Apply" actionListener="#{hornPhonesBean.generateReport}"/>
</div>
and:
public void generateReport(ActionEvent event) {
System.out.println("GENERATE REPORT FROM = " + this.dateFrom + "; TO = " + this.dateTo);
try {
XSSFWorkbook workbook = (XSSFWorkbook) HornReportGenerator.getWorkbook(null, null);
String fileName = "1.xlsx";
FacesContext fc = FacesContext.getCurrentInstance();
ExternalContext ec = fc.getExternalContext();
// Some JSF component library or some Filter might have set some headers in the buffer beforehand. We want to get rid of them, else it may collide.
ec.responseReset();
// Check http://www.w3schools.com/media/media_mimeref.asp for all types. Use if necessary ExternalContext#getMimeType() for auto-detection based on filename.
ec.setResponseContentType("application/vnd.ms-excel");
// Set it with the file size. This header is optional. It will work if it's omitted, but the download progress will be unknown.
//ec.setResponseContentLength(contentLength);
// The Save As popup magic is done here. You can give it any file name you want, this only won't work in MSIE, it will use current request URL as file name instead.
ec.setResponseHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
OutputStream output = ec.getResponseOutputStream();
workbook.write(output);
output.flush();
output.close();
fc.responseComplete(); // Important! Otherwise JSF will attempt to render the response which obviously will fail since it's already written with a file and closed.
System.out.println("END");
} catch (Exception e) {
e.printStackTrace();
}
}
I read suggestions here and from another forums - everyone says I shouldnt use , but I didn't use it at all.
Then I thought that the problem could be in the
<ice:form>,
where I kept the
<h:commandButton>,
and I changed to
<h:form>,
but it didn't help.
Maybe the problem in the request - it has header Faces-Request partial/ajax. But I am not sure.
Please give me some ideas - I already spent 4 hours for this crazy jsf download issue)
Maybe the problem in the request - it has header Faces-Request partial/ajax. But I am not sure.
This suggests that the request is an ajax request. You can't download files by ajax. Ajax requests are processed by JavaScript which has for obvious security reasons no facilities to programmatically pop a Save As dialogue nor to access/manipulate client's disk file system.
Your code snippet does however not show that you're using ajax. Perhaps you oversimplified it too much or you're using ICEfaces which silently auto-enables ajax on all standard JSF command components.
In any case, you need to make sure that it's not sending an ajax request.
See also:
How to provide a file download from a JSF backing bean?
ICEfaces libary in classpath prevents Save As dialog from popping up on file download

ExtJS Ext.ux.form.FileUploadField issue with IE8

File upload works fine with Firefox but when running the code from IE8, and after the upload has been successful IE displays a:
"File Download - Security Warning"
Do you want to save this file, or find a program online to open it?
Name: uploadImage7c8c609b
Type: Unknown File Type, 163 bytes
From: 127.0.0.1
(sorry as a new user I can't post the screenshot!)
How can I switch this off?
dude, exactly the same thing happens to me.
The way out of this mess, in my case, is to change the mimetype of the response.
(hint comes from here: https://stackoverflow.com/a/4723649)
I use spring's utillity, which in turn uses Jackson, to automatically transform objects to json strings.(Well, there's some configuration needs to be done, annotation-driven, to name but a few.)
So instead of writing like this:
public #ResponseBody
ExtjsJsonCmd upload(..., MultipartFile file) {
....
return new ExtjsJsonCmd(true,"certain message...");
}
I put down this:
public #ResponseBody
String upload(..., MultipartFile file){
....
return "{success:true, msg:'certain message...'}";
}
Hope it will make it through.

Windows Phone 7 App Quits when I attempt to deserialize JSON

I'm developing my first windows phone 7 app, and I've hit a snag. basically it's just reading a json string of events and binding that to a list (using the list app starting point)
public void Load()
{
// form the URI
UriBuilder uri = new UriBuilder("http://mysite.com/events.json");
WebClient proxy = new WebClient();
proxy.OpenReadCompleted += new OpenReadCompletedEventHandler(OnReadCompleted);
proxy.OpenReadAsync(uri.Uri);
}
void OnReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
if (e.Error == null)
{
var serializer = new DataContractJsonSerializer(typeof(EventList));
var events = (EventList)serializer.ReadObject(e.Result);
foreach (var ev in events)
{
Items.Add(ev);
}
}
}
public ObservableCollection<EventDetails> Items { get; private set; }
EventDetails is my class that wraps the json string. this class has to be correct because it is an exact copy of the class used by that website internally from which the json is generated...
I get the json string correctly from the webclient call (I read the memorystream and the json is indeed there) but as soon as I attempt to deserialize the string, the application exits and the debugger stops.
I get no error message or any indication that anything happen, it just stops. This happens if I type the deserialize method into the watch window as well...
I have already tried using JSON.net in fact I thought maybe it was a problem with JSON.net so I converted it to use the native deserializer in the .net framework but the error is the same either way.
why would the application just quit? shouldn't it give me SOME kind of error message?
what could I be doing wrong?
many thanks!
Firstly, the fact that you have some string there that looks like JSON does not mean that you have a valid JSON. Try converting a simple one.
If your JSON is valid, it might be that your JSON implementation does not know how to convert a list to EventList. Give it a try with ArrayList instead and let me know.
The application closes because an unhandled exception happens. If check the App.xaml.cs file you will find the code that closes your app. What you need to do is try catch your deserialization process and handle it locally. So most likely you have some JSON the DataContractJsonSerializer does not like. I have been having issue with it deserializing WCF JSON and have had to go other routes.
You may want to check to ensure your JSON is valid, just because your website likes it does not mean it is actually valid, the code on your site may be helping to correct the issue. Drop a copy of your JSON object (the string) in http://jsonlint.com/ to see if it is valid or not. Crokford (the guy who created JSON) wrote this site to validate JSON, so I would rely on it more than your site ;) This little site has really helped me out of some issues over the past year.
I ran into this same kind of problem when trying to migrate some existing WM code to run on WP7. I believe that the WP7 app crashes whenever it loads an assembly (or class?) that references something that's not available in WP7. In my case, I think it was Assembly.Load or something in the System.IO namespace, related to file access via paths.
While your case might be something completely different, the symptoms were exactly the same.
The only thing I can recommend is to go through the JSON library and see if it's referencing base classes that are not allowed in WP7. Note that it doesn't even have to hit the line of code that's causing the issue - it'll crash as soon as it tries to hit the class that contains the bad reference.
If you can step into the JSON library, you can get a better idea of which class is causing the problem, because as soon as the code references it, the whole app will crash and the debugger will stop.

Resources