How can I change the proxy every time when I make a new request?
If I start the application and run this request multiple times, I can see it uses the first proxy used, even if I change the parameters for proxy from a list of proxy servers every time I call this function. I tried to dispose of the client and also to set the client proxy to null, but it didn't help.
If I close the program and restart it, the proxy uses a different IP address.
string GetRequest()
{
WebClient client = new WebClient();
WebProxy wp = new WebProxy(randomProxyAddress + ":" + portNum);
wp.UseDefaultCredentials = false;
wp.Credentials = new NetworkCredential(username, password);
client.Proxy = wp;
string str = client.DownloadString("http://www.example.com");
client.Proxy = null;
client.Dispose();
return str;
}
I found out I had to set request.KeepAlive = false in WebClient.
Related
I try to call an external web service (not mine) from my GWT application working with App Engine.
I know it's impossible to do it from the client due to the SOP (Same Origin Policy) and RequestBuilder is not a solution on the server. I followed the tutorial on the web site and using java.net as well
Here is the client
AsyncCallback<CustomObject> callback = new AsyncCallback<CustomObjectCustomObject>() {
#Override
public void onFailure(Throwable caught) {
caught.printStackTrace();
}
#Override
public void onSuccess(CustomObject result) {
// code omitted
}
};
service.callMethod(aString, callback);
And this is the server
try {
String xmlRequest = "xmlToSend";
URL url = new URL("https://www.externalWebService.com");
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod("POST");
conn.setAllowUserInteraction(false);
conn.setDoOutput(true);
conn.setRequestProperty("Content-Type","application/soap+xml");
conn.setRequestProperty( "Content-length", Integer.toString(xmlRequest.length()));
conn.setRequestProperty("charset", "utf-8");
conn.setConnectTimeout(10000);
OutputStream rawOutStream = conn.getOutputStream();
PrintWriter pw = new PrintWriter(rawOutStream);
pw.print(xmlRequest);
pw.flush();
pw.close();
if(conn.getResponseCode() != 200){
// Something...
}
I keep having the same error at conn.getResponseCode() :
java.lang.ClassCastException: com.google.appengine.repackaged.org.apache.http.message.BasicHttpRequest cannot be cast to com.google.appengine.repackaged.org.apache.http.client.methods.HttpUriRequest
Without making a real request, the remote service works well : it's able to serialize and return objects to the client. The issue is not linked to the communication between the client and the server, it's more like AppEngine doesn't support HttpURLConnection. But it should on the server (isn't it?)
Any thoughts would be hightly appreciated! Thanks in advance
Your problem has nothing to do with GWT: as long as you are running on the server, you can use any 'normal' Java and it will work unless AppEngine has restrictions.
It seems you have imported the repackaged version of Apache HttpClient in your class. You should not do that: download your own HttpClient .jar, add it to the dependencies and use that one.
AppEngine also has some issues with HttpClient. There's an adapter available here that fixes most of the issues.
Thanks #Marcelo, you were right!
Here is the solution I found.
I added httpcore.jar and httpclient.jar to my build path and wrote the code below for the server (the client is the same) :
String xmlRequest = "xmlToSend";
CloseableHttpClient httpclient = HttpClients.custom().build();
//RequestConfig requestConfig = RequestConfig.custom()
// .setConnectionRequestTimeout(10000)
// .build();
try {
ByteArrayOutputStream out = new ByteArrayOutputStream();
Writer writer = new OutputStreamWriter(out);
writer.write(xmlToSend);
writer.flush();
writer.close();
HttpPost request = new HttpPost("https://www.externalWebService.com/path");
request.setEntity(new ByteArrayEntity(out.toByteArray()));
//request.setConfig(requestConfig);
CloseableHttpResponse response = httpclient.execute(request);
if(response.getStatusLine().getStatusCode() == 200){
// retrieve content with a BufferReader
// from response.getEntity().getContent()
...
}
The code works and is up to date.
Edit
Here is the rest of the solution when using a proxy. Mine only deals with NTCredentials but otherwise UsernamePasswordCredentials can be used instead.
HttpHost proxy = new HttpHost("addresse.proxy.com", port);
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(proxy),
new NTCredentials(System.getProperty("user.name") + ":" + password));
RequestConfig requestConfig = RequestConfig.custom()
.setProxy(proxy)
.build();
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCredentialsProvider(credsProvider)
.setDefaultRequestConfig(requestConfig)
.build();
Thanks again for your help, I really appreciated!
I am trying to retrieve a file from an ftp server with anonymous authentication using java.net.URLConnection.
try {
url = new URL("ftp://ftp2.sat.gob.mx/Certificados/FEA/000010/000002/02/03/05/00001000000202030500.cer");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = in.read(buffer)) >= 0)
{
baos.write(buffer, 0, bytesRead);
}
baos.flush();
arr = baos.toByteArray();
in.close();
} catch (Exception e) {
throw new Exception("Error SAT: " + e.getMessage());
}
The file i am trying to get is this, its in an anonymous authentication ftp site:
ftp://ftp2.sat.gob.mx/Certificados/FEA/000010/000002/02/03/05/00001000000202030500.cer
But every time I get this error:
Permission denied: Attempt to bind port without permission.
I am using GoogleAppEngine Java 1.7
Any kind of advise is welcome.
I'm not a Java guy, but I suspect you're trying to use "active" FTP, which is likely the default.
Active FTP works by binding to a port on the receiving computer (the client in this case) to which the sending server can connect to send the file; the port number is sent over in the get request. This doesn't work in many environments, e.g. NAT.
The usual solution is to use "passive" mode, which behaves more like HTTP and doesn't require any port binding. If there's a way in Java to twiddle that connection to use passive mode, it should bypass the permissions issue.
Most likely you have a non billing-enabled app, according to this post and this AppEngine Socket Java API documentation you just have to enable billing, if you have no budget set the limits to $0.
I'm developing an application on GAE that fetches a web page and searches it for a link.
This page gets updated every morning, so a cron job is executed each morning every 15 minutes for a couple of hours, to obtain current day's page.
Here's the problem: if at the first execution of the cron job the application finds the older page (yesterday's one), it keeps fetching that one, although a new page is available at the same URL.
Seems that a cache is used somewhere, but I can't disable it.
The code that the application uses for downloading the page is simply Java I/O:
InputStream input = null;
ByteArrayOutputStream output = null;
HttpURLConnection conn = null;
URL url = new URL("http://www.page.url.net");
try {
conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(0);
conn.setUseCaches(false);
int httpResponseCode = conn.getResponseCode();
if (httpResponseCode == HttpURLConnection.HTTP_OK) {
input = conn.getInputStream();
output = writeByteArrayOutputStreamFromInputStream(input);
} else {
throw new IOException("response code " + httpResponseCode);
}
} finally {
if (input != null) {
output.close();
conn.disconnect();
}
}
What's wrong?
In order to avoid caching, I suggest to use this simple trick: add a "fake" query parameter to the end of the query string, for example if the page you are fetching is
http://www.page.url.net
add a parameter named dummy= so the url becomes:
http://www.page.url.net?dummy=2013-05-25
Just be sure the "dummy" paramater is not actually interpreted by the remote server.
Hope this helps.
I have a policy file server up and running. For a while I was getting the AccessDenied because the policy file was not set properly. Now I no longer receive that error, so I know that's not the issue. I have a simple server running that simple loops on accepting client connections from any address. I also wrote a simple client, so I know the server works. In Silverlight I set my args and then call ConnectAsync. It return immedately on localhost (makes sense) and when I check the event args LastOperation is Connect and SocketError is Success. However, when I check my socket, it is not connected at all. Any ideas..? Been banging my head against a wall for hours over this.
A few other things I've tried. I moved the servers off my local box onto another server. Still didn't work. I did a packet capture and noticed that it is receiving the Poilcy File, but after that, there is no packet sent out by the browser to even attempt to connect to the other server.
public void Connect(string ip)
{
SocketAsyncEventArgs saea = new SocketAsyncEventArgs();
Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
var endpoint = new IPEndPoint(IPAddress.Parse(ip), 4502);
saea.UserToken = socket;
saea.RemoteEndPoint = endpoint;
saea.Completed += new EventHandler<SocketAsyncEventArgs>(AsyncEventComplete);
var completedSync = socket.ConnectAsync(saea);
if (completedSync)
{
AsyncEventComplete(null, saea);
}
Result = ip;
}
void AsyncEventComplete(object sender, SocketAsyncEventArgs e)
{
switch (e.LastOperation)
{
case SocketAsyncOperation.Connect:
MessageBox.Show("CONNECTED");
break;
case SocketAsyncOperation.Receive:
MessageBox.Show("DATA RECEIEVED");
// do stuff
break;
}
}
I think you should use e.SocketError and not e.LastOperation
You could also use e.ConnectSocket (in Silverlight only)
You should also add a "not" in this condition : if ( ! completedSync )
Is it possible to know which node the selenium grid hub assigned to your test? My tests need to talk to other services on the node machine in order to perform configurations which are not supported by selenium.
Mark
Generally you shouldn't rely on knowing what machine your test is running on. Grid 2 provides a series of callback listeners that you could implement to provide machine configuration. But, if you really want to see what node a test is running on, you could use one of the API calls. Both endpoints can be found on the hub:
http://localhost:4444/grid/api/proxy
http://localhost:4444/grid/api/testsession
Neither are documented yet. But if you view the source, it's straightforward to see how they work. You want to look at the ProxyStatusServlet and TestSessionStatusServlet.
String hub = "grid_server_host"; //IP or hostname of GRID
int port = 4444; // port no.
HttpHost host = new HttpHost(hub,port);
DefaultHttpClient client = new DefaultHttpClient();
String url = host + "/grid/api/testsession?session=";
URL session = new URL(url + ((RemoteWebDriver) webdriver).getSessionId());
BasicHttpEntityEnclosingRequest req;
req = new BasicHttpEntityEnclosingRequest("POST", session.toExternalForm());
org.apache.http.HttpResponse response = client.execute(host,req);
JSONObject object = new JSONObject(EntityUtils.toString(response.getEntity()));
String proxyID = (String) object.get("proxyId");
String node = (proxyID.split("//")[1].split(":")[0]);