I'm opening a url and getting a SocketTimeoutException:
long now = System.currentTimeMillis();
try {
URL url = new URL("https://example.com");
InputStreamReader is = new InputStreamReader(url.openStream());
..
}
catch (SocketTimeoutException ex) {
long diff = System.currentTimeMillis() - now;
System.err.println("Timeout!: " + diff + "ms"); // ~4 seconds
}
java.net.SocketTimeoutException: Timeout while fetching URL: https://example.com
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:142)
but the elapsed time is only 4 seconds. This code of mine hasn't changed since February, same with the "example.com" url it's hitting (which is also under my control).
Could something have changed at a lower level by the app engine team to reduce the length of time before a timeout exception is thrown?
Thanks
Related
I am finding my database is the bottleneck in my application, as part of this it looks like Prepared statements are not being reused.
For example here method I use
public static CoverImage findCoverImageBySource(Session session, String src)
{
try
{
Query q = session.createQuery("from CoverImage t1 where t1.source=:source");
q.setParameter("source", src, StandardBasicTypes.STRING);
CoverImage result = (CoverImage)q.setMaxResults(1).uniqueResult();
return result;
}
catch (Exception ex)
{
MainWindow.logger.log(Level.SEVERE, ex.getMessage(), ex);
}
return null;
}
But using Yourkit profiler it says
com.mchange.v2.c3po.impl.NewProxyPreparedStatemtn.executeQuery() Count 511
com.mchnage.v2.c3po.impl.NewProxyConnection.prepareStatement() Count 511
and I assume that the count for prepareStatement() call should be lower, ais it is looks like we create a new prepared statment every time instead of reusing.
https://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html
I am using C3po connecting poolng wehich complicates things a little, but as I understand it I have it configured correctly
public static Configuration getInitializedConfiguration()
{
//See https://www.mchange.com/projects/c3p0/#hibernate-specific
Configuration config = new Configuration();
config.setProperty(Environment.DRIVER,"org.h2.Driver");
config.setProperty(Environment.URL,"jdbc:h2:"+Db.DBFOLDER+"/"+Db.DBNAME+";FILE_LOCK=SOCKET;MVCC=TRUE;DB_CLOSE_ON_EXIT=FALSE;CACHE_SIZE=50000");
config.setProperty(Environment.DIALECT,"org.hibernate.dialect.H2Dialect");
System.setProperty("h2.bindAddress", InetAddress.getLoopbackAddress().getHostAddress());
config.setProperty("hibernate.connection.username","jaikoz");
config.setProperty("hibernate.connection.password","jaikoz");
config.setProperty("hibernate.c3p0.numHelperThreads","10");
config.setProperty("hibernate.c3p0.min_size","1");
//Consider that if we have lots of busy threads waiting on next stages could we possibly have alot of active
//connections.
config.setProperty("hibernate.c3p0.max_size","200");
config.setProperty("hibernate.c3p0.max_statements","5000");
config.setProperty("hibernate.c3p0.timeout","2000");
config.setProperty("hibernate.c3p0.maxStatementsPerConnection","50");
config.setProperty("hibernate.c3p0.idle_test_period","3000");
config.setProperty("hibernate.c3p0.acquireRetryAttempts","10");
//Cancel any connection that is more than 30 minutes old.
//config.setProperty("hibernate.c3p0.unreturnedConnectionTimeout","3000");
//config.setProperty("hibernate.show_sql","true");
//config.setProperty("org.hibernate.envers.audit_strategy", "org.hibernate.envers.strategy.ValidityAuditStrategy");
//config.setProperty("hibernate.format_sql","true");
config.setProperty("hibernate.generate_statistics","true");
//config.setProperty("hibernate.cache.region.factory_class", "org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory");
//config.setProperty("hibernate.cache.use_second_level_cache", "true");
//config.setProperty("hibernate.cache.use_query_cache", "true");
addEntitiesToConfig(config);
return config;
}
Using H2 1.3.172, Hibernate 4.3.11 and the corresponding c3po for that hibernate version
With reproducible test case we have
HibernateStats
HibernateStatistics.getQueryExecutionCount() 28
HibernateStatistics.getEntityInsertCount() 119
HibernateStatistics.getEntityUpdateCount() 39
HibernateStatistics.getPrepareStatementCount() 189
Profiler, method counts
GooGooStaementCache.aquireStatement() 35
GooGooStaementCache.checkInStatement() 189
GooGooStaementCache.checkOutStatement() 189
NewProxyPreparedStatement.init() 189
I don't know what I shoud be counting as creation of prepared statement rather than reusing an existing prepared statement ?
I also tried enabling c3p0 logging by adding a c3p0 logger ands making it use same log file in my LogProperties but had no effect.
String logFileName = Platform.getPlatformLogFolderInLogfileFormat() + "songkong_debug%u-%g.log";
FileHandler fe = new FileHandler(logFileName, LOG_SIZE_IN_BYTES, 10, true);
fe.setEncoding(StandardCharsets.UTF_8.name());
fe.setFormatter(new com.jthink.songkong.logging.LogFormatter());
fe.setLevel(Level.FINEST);
MainWindow.logger.addHandler(fe);
Logger c3p0Logger = Logger.getLogger("com.mchange.v2.c3p0");
c3p0Logger.setLevel(Level.FINEST);
c3p0Logger.addHandler(fe);
Now that I have eventually got c3p0Based logging working and I can confirm the suggestion of #Stevewaldman is correct.
If you enable
public static Logger c3p0ConnectionLogger = Logger.getLogger("com.mchange.v2.c3p0.stmt");
c3p0ConnectionLogger.setLevel(Level.FINEST);
c3p0ConnectionLogger.setUseParentHandlers(false);
Then you get log output of the form
24/08/2019 10.20.12:BST:FINEST: com.mchange.v2.c3p0.stmt.DoubleMaxStatementCache ----> CACHE HIT
24/08/2019 10.20.12:BST:FINEST: checkoutStatement: com.mchange.v2.c3p0.stmt.DoubleMaxStatementCache stats -- total size: 347; checked out: 1; num connections: 13; num keys: 347
24/08/2019 10.20.12:BST:FINEST: checkinStatement(): com.mchange.v2.c3p0.stmt.DoubleMaxStatementCache stats -- total size: 347; checked out: 0; num connections: 13; num keys: 347
making it clear when you get a cache hit. When there is no cache hit yo dont get the first line, but get the other two lines.
This is using C3p0 9.2.1
I was trying to execute some code in the first second after every new Minute in my AngularJs Website.
Now i got the following code:
let msToNextMin: number = moment().endOf('minute').diff(moment());
console.log("msToNextMin: " + msToNextMin);
this.$timeout(() => {
this.$interval(() => {
console.log(moment().toString() + "New Minute started now");
}, 60000);
}, msToNextMin + 1000);
This code works how it is but it leaves the first minute out.
If I start the code at:
13:09:33 output will be:
msToNextMin: 27215
that's all right but the next output is always:
Thu Oct 20 2016 13:11:01 GMT+0200New Minute started now
Somebody has an explanation for this issue?
I am implementing a function to have a countdown in Angular form current time - existing time in future. If the time has elapsed then display a message. Timer ran out in ..... HH:MM:SS
The end time. Lets call it endTime eg:
9/15/2016 9:16:00 PM
Current time. Time current moment we live.
Lets call it currentTime.
The goal is to get a timer that is Current time - end time. Save it to a Variable TotalHours.
Then calculate the time remaining for NOW to total hours. For example TotalHours = 5. And NOW is 9/14/2016 1:16:00 PM then FinalCountDown = 6:16:00 PM. That is the timer I want running...
Here is how I am doing it...
if (info.endTime) {
var CurrentTime = new Date().toLocaleString('en-US');
moment.locale(); // en
var TotalHours = moment.utc(moment(info.diffTime, "DD/MM/YYYY HH:mm:ss").diff(moment(CurrentTime, "DD/MM/YYYY HH:mm:ss"))).format("HH:mm:ss");
info.finalCountDown= TotalHours;
};
The issue here is the following:
Case 1:
endTime = 9/15/2016 9:16:00 AM
currentTime = 9/15/2016 1:21:00 PM
TotalHours = 4:05:00
But... if its after next 2 days...
Case 2:
endTime = 9/17/2016 9:16:00 AM
currentTime = 9/15/2016 1:21:00 PM
TotalHours = 4:05:00
Total hours is still the same...
I need it to add 24hours + 24 hours + extra time = 48 + 4:05:00 = 52:05:00
also I want it to display as: 52h:05m:00s
Please let me know how to solve this...
A quick and dirty solution would be to simply convert the difference between the two date/time objects to milliseconds and then do some math on the milliseconds and format the output as follows:
var currentTime = new Date("9-15-2016 13:21:00");
var endTime = new Date("9-17-2016 09:16:00");
var ms = (endTime - currentTime); // ms of difference
var days = Math.round(ms/ 86400000);
var hrs = Math.round((ms% 86400000) / 3600000);
var mins = Math.round(((ms% 86400000) % 3600000) / 60000);
$scope.finalCountdown = (days + "d:" + hrs + " h:" + mins + "m left");
You could add in a calculation for the seconds if you needed and you can do some formatting of the numbers to have leading zeros.
However, doing this doesn't account for issues such as leap-years and other data and time anomalies. A better suggestion would be to use angular-moment which utilizes Moment.js as it can handle differences and formatting with ease.
I have following scenario which has two requests (RequestOne and RequestTwo). It is setup to run for 3 users and 1 repetition. The simulation should have taken at least 20 seconds to finish as I am using 20 seconds as pacing. However, every time I run it, it finishes in less than 20 seconds. I tried with different values for pacing as well.
val Workload = scenario("Load Test")
.repeat(1, "repetition") {
pace(20 seconds)
.exitBlockOnFail {
.feed(requestIdFeeder)
.group("Load Test") {
.exec(session => {
session.set("url", spURL)
})
.group("RequestOne") {exec(requestOne)}
.feed(requestIdFeeder)
.group("RequestTwo") {exec(requestTwo)}
}
}
}
setUp(Workload.inject(atOnceUsers(3))).protocols(httpProtocol)
output
Simulation com.performance.LoadTest completed in 11 seconds
Found the problem. I used only 1 repetition so the scenario didn't need to wait for the 20sec pacing to complete and it exited early. Setting repetition to > 1 helped achieve the desired rate.
val Workload = scenario("Load Test")
.repeat(10, "repetition") {
pace(20 seconds)
.exitBlockOnFail {
So, if you want to achieve fixed number of transactions in your simulation, use repetition, otherwise use "forever (" as mentioned in gatling docoumentation to achieve consistent rate.
val Workload = scenario("Load Test")
.forever (
pace(20 seconds)
.exitBlockOnFail {
I have solr installed on my localhost.
I started standard solr cloud example with embedded zookeepr.
collection: gettingstarted
shards: 2
replication : 2
500 records/docs to process time took 115 seconds[localhost tetsing] -
why is this taking this much time to process just 500 records.
is there a way to improve this to some millisecs/nanosecs
NOTE:
I have tested the same on remote machine solr instance, localhost having data index on remote solr [inside java commented]
I started my solr myCloudData collection with Ensemble with single zookeepr.
2 solr nodes,
1 Ensemble zookeeper standalone
collection: myCloudData,
shards: 2,
replication : 2
Solr colud java code
package com.test.solr.basic;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.CloudSolrClient;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.common.SolrInputDocument;
public class SolrjPopulatorCloudClient2 {
public static void main(String[] args) throws IOException,SolrServerException {
//String zkHosts = "64.101.49.57:2181/solr";
String zkHosts = "localhost:9983";
CloudSolrClient solrCloudClient = new CloudSolrClient(zkHosts, true);
//solrCloudClient.setDefaultCollection("myCloudData");
solrCloudClient.setDefaultCollection("gettingstarted");
/*
// Thread Safe
solrClient = new ConcurrentUpdateSolrClient(urlString, queueSize, threadCount);
*/
// Depreciated - client
//HttpSolrServer server = new HttpSolrServer("http://localhost:8983/solr");
long start = System.nanoTime();
for (int i = 0; i < 500; ++i) {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
solrCloudClient.add(doc);
if (i % 100 == 0)
System.out.println(" Every 100 records flush it");
solrCloudClient.commit(); // periodically flush
}
solrCloudClient.commit();
solrCloudClient.close();
long end = System.nanoTime();
long seconds = TimeUnit.NANOSECONDS.toSeconds(end - start);
System.out.println(" All records are indexed, took " + seconds + " seconds");
}
}
You are committing every new document, which is not necessary. It will run a lot faster if you change the if (i % 100 == 0) block to read
if (i % 100 == 0) {
System.out.println(" Every 100 records flush it");
solrCloudClient.commit(); // periodically flush
}
On my machine, this indexes your 500 records in 14 seconds. If I remove the commit() call from the for loop, it indexes in 7 seconds.
Alternatively, you can add a commitWithinMs parameter to the solrCloudClient.add() call:
solrCloudClient.add(doc, 15000);
This will guarantee your records are committed within 15 seconds, and also increase your indexing speed.