I'm running the below query through jdbcTemplate.update()
"UPDATE <tblnme> as aa set _r=b._r from(SELECT id,parent,LEAD(id,1) OVER (partition by parent ORDER BY id) _r from <tblnme>) as b where aa.id=b.id"
This update (1 table - 28 million records) runs fine around 15 min on below m.c config
SSD 32 GB RAM Intel(R) Core(TM) i7-7700HQ CPU # 2.80GHz 2.80 GHz
but gets stuck in below m.c config
HHD 32 GB RAM Intel(R) Core(TM) i7-6700 CPU # 3.40GHz 3.41 GHz
There's no message in postgres server log during the stuck phase. Had to terminate after 4 hrs wait.
have set below in postgresql.conf in both m/c:
a. max_connections = 375 b. shared_buffers = 2500MB c. max_wal_size
= 16GB d. min_wal_size = 4096MB
What could be done to resolve this issue?
Related
I am using mongodb version 2.4 due to 32 bit system limitation (nanopi m1 plus). I have a debian jessie OS image (Debian 8) with 4.2 GB space available (emmc). I have a total of about 2.2 GB available after loading my application files. However, my flash gets filled up quickly to 100% after I start running my application.
Then I get the error "Unable to get database instance and mongodb stopped working " and my application stops working.
Can someone please help me with this problem. Thanks in advance!
Memory status of my device when it stopped working:
df –h:
Filesystem- overlay Size- 4.2G Used- 4.2G Avail- 0 Use%- 100% Mounted on- /
df -h command
du –shx /var/lib/mongodb/ | sort –rh | head –n 20
512M /var/lib/mongodb/xyz.6
512M /var/lib/mongodb/xyz.5
257M /var/lib/mongodb/xyz.4
128M /var/lib/mongodb/xyz.3
64M /var/lib/mongodb/xyz.2
32M /var/lib/mongodb/xyz.1
17M /var/lib/mongodb/xyz.ns
17M /var/lib/mongodb/xyz.0
16M /var/lib/mongodb/local.ns
16M /var/lib/mongodb/local.0
4.0K /var/lib/mongodb/journal
0 /var/lib/mongodb/mongodb.lock
du –shx /var/lib/mongodb/journal/* | sort –rh | head –n 20
257M /var/lib/mongodb/journal/prealloc.2
257M /var/lib/mongodb/journal/prealloc.1
257M /var/lib/mongodb/journal/prealloc.0
du –shx /var/log/mongodb/journal/* | sort –rh | head –n 20
399M /var/lib/mongodb/mongodb.log
353M /var/lib/mongodb/mongodb.log.1
3.3M /var/lib/mongodb/mongodb.log.2.gz
752K /var/lib/mongodb/mongodb.log.1.gz
How can I track down this dump error?
And most important, which process is causing it?
What are the consequences?
It happens almost every weekend:
See sql dump output below:
*Current time is 23:26:40 11/05/17.
=====================================================================
BugCheck Dump
=====================================================================
This file is generated by Microsoft SQL Server
version 13.0.4446.0
upon detection of fatal unexpected error. Please return this file,
the query or program that produced the bugcheck, the database and
the error log, and any other pertinent information with a Service Request.
Computer type is Intel(R) Xeon(R) CPU E5-2698B v3 # 2.00GHz.
Bios Version is VRTUAL - 5001223
Intel(R) Xeon(R) CPU E5-2698B v3 # 2.00GHz
4 X64 level 8664, 10 Mhz processor (s).
Windows NT 6.2 Build 9200 CSD .
Memory
MemoryLoad = 96%
Total Physical = 32767 MB
Available Physical = 994 MB
Total Page File = 39679 MB
Available Page File = 5602 MB
Total Virtual = 134217727 MB
Available Virtual = 134132460 MB
**Dump thread - spid = 0, EC = 0x000001DE6E277240
***Stack Dump being sent to C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\LOG\SQLDump0006.txt
* *******************************************************************************
*
* BEGIN STACK DUMP:
* 11/05/17 23:26:40 spid 38
*
* Latch timeout
*
*
* *******************************************************************************
* -------------------------------------------------------------------------------
* Short Stack Dump*
You can use SQL Server Diagnostics (Preview) available as an extension from SSMS 17.1 onwards to check for any potential causes and any available resolutions
After installing you will find a screen like below and after uploading dump, you can find potential solutions or patches which may help you.Ensure you upload DUMP to a location near to you
You also can load the dump using windbg and play with it if you have right symbols..Further event logs can show you more info
Most of the times Stackdumps are dumped due to bugs..Best way to proceed with them is to raise a ticket with microsoft.
Elapsed time: 0 hours 0 minu
tes 6 seconds. Internal database snapshot has split point LSN = 00014377:000000a5:0001 and first LSN = 00014377:
000000a3:0001.
repair_allow_data_loss is the minimum repair level for the errors found by DBCC CHECKDB (master.
**Dump thread - spid = 0, EC = 0x0000022824F95600
You nee check dump file where stack dump detail provides.
We are sharing small piece of code for analysis.
In my server-status apache2.4 in ubuntu16.04 i have many G status. somtimes most of 3000 thread is G . Some queries failed.
High G status is problem or not related?
server: 32 core cpu and 128GB RAM
server status :
Server load: 21.45 22.57 22.66
Total accesses: 1153411 - Total Traffic: 11.5 GB
CPU Usage: u322.14 s2221.14 cu0 cs0 - 24.1% CPU load
109 requests/sec - 1.1 MB/second - 10.4 kB/request
900 requests currently being processed, 60 idle workers
mpm_event config:
ServerLimit 140
StartServers 30
MinSpareThreads 35
MaxSpareThreads 80
# ThreadLimit 64
ThreadsPerChild 30
MaxRequestWorkers 4000
MaxConnectionsPerChild 500
If you don't use the event MPM, these tie up a thread. "MaxConnectionsPerChild 500" is exceptionally low and under any non-trivial load you will be constantly killing off processes which will send all their active requests into 'G' state.
I have compiled Redis 3.0.6 to in an AIX 6.1 ppc systems using gcc, but it crashes when the database tries to exceed 2GB of RAM.
The compiler flags and environment variables are:
alias make="gmake"
export CC=gcc
export CFLAGS="-DAIX64 -DBIT64 -maix64 -D_AIX -D_AIX53 -D_AIX61 -I/opt/freeware/include -I/usr/include${CFLAGS_XTRA}"
export CXX=g++
export CXXFLAGS=$CFLAGS
export LDFLAGS="-DAIX64 -DBIT64 -maix64 -L/opt/freeware/lib64 -L/opt/freeware/lib -Wl,-blibpath:/opt/freeware/lib64:/opt/freeware/lib/pthread/ppc64:/opt/freeware/lib:/usr/lib:/lib,-bmaxdata:0x80000000"
export AR="ar -X64"
export LIBPATH=.:/opt/freeware/lib64:/opt/freeware/lib:/usr/lib:/lib
export OBJECT_MODE=64
echo Gcc environment variables \(CC,CFLAGS,LDFLAGS,LIBPATH,etc\) are set.
But when it reaches 2GB of RAM it crashes with the following log:
41680910:M 12 Jan 12:46:13.056 # Out Of Memory allocating 102709 bytes!
41680910:M 12 Jan 12:46:13.056 # === REDIS BUG REPORT START: Cut & paste starting from here ===
41680910:M 12 Jan 12:46:13.056 # ------------------------------------------------
41680910:M 12 Jan 12:46:13.056 # !!! Software Failure. Press left mouse button to continue
41680910:M 12 Jan 12:46:13.057 # Guru Meditation: "Redis aborting for OUT OF MEMORY" #redis.c:3572
41680910:M 12 Jan 12:46:13.057 # ------------------------------------------------
It also crashes in I try to execute
redis-server --test-memory 2048
But not if I execute
redis-server --test-memory 2047
So clearly it has to be with a 2GB limit that I can't find.
Any ideas or further information required?
Try omitting -bmaxdata:0x80000000. On 64-bit builds, this parameter limits the heap size to 2GB. On 32-bit builds, it will increase the low default to 2GB.
First I'll give a run down on what I did.
I downloaded dhry.h dhry_1.c and dhry_2.c from here:
http://giga.cps.unizar.es/~spd/src/other/dhry/
Then I made some corrections (so that I would compile) according to this:
https://github.com/maximeh/buildroot/blob/master/package/dhrystone/dhrystone-2-HZ.patch
And this
Errors while compiling dhrystone in unix
I've compiled the files with the following command line:
gcc dhry_1.c dhry_2.c -O2 -o run
I finally entered the number of runs to be 1000000000
And waited. I compiled with four different optimization levels and I got these values of DMIPS (According to http://en.wikipedia.org/wiki/Dhrystone this is the Dhrystones per Second divided by 1757):
O0: 8112 O1: 16823.9 O2: 22977.5 O3: 23164.5 (these represent the compiler flags like -O2 is optimization level two and O0 is none).
This would give the following DMIPS/MHz (base frequency for my processor is 3.4 GHz):
2.3859 4.9482 6.7581 6.8131
However, I get the feelign tha 6.7 is way to low. According to what I've read an A15 has between 3.5 to 4 DMIPS/MHz and a third generation I7 only has double that? Shouldn't it be a lot higher?
Can anyone tell me from my procedure if they can see that I might have done something wrong? Or maybe I'm interpretting the results incorrectly?
Except with a broad brush treatment, you cannot compare benchmark results produced by different compilers. As the design authority of the first standard benchmark (Whetstone), I can advise that it even less safe to include comparisons with results from a computer manufacturer’s in-house compiler. In minicomputer days, manufacturers found that sections of the Whetstone benchmark could be optimised out, to double the score. I arranged for changes and more detailed results to avoid and later highlight over optimisation.
Below are example results on PCs from my original (1990’s) Dhrystone Benchmarks. For details, more results and (free) execution and source files see:
http://www.roylongbottom.org.uk/dhrystone%20results.htm
Also included, and compiled from the same source code, are results from a later MS compiler and some via Linux and on Android, via ARM CPUs, plus one for an Intel Atom, via Houdini compatibility layer. I prefer the term VAX MIPS instead to DMIPS, as the 1757 divisor is the result on DEC VAX 11/780. Anyway, MIPS/MHz calculations are also shown. Note differences due to compilers and the particularly low ratios on Android ARM CPUs.
Dhry1 Dhry1 Dhry2 Dhry2 Dhry2
Opt NoOpt Opt NoOpt Opt
VAX VAX VAX VAX MIPS/
CPU MHz MIPS MIPS MIPS MIPS MHz
AMD 80386 40 17.5 4.32 13.7 4.53 0.3
80486 DX2 66 45.1 12 35.3 12.4 0.5
Pentium 100 169 31.8 122 32.2 1.2
Pentium Pro 200 373 92.4 312 91.9 1.6
Pentium II 300 544 132 477 136 1.6
Pentium III 450 846 197 722 203 1.6
Pentium 4 1900 2593 261 2003 269 1.1
Atom 1666 2600 772 1948 780 1.2
Athlon 64 2211 5798 1348 4462 1312 2.0
Core 2 Duo 1 CP 2400 7145 1198 6446 1251 2.7
Phenom II 1 CP 3000 9462 2250 7615 2253 2.5
Core i7 4820K 3900 14776 2006 11978 2014 3.1
Later Intel Compiler
Pentium 4 1900 2613 1795 0.9
Athlon 64 2211 6104 3720 1.7
Core 2 Duo 2400 8094 5476 2.3
Phenom II 3000 9768 6006 2.0
Core i7 4820K 3900 15587 10347 2.7
Linux Ubuntu GCC Compiler
Atom 1666 5485 1198 2055 1194 1.2
Athlon 64 2211 9034 2286 4580 2347 2.1
Core 2 Duo 2400 13599 3428 5852 3348 2.4
Phenom II 3000 13406 3368 6676 3470 2.2
Core i7 4820K 3900 29277 7108 16356 7478 4.2
ARM Android NDK
926EJ 800 356 196 0.4
v7-A9 1500 1650 786 1.1
v7-A15 1700 3189 1504 1.9
Atom Houdini 1866 1840 1310 1.0