Redis using high memory with small data set - database

I'm running redis-server and according to prstat it's utilizing 3721M of memory. When I run info in redis-cli, I get this for memory:
used_memory:8739028
used_memory_human:8.33M
used_memory_rss:8739028
mem_fragmentation_ratio:1.00
This is running on a 2GB cloud instance, so my memory utilization is always over. However, even when I flushall on redis it doesn't seem to effect the memory consumption.
Could this be do to a setting that maybe is allocating memory or any idea why redis is using so much over my actual data?
Thank you!
EDIT:
Also just to add, I'm running the same setup (application/redis) on another instance and the memory consumption is only 554M with used_memory_human:5.68M of data.
Response from pmap:
[root#]# pmap -x 27425
27425: /opt/local/bin/redis-server /opt/local/etc/redis.conf
Address Kbytes RSS Anon Locked Mode Mapped File
08046000 8 8 8 - rw--- [ stack ]
08050000 208 160 - - r-x-- redis-server
08093000 8 8 4 - rwx-- redis-server
08095000 3807836 3804160 3764928 - rwx-- [ heap ]
FE9BD000 12 12 - - r-x-- libpthread.so.1
FEAF0000 456 456 - - r-x-- libnsl.so.1
FEB72000 8 8 4 - rw--- libnsl.so.1
FEB74000 20 16 4 - rw--- libnsl.so.1
FEBA0000 304 304 - - r-x-- libm.so.2
FEBFB000 16 16 - - rwx-- libm.so.2
FEDA0000 4 4 - - r--s- dev:531,392 ino:3736139179
FEDB0000 4 4 - - rwxs- [ anon ]
FEDC0000 24 12 8 - rwx-- [ anon ]
FEDD0000 4 4 4 - rwx-- [ anon ]
FEDE0000 1216 1216 - - r-x-- libc.so.1
FEF10000 36 36 24 - rwx-- libc.so.1
FEF19000 8 8 4 - rwx-- libc.so.1
FEF20000 4 4 4 - rwx-- [ anon ]
FEF30000 56 56 - - r-x-- libsocket.so.1
FEF4E000 4 4 - - rw--- libsocket.so.1
FEF50000 4 4 4 - rwx-- [ anon ]
FEF60000 4 4 - - r-x-- libdl.so.1
FEF70000 4 4 - - r--s- ld.config
FEF80000 4 4 4 - rw--- [ anon ]
FEF90000 4 - - - rw--- [ anon ]
FEFA0000 4 4 4 - rwx-- [ anon ]
FEFB0000 4 4 - - rwx-- [ anon ]
FEFB7000 208 208 - - r-x-- ld.so.1
FEFFB000 8 8 4 - rwx-- ld.so.1
FEFFD000 4 4 - - rwx-- ld.so.1
The limits has nothing enabled in the config, here's virtual memory and advanced:
VIRTUAL MEMORY
vm-enabled no
vm-swap-file /tmp/redis.swap
vm-max-memory 0
vm-page-size 32
vm-pages 134217728
vm-max-threads 4
ADVANCED CONFIG
hash-max-zipmap-entries 512
hash-max-zipmap-value 64

Question is a bit old but I had this issue before. If I remember correctly, I shutdown my redis instance, deleted the RDB file if applicable and restart redis. If don't want to remove the RDB file because it contains data you need, I do NOT recommend this. You can try to DEL what you can, then do a SAVE (I believe redis does SAVE when it's shutdown so that might be redundant) and then restart.
It would be good time to upgrade redis to latest stable if you haven't.

Related

how long takes mariadb memory to stabilize?

5.13
We made tweaks on my.cnf according to mysql tuner, and tuning primer,
but i can't really figure out when the memory will be stabilize usage of mariadb.
Here mysql tuner result -
------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 5d 8h 19m 28s (226M q [491.299 qps], 1M conn, TX: 4653G, RX: 49G)
[--] Reads / Writes: 95% / 5%
[--] Binary logging is disabled
[--] Physical Memory : 125.3G
[--] Max MySQL memory : 67.1G
[--] Other process memory: 0B
[--] Total buffers: 15.4G global + 264.8M per thread (200 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 34.3G (27.34% of installed RAM)
[OK] Maximum possible memory usage: 67.1G (53.54% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (78/226M)
[OK] Highest usage of available connections: 36% (73/200)
[OK] Aborted connections: 0.02% (447/1823123)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[OK] Query cache is disabled by default due to mutex contention on multiprocessor machines.
[OK] Sorts requiring temporary tables: 0% (4K temp sorts / 56M sorts)
[!!] Joins performed without indexes: 815533
[!!] Temporary tables created on disk: 87% (26M on disk / 30M total)
[OK] Thread cache hit rate: 99% (73 created / 1M connections)
[OK] Table cache hit rate: 99% (286M hits / 287M requests)
[OK] table_definition_cache(37000) is upper than number of tables(36204)
[OK] Open file limit used: 69% (51K/74K)
[OK] Table locks acquired immediately: 99% (170M immediate / 170M locks)
Here tuning primer -
WORKER THREADS
Current thread_cache_size = 200
Current threads_cached = 72
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine
MAX CONNECTIONS
Current max_connections = 200
Current threads_connected = 1
Historic max_used_connections = 73
The number of used connections is 36% of the configured maximum.
Your max_connections variable seems to be fine.
INNODB STATUS
Current InnoDB index space = 1.46 G
Current InnoDB data space = 10.79 G
Current InnoDB buffer pool free = 24 %
Current innodb_buffer_pool_size = 15.00 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory
MEMORY USAGE
Max Memory Ever Allocated : 15.76 G
Configured Max Per-thread Buffers : 1.71 G
Configured Max Global Buffers : 15.14 G
Configured Max Memory Limit : 16.85 G
Physical Memory : 125.34 G
Max memory limit seem to be within acceptable norms
Here a sample of resources from top since start up until 5 days and 8 hours uptime of mariadb -
mysql 20 0 21.4g 4.1g 24576 S 4.7 3.3 0:08.90 mariadbd
mysql 20 0 21.4g 4.8g 25728 S 4.7 3.8 2:13.62 mariadbd
mysql 20 0 25.9g 8.7g 26480 S 15.0 6.9 56:52.91 mariadbd
mysql 20 0 26.2g 8.8g 26480 S 3.7 7.0 58:37.38 mariadbd
mysql 20 0 26.7g 9.3g 26480 S 10.6 7.4 66:38.53 mariadbd
mysql 20 0 26.7g 9.3g 26480 S 6.3 7.4 76:34.20 mariadbd
mysql 20 0 26.9g 9.6g 26480 S 10.0 7.7 80:31.37 mariadbd
mysql 20 0 26.9g 9.8g 26480 S 10.3 7.8 95:36.03 mariadbd
mysql 20 0 27.4g 10.2g 26480 S 8.0 8.1 96:28.67 mariadbd
mysql 20 0 27.7g 12.4g 26496 S 2.7 9.9 127:20.78 mariadbd
mysql 20 0 27.7g 15.0g 26496 S 11.3 12.0 129:35.15 mariadbd
mysql 20 0 27.7g 16.3g 26492 S 4.7 13.0 130:34.33 mariadbd
mysql 20 0 27.7g 16.3g 26492 S 22.9 13.0 135:47.21 mariadbd
mysql 20 0 27.7g 16.4g 26460 S 13.3 13.1 182:57.25 mariadbd
mysql 20 0 27.7g 16.9g 26436 S 5.0 13.5 186:18.78 mariadbd
mysql 20 0 28.2g 17.4g 26436 S 27.9 13.9 220:29.14 mariadbd
mysql 20 0 28.2g 17.4g 26436 S 10.3 13.9 240:20.25 mariadbd
mysql 20 0 28.6g 17.8g 26436 S 15.9 14.2 242:50.85 mariadbd
mysql 20 0 28.6g 17.9g 26436 S 6.6 14.2 244:48.40 mariadbd
mysql 20 0 29.0g 18.3g 26436 S 5.0 14.6 271:35.53 mariadbd
mysql 20 0 29.4g 18.5g 26480 S 11.6 14.7 327:44.08 mariadbd
mysql 20 0 29.8g 18.9g 26480 S 21.6 15.1 328:47.99 mariadbd
mysql 20 0 29.8g 18.9g 26480 S 3.7 15.1 340:27.10 mariadbd
mysql 20 0 29.8g 18.9g 26480 S 7.0 15.1 360:15.26 mariadbd
mysql 20 0 30.2g 19.6g 26460 S 3.6 15.7 397:31.67 mariadbd
mysql 20 0 31.4g 20.1g 26460 S 6.6 16.1 424:18.82 mariadbd
mysql 20 0 31.8g 20.6g 26460 S 1.7 16.5 434:33.15 mariadbd
mysql 20 0 31.8g 20.7g 26460 S 12.3 16.5 474:25.23 mariadbd
mysql 20 0 32.1g 21.1g 26444 S 9.6 16.9 501:38.46 mariadbd
mysql 20 0 32.1g 21.1g 26448 S 14.0 16.9 509:05.98
mysql 20 0 32.5g 21.6g 26448 S 10.3 17.2 524:06.26 mariadbd
mysql 20 0 33.0g 22.0g 26448 S 7.0 17.6 527:11.55 mariadbd
mysql 20 0 33.5g 22.5g 25612 S 29.9 17.9 578:12.81 mariadbd
mysql 20 0 33.5g 22.5g 25612 S 2.3 18.0 580:04.96 mariadbd
mysql 20 0 34.4g 23.4g 25044 S 17.9 18.7 647:25.39 mariadbd
mysql 20 0 35.3g 24.5g 25272 S 10.0 19.5 753:34.30 mariadbd
mysql 20 0 35.7g 24.9g 25272 S 10.0 19.9 766:17.57 mariadbd
if im trying to determine the memory usage via service status the following result -
service mariadb status
Redirecting to /bin/systemctl status mariadb.service
● mariadb.service - MariaDB 10.5.13 database server
Active: active (running) since Mon 2022-01-03 03:03:15 IST; 5 days ago
Docs: man:mariadbd(8)
https://mariadb.com/kb/en/library/systemd/
Main PID: (mariadbd)
Status: "Taking your SQL requests now..."
Tasks: 87 (limit: 820947)
Memory: 37.4G
now i just see so many indiffrent current memory usage, and for example tuning primer returns -
Configured Max Memory Limit : 16.85 G
but mysql tuner returns -
[OK] Maximum reached memory usage: 34.3G (27.34% of installed RAM)
[OK] Maximum possible memory usage: 67.1G (53.54% of installed RAM)
attaching my.cnf configurations -
performance-schema=0
#innodb_buffer_pool_size=134217728
max_allowed_packet=268435456
open_files_limit=74000
innodb_file_per_table=1
unix_socket=OFF
max_connections=200
#wait_timeout=600
#interactive_timeout=600
##
innodb_buffer_pool_size=15G
max_heap_table_size=128M
tmp_table_size=128M
#max_connections=400
table_open_cache=37000
table_definition_cache=37000
thread_cache_size=200
key_buffer_size=128M
sort_buffer_size=1M
read_buffer_size=4M
read_rnd_buffer_size=512k
join_buffer_size=3M
##
But as you can see, top command returns 24.9gb ram usage, and service status returns 37.4GB ram usage? and tuning primer saying max memory is 16.85GB, but mysql tuner says 67.1GB?
I'm completly lost in this one..
when the memory will build up until getting a stabilize memory usage?
why tuning primer, and mysql tuner memory usage is so different?from the current usage of the server itself for mariadb service?
Could anyone share his tought regards it?
Thanks!
Edit -
Hey thanks for the reply, it just keep increasing after 7 and half days -
mysql 20 0 36.2g 25.4g 25280 S 10.3 20.3 811:15.91 mariadbd
mysql 20 0 36.7g 25.8g 25296 S 5.0 20.6 840:23.93 mariadbd
mysql 20 0 37.1g 26.3g 25296 S 3.3 20.9 843:11.06 mariadbd
mysql 20 0 37.1g 26.3g 25296 S 7.6 21.0 846:27.98 mariadbd
mysql 20 0 37.1g 26.5g 25296 S 14.9 21.1 898:34.45 mariadbd
mysql 20 0 37.2g 26.5g 25272 S 13.0 21.1 933:06.45 mariadbd
mysql 20 0 37.2g 26.5g 25260 S 25.0 21.2 956:38.96 mariadbd
mysql 20 0 37.6g 26.9g 25260 S 4.3 21.5 992:54.48 mariadbd
mysql 20 0 38.0g 27.4g 25252 S 17.5 21.9 1068:31 mariadbd
mariadb.service - MariaDB 10.5.13 database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/mariadb.service.d
└─migrated-from-my.cnf-settings.conf
Active: active (running) since Mon 2022-01-03 03:03:15 IST; 1 weeks 0 days ago
Docs: man:mariadbd(8)
https://mariadb.com/kb/en/library/systemd/
Main PID: 4105634 (mariadbd)
Status: "Taking your SQL requests now..."
Tasks: 86 (limit: 820947)
Memory: 41.0G
total used free shared buff/cache available
Mem: 128350 35498 9313 2356 83538 89236
Swap: 0 0 0
systemd service status 37.4G looks consistent with the 35.7g figure from top (GiB vs GB perhaps explains the differences).
The 24.9G in top is the amount paged it. The difference to 37.4G is memory not paged in (or perhaps swapped out).
Tuning primer's 16.58G looks like it hasn't accounted all threads memory usage
mysqltuner's maximum looks like it has taken the pessimistic allocation of the maximum possible per thread allocation, which isn't occurring because your workload evidently doesn't to all the things that would get to this figure.
If your 5 days of uptime is pretty consistent to your workload, then 35/37G looks to be about the maximum used.
The thing that would make it rise from this point is if your 24% free buffer pool gets used. Your tuning primer shows 24% free which on 15G gives another 3.6G of potentially memory used on our current workload.
MySQL Tuner full from around 3h ago -
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 7d 9h 28m 52s (316M q [495.709 qps], 2M conn, TX: 6533G, RX: 69G)
[--] Reads / Writes: 95% / 5%
[--] Binary logging is disabled
[--] Physical Memory : 125.3G
[--] Max MySQL memory : 67.1G
[--] Other process memory: 0B
[--] Total buffers: 15.4G global + 264.8M per thread (200 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 34.3G (27.34% of installed RAM)
[OK] Maximum possible memory usage: 67.1G (53.54% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (111/316M)
[OK] Highest usage of available connections: 36% (73/200)
[OK] Aborted connections: 0.02% (605/2516029)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[OK] Query cache is disabled by default due to mutex contention on multiprocessor machines.
[OK] Sorts requiring temporary tables: 0% (6K temp sorts / 78M sorts)
[!!] Joins performed without indexes: 1135010
[!!] Temporary tables created on disk: 87% (36M on disk / 41M total)
[OK] Thread cache hit rate: 99% (73 created / 2M connections)
[OK] Table cache hit rate: 99% (398M hits / 400M requests)
[OK] table_definition_cache(37000) is upper than number of tables(36425)
[OK] Open file limit used: 69% (51K/74K)
[OK] Table locks acquired immediately: 99% (242M immediate / 242M locks)
-------- Performance schema ------------------------------------------------------------------------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.
-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 36 thread(s).
[--] Using default value is good enough for your version (10.5.13-MariaDB)
-------- MyISAM Metrics ----------------------------------------------------------------------------
[OK] Key buffer used: 99.7% (127.6M used / 128.0M cache)
[OK] Key buffer size / total MyISAM indexes: 128.0M/1.3G
[OK] Read Key buffer hit rate: 99.9% (3B cached / 1M reads)
[!!] Write Key buffer hit rate: 43.1% (31M cached / 13M writes)
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 15.0G/12.2G
[!!] Ratio InnoDB log file size / InnoDB Buffer pool size (0.625 %): 96.0M * 1/15.0G should be equal to 25%
[--] Number of InnoDB Buffer Pool Chunk : 120 for 1 Buffer Pool Instance(s)
[OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances
[OK] InnoDB Read buffer efficiency: 99.99% (7707381322 hits/ 7708055862 total)
[!!] InnoDB Write Log efficiency: 335.3% (2710443 hits/ 808371 total)
[OK] InnoDB log waits: 0.00% (0 waits / 3518814 writes)
-------- Aria Metrics ------------------------------------------------------------------------------
[--] Aria Storage Engine is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/896.0K
[OK] Aria pagecache hit rate: 97.8% (1B cached / 30M reads)
-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.
-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.
-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is disabled.
-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: MIXED
[--] XA support enabled: ON
[--] Semi synchronous replication Master: OFF
[--] Semi synchronous replication Slave: OFF
[--] This is a standalone server
Heres tops -
top - 17:06:15 up 67 days, 14:15, 1 user, load average: 2.88, 2.79, 2.77
Tasks: 560 total, 3 running, 557 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.0 us, 0.6 sy, 0.1 ni, 95.3 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 128350.7 total, 7895.4 free, 35970.4 used, 84484.8 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 89024.2 avail Mem
nothing except mariadb using alot of memory, some LSPHP proccesses spawn and go as for php requests to sites hosted within the server but they using memory only for proccesing the requests and then releasing it..
but overall cpu average is around 3-12% cpu usage, so server not under heavy load.
Server specs -
Intel xeon w-2295
software raid 1 with 2 NVMEs hard drives
OS - Cloudlinux 8.5 (based of almalinux 8.5 + redhat)
server using cPanel with mariadb 10.5.3 of cPanel.
128GB ram ecc memory
SELECT COUNT() FROM information_schema.tables;
COUNT()
94
Proccess list -
Show MySQL Processes
Id User Host db Command Time State Info Progress
root localhost NULL Query 0 starting SHOW PROCESSLIST 0.000
sometimes you see around 4-14 proccessess in the proccess lists.
Variable_name
Value
alter_algorithm
DEFAULT
analyze_sample_percentage
100.000000
aria_block_size
8192
aria_checkpoint_interval
30
aria_checkpoint_log_activity
1048576
aria_encrypt_tables
OFF
aria_force_start_after_recovery_failures
0
aria_group_commit
none
aria_group_commit_interval
0
aria_log_file_size
1073741824
aria_log_purge_type
immediate
aria_max_sort_file_size
9223372036853727232
aria_page_checksum
ON
aria_pagecache_age_threshold
300
aria_pagecache_buffer_size
134217728
aria_pagecache_division_limit
100
aria_pagecache_file_hash_size
512
aria_recover_options
BACKUP,QUICK
aria_repair_threads
1
aria_sort_buffer_size
268434432
aria_stats_method
nulls_unequal
aria_sync_log_dir
NEWFILE
aria_used_for_temp_tables
ON
auto_increment_increment
1
auto_increment_offset
1
autocommit
ON
automatic_sp_privileges
ON
back_log
90
basedir
/usr/
big_tables
OFF
bind_address
binlog_annotate_row_events
ON
binlog_cache_size
32768
binlog_checksum
CRC32
binlog_commit_wait_count
0
binlog_commit_wait_usec
100000
binlog_direct_non_transactional_updates
OFF
binlog_file_cache_size
16384
binlog_format
MIXED
binlog_optimize_thread_scheduling
ON
binlog_row_image
FULL
binlog_row_metadata
NO_LOG
binlog_stmt_cache_size
32768
bulk_insert_buffer_size
8388608
character_set_client
latin1
character_set_connection
latin1
character_set_database
latin1
character_set_filesystem
binary
character_set_results
latin1
character_set_server
latin1
character_set_system
utf8
character_sets_dir
/usr/share/mysql/charsets/
check_constraint_checks
ON
collation_connection
latin1_swedish_ci
collation_database
latin1_swedish_ci
collation_server
latin1_swedish_ci
column_compression_threshold
100
column_compression_zlib_level
6
column_compression_zlib_strategy
DEFAULT_STRATEGY
column_compression_zlib_wrap
OFF
completion_type
NO_CHAIN
concurrent_insert
AUTO
connect_timeout
10
core_file
OFF
datadir
/var/lib/mysql/
date_format
%Y-%m-%d
datetime_format
%Y-%m-%d %H:%i:%s
deadlock_search_depth_long
15
deadlock_search_depth_short
4
deadlock_timeout_long
50000000
deadlock_timeout_short
10000
debug_no_thread_alarm
OFF
default_password_lifetime
0
default_regex_flags
default_storage_engine
InnoDB
default_tmp_storage_engine
default_week_format
0
delay_key_write
ON
delayed_insert_limit
100
delayed_insert_timeout
300
delayed_queue_size
1000
disconnect_on_expired_password
OFF
div_precision_increment
4
encrypt_binlog
OFF
encrypt_tmp_disk_tables
OFF
encrypt_tmp_files
OFF
enforce_storage_engine
eq_range_index_dive_limit
200
event_scheduler
OFF
expensive_subquery_limit
100
expire_logs_days
0
explicit_defaults_for_timestamp
OFF
extra_max_connections
1
extra_port
0
flush
OFF
flush_time
0
foreign_key_checks
ON
ft_boolean_syntax
+ -><()~*:""&|
ft_max_word_len
84
ft_min_word_len
4
Variable_name
Value
ft_query_expansion_limit
20
ft_stopword_file
(built-in)
general_log
OFF
general_log_file
servername.log
group_concat_max_len
1048576
gtid_binlog_pos
gtid_binlog_state
gtid_cleanup_batch_size
64
gtid_current_pos
gtid_domain_id
0
gtid_ignore_duplicates
OFF
gtid_pos_auto_engines
gtid_slave_pos
gtid_strict_mode
OFF
have_compress
YES
have_crypt
YES
have_dynamic_loading
YES
have_geometry
YES
have_openssl
YES
have_profiling
YES
have_query_cache
YES
have_rtree_keys
YES
have_ssl
DISABLED
have_symlink
YES
histogram_size
254
histogram_type
DOUBLE_PREC_HB
host_cache_size
328
hostname
hostname
idle_readonly_transaction_timeout
0
idle_transaction_timeout
0
idle_write_transaction_timeout
0
ignore_builtin_innodb
OFF
ignore_db_dirs
in_predicate_conversion_threshold
1000
init_connect
init_file
init_slave
innodb_adaptive_flushing
ON
innodb_adaptive_flushing_lwm
10.000000
innodb_adaptive_hash_index
OFF
innodb_adaptive_hash_index_parts
8
innodb_adaptive_max_sleep_delay
0
innodb_autoextend_increment
64
innodb_autoinc_lock_mode
1
innodb_background_scrub_data_check_interval
0
innodb_background_scrub_data_compressed
OFF
innodb_background_scrub_data_interval
0
innodb_background_scrub_data_uncompressed
OFF
innodb_buf_dump_status_frequency
0
innodb_buffer_pool_chunk_size
134217728
innodb_buffer_pool_dump_at_shutdown
ON
innodb_buffer_pool_dump_now
OFF
innodb_buffer_pool_dump_pct
25
innodb_buffer_pool_filename
ib_buffer_pool
innodb_buffer_pool_instances
1
innodb_buffer_pool_load_abort
OFF
innodb_buffer_pool_load_at_startup
ON
innodb_buffer_pool_load_now
OFF
innodb_buffer_pool_size
16106127360
innodb_change_buffer_max_size
25
innodb_change_buffering
all
innodb_checksum_algorithm
full_crc32
innodb_cmp_per_index_enabled
OFF
innodb_commit_concurrency
0
innodb_compression_algorithm
zlib
innodb_compression_default
OFF
innodb_compression_failure_threshold_pct
5
innodb_compression_level
6
innodb_compression_pad_pct_max
50
innodb_concurrency_tickets
0
innodb_data_file_path
ibdata1:12M:autoextend
innodb_data_home_dir
innodb_deadlock_detect
ON
innodb_default_encryption_key_id
1
innodb_default_row_format
dynamic
innodb_defragment
OFF
innodb_defragment_fill_factor
0.900000
innodb_defragment_fill_factor_n_recs
20
innodb_defragment_frequency
40
innodb_defragment_n_pages
7
innodb_defragment_stats_accuracy
0
innodb_disable_sort_file_cache
OFF
innodb_disallow_writes
OFF
innodb_doublewrite
ON
innodb_encrypt_log
OFF
innodb_encrypt_tables
OFF
innodb_encrypt_temporary_tables
OFF
innodb_encryption_rotate_key_age
1
innodb_encryption_rotation_iops
100
innodb_encryption_threads
0
innodb_fast_shutdown
1
innodb_fatal_semaphore_wait_threshold
600
innodb_file_format
innodb_file_per_table
ON
innodb_fill_factor
100
innodb_flush_log_at_timeout
1
innodb_flush_log_at_trx_commit
1
innodb_flush_method
fsync
innodb_flush_neighbors
1
innodb_flush_sync
ON
Variable_name
Value
innodb_flushing_avg_loops
30
innodb_force_load_corrupted
OFF
innodb_force_primary_key
OFF
innodb_force_recovery
0
innodb_ft_aux_table
innodb_ft_cache_size
8000000
innodb_ft_enable_diag_print
OFF
innodb_ft_enable_stopword
ON
innodb_ft_max_token_size
84
innodb_ft_min_token_size
3
innodb_ft_num_word_optimize
2000
innodb_ft_result_cache_limit
2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree
2
innodb_ft_total_cache_size
640000000
innodb_ft_user_stopword_table
innodb_immediate_scrub_data_uncompressed
OFF
innodb_instant_alter_column_allowed
add_drop_reorder
innodb_io_capacity
200
innodb_io_capacity_max
2000
innodb_large_prefix
innodb_lock_schedule_algorithm
fcfs
innodb_lock_wait_timeout
50
innodb_log_buffer_size
16777216
innodb_log_checksums
ON
innodb_log_compressed_pages
ON
innodb_log_file_size
100663296
innodb_log_files_in_group
1
innodb_log_group_home_dir
./
innodb_log_optimize_ddl
OFF
innodb_log_write_ahead_size
8192
innodb_lru_flush_size
32
innodb_lru_scan_depth
1536
innodb_max_dirty_pages_pct
90.000000
innodb_max_dirty_pages_pct_lwm
0.000000
innodb_max_purge_lag
0
innodb_max_purge_lag_delay
0
innodb_max_purge_lag_wait
4294967295
innodb_max_undo_log_size
10485760
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_old_blocks_pct
37
innodb_old_blocks_time
1000
innodb_online_alter_log_max_size
134217728
innodb_open_files
36885
innodb_optimize_fulltext_only
OFF
innodb_page_cleaners
1
innodb_page_size
16384
innodb_prefix_index_cluster_optimization
OFF
innodb_print_all_deadlocks
OFF
innodb_purge_batch_size
300
innodb_purge_rseg_truncate_frequency
128
innodb_purge_threads
4
innodb_random_read_ahead
OFF
innodb_read_ahead_threshold
56
innodb_read_io_threads
4
innodb_read_only
OFF
innodb_replication_delay
0
innodb_rollback_on_timeout
OFF
innodb_scrub_log
OFF
innodb_scrub_log_speed
256
innodb_sort_buffer_size
1048576
innodb_spin_wait_delay
4
innodb_stats_auto_recalc
ON
innodb_stats_include_delete_marked
OFF
innodb_stats_method
nulls_equal
innodb_stats_modified_counter
0
innodb_stats_on_metadata
OFF
innodb_stats_persistent
ON
innodb_stats_persistent_sample_pages
20
innodb_stats_traditional
ON
innodb_stats_transient_sample_pages
8
innodb_status_output
OFF
innodb_status_output_locks
OFF
innodb_strict_mode
ON
innodb_sync_array_size
1
innodb_sync_spin_loops
30
innodb_table_locks
ON
innodb_temp_data_file_path
ibtmp1:12M:autoextend
innodb_thread_concurrency
0
innodb_thread_sleep_delay
0
innodb_tmpdir
innodb_undo_directory
./
innodb_undo_log_truncate
OFF
innodb_undo_logs
128
innodb_undo_tablespaces
0
innodb_use_atomic_writes
ON
innodb_use_native_aio
ON
innodb_version
10.5.13
innodb_write_io_threads
4
interactive_timeout
28800
join_buffer_size
3145728
join_buffer_space_limit
2097152
join_cache_level
2
keep_files_on_create
OFF
key_buffer_size
134217728
key_cache_age_threshold
300
key_cache_block_size
1024
Variable_name
Value
key_cache_division_limit
100
key_cache_file_hash_size
512
key_cache_segments
0
large_files_support
ON
large_page_size
0
large_pages
OFF
lc_messages
en_US
lc_messages_dir
lc_time_names
en_US
license
GPL
local_infile
ON
lock_wait_timeout
86400
locked_in_memory
OFF
log_bin
OFF
log_bin_basename
log_bin_compress
OFF
log_bin_compress_min_len
256
log_bin_index
log_bin_trust_function_creators
OFF
log_disabled_statements
sp
log_error
/var/lib/mysql/hostname.err
log_output
FILE
log_queries_not_using_indexes
OFF
log_slave_updates
OFF
log_slow_admin_statements
ON
log_slow_disabled_statements
sp
log_slow_filter
admin,filesort,filesort_on_disk,filesort_priority_...
log_slow_rate_limit
1
log_slow_slave_statements
ON
log_slow_verbosity
log_tc_size
24576
log_warnings
2
long_query_time
10.000000
low_priority_updates
OFF
lower_case_file_system
OFF
lower_case_table_names
0
master_verify_checksum
OFF
max_allowed_packet
268435456
max_binlog_cache_size
18446744073709547520
max_binlog_size
1073741824
max_binlog_stmt_cache_size
18446744073709547520
max_connect_errors
100
max_connections
200
max_delayed_threads
20
max_digest_length
1024
max_error_count
64
max_heap_table_size
134217728
max_insert_delayed_threads
20
max_join_size
18446744073709551615
max_length_for_sort_data
1024
max_password_errors
4294967295
max_prepared_stmt_count
16382
max_recursive_iterations
4294967295
max_relay_log_size
1073741824
max_rowid_filter_size
131072
max_seeks_for_key
4294967295
max_session_mem_used
9223372036854775807
max_sort_length
1024
max_sp_recursion_depth
0
max_statement_time
0.000000
max_tmp_tables
32
max_user_connections
0
max_write_lock_count
4294967295
metadata_locks_cache_size
1024
metadata_locks_hash_instances
8
min_examined_row_limit
0
mrr_buffer_size
262144
myisam_block_size
1024
myisam_data_pointer_size
6
myisam_max_sort_file_size
9223372036853727232
myisam_mmap_size
18446744073709551615
myisam_recover_options
BACKUP,QUICK
myisam_repair_threads
1
myisam_sort_buffer_size
134216704
myisam_stats_method
NULLS_UNEQUAL
myisam_use_mmap
OFF
mysql56_temporal_format
ON
net_buffer_length
16384
net_read_timeout
30
net_retry_count
10
net_write_timeout
60
old
OFF
old_alter_table
DEFAULT
old_mode
old_passwords
OFF
open_files_limit
74000
optimizer_max_sel_arg_weight
32000
optimizer_prune_level
1
optimizer_search_depth
62
optimizer_selectivity_sampling_limit
100
optimizer_switch
index_merge=on,index_merge_union=on,index_merge_so...
optimizer_trace
enabled=off
optimizer_trace_max_mem_size
1048576
optimizer_use_condition_selectivity
4
performance_schema
OFF
performance_schema_accounts_size
-1
performance_schema_digests_size
-1
performance_schema_events_stages_history_long_size
-1
performance_schema_events_stages_history_size
-1
performance_schema_events_statements_history_long_...
-1
Variable_name
Value
performance_schema_events_statements_history_size
-1
performance_schema_events_transactions_history_lon...
-1
performance_schema_events_transactions_history_siz...
-1
performance_schema_events_waits_history_long_size
-1
performance_schema_events_waits_history_size
-1
performance_schema_hosts_size
-1
performance_schema_max_cond_classes
90
performance_schema_max_cond_instances
-1
performance_schema_max_digest_length
1024
performance_schema_max_file_classes
80
performance_schema_max_file_handles
32768
performance_schema_max_file_instances
-1
performance_schema_max_index_stat
-1
performance_schema_max_memory_classes
320
performance_schema_max_metadata_locks
-1
performance_schema_max_mutex_classes
210
performance_schema_max_mutex_instances
-1
performance_schema_max_prepared_statements_instanc...
-1
performance_schema_max_program_instances
-1
performance_schema_max_rwlock_classes
50
performance_schema_max_rwlock_instances
-1
performance_schema_max_socket_classes
10
performance_schema_max_socket_instances
-1
performance_schema_max_sql_text_length
1024
performance_schema_max_stage_classes
160
performance_schema_max_statement_classes
222
performance_schema_max_statement_stack
10
performance_schema_max_table_handles
-1
performance_schema_max_table_instances
-1
performance_schema_max_table_lock_stat
-1
performance_schema_max_thread_classes
50
performance_schema_max_thread_instances
-1
performance_schema_session_connect_attrs_size
-1
performance_schema_setup_actors_size
-1
performance_schema_setup_objects_size
-1
performance_schema_users_size
-1
pid_file
/var/lib/mysql/servername.pid
plugin_dir
/usr/lib64/mysql/plugin/
plugin_maturity
gamma
port
3306
preload_buffer_size
32768
profiling
OFF
profiling_history_size
15
progress_report_time
5
protocol_version
10
proxy_protocol_networks
query_alloc_block_size
16384
query_cache_limit
1048576
query_cache_min_res_unit
4096
query_cache_size
1048576
query_cache_strip_comments
OFF
query_cache_type
OFF
query_cache_wlock_invalidate
OFF
query_prealloc_size
24576
range_alloc_block_size
4096
read_binlog_speed_limit
0
read_buffer_size
4194304
read_only
OFF
read_rnd_buffer_size
524288
relay_log
relay_log_basename
relay_log_index
relay_log_info_file
relay-log.info
relay_log_purge
ON
relay_log_recovery
OFF
relay_log_space_limit
0
replicate_annotate_row_events
ON
replicate_do_db
replicate_do_table
replicate_events_marked_for_skip
REPLICATE
replicate_ignore_db
replicate_ignore_table
replicate_wild_do_table
replicate_wild_ignore_table
report_host
report_password
report_port
3306
report_user
require_secure_transport
OFF
rowid_merge_buff_size
8388608
rpl_semi_sync_master_enabled
OFF
rpl_semi_sync_master_timeout
10000
rpl_semi_sync_master_trace_level
32
rpl_semi_sync_master_wait_no_slave
ON
rpl_semi_sync_master_wait_point
AFTER_COMMIT
rpl_semi_sync_slave_delay_master
OFF
rpl_semi_sync_slave_enabled
OFF
rpl_semi_sync_slave_kill_conn_timeout
5
rpl_semi_sync_slave_trace_level
32
secure_auth
ON
secure_file_priv
secure_timestamp
NO
server_id
1
session_track_schema
ON
session_track_state_change
OFF
session_track_system_variables
autocommit,character_set_client,character_set_conn...
session_track_transaction_info
OFF
skip_external_locking
ON
skip_name_resolve
OFF
skip_networking
OFF
Variable_name
Value
skip_show_database
OFF
slave_compressed_protocol
OFF
slave_ddl_exec_mode
IDEMPOTENT
slave_domain_parallel_threads
0
slave_exec_mode
STRICT
slave_load_tmpdir
/tmp
slave_max_allowed_packet
1073741824
slave_net_timeout
60
slave_parallel_max_queued
131072
slave_parallel_mode
optimistic
slave_parallel_threads
0
slave_parallel_workers
0
slave_run_triggers_for_rbr
NO
slave_skip_errors
OFF
slave_sql_verify_checksum
ON
slave_transaction_retries
10
slave_transaction_retry_errors
1158,1159,1160,1161,1205,1213,1429,2013,12701
slave_transaction_retry_interval
0
slave_type_conversions
slow_launch_time
2
slow_query_log
OFF
slow_query_log_file
servername-slow.log
socket
/var/lib/mysql/mysql.sock
sort_buffer_size
1048576
sql_auto_is_null
OFF
sql_big_selects
ON
sql_buffer_result
OFF
sql_if_exists
OFF
sql_log_bin
ON
sql_log_off
OFF
sql_mode
STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_...
sql_notes
ON
sql_quote_show_create
ON
sql_safe_updates
OFF
sql_select_limit
18446744073709551615
sql_slave_skip_counter
0
sql_warnings
OFF
ssl_ca
ssl_capath
ssl_cert
ssl_cipher
ssl_crl
ssl_crlpath
ssl_key
standard_compliant_cte
ON
storage_engine
InnoDB
stored_program_cache
256
strict_password_validation
ON
sync_binlog
0
sync_frm
ON
sync_master_info
10000
sync_relay_log
10000
sync_relay_log_info
10000
system_time_zone
IST
system_versioning_alter_history
ERROR
system_versioning_asof
DEFAULT
table_definition_cache
37000
table_open_cache
36885
table_open_cache_instances
1
tcp_keepalive_interval
0
tcp_keepalive_probes
0
tcp_keepalive_time
0
tcp_nodelay
ON
thread_cache_size
200
thread_handling
one-thread-per-connection
thread_pool_dedicated_listener
OFF
thread_pool_exact_stats
OFF
thread_pool_idle_timeout
60
thread_pool_max_threads
65536
thread_pool_oversubscribe
3
thread_pool_prio_kickup_timer
1000
thread_pool_priority
auto
thread_pool_size
36
thread_pool_stall_limit
500
thread_stack
299008
time_format
%H:%i:%s
time_zone
SYSTEM
tls_version
TLSv1.1,TLSv1.2,TLSv1.3
tmp_disk_table_size
18446744073709551615
tmp_memory_table_size
134217728
tmp_table_size
134217728
tmpdir
/tmp
Couldn't paste full global prams as stack limit but thats around 95% of it.
We have disabled swap in the previous my.cnf edit, as for mariadb even when there are alot of free memory within the system is still building swap until swap reached 100%(4GB of swap).

Why cannot I get 2 gigabytes from malloc?

Having this:
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#define GB (1<<30)
#define N 10000L //number of virtual blocks(arrays)
int main (void) {
int *ar[N];
for(int i =0; i<N ; i++)
{
ar[i]=malloc(GB); //alloc virtually one GB
if(!ar[i]){
printf("done at %i\n",i);
return 0;
}
}
printf("allocated %li blocks\n",N);
for(int i =0; i<N ; i++)
{
memset(ar[i],2,GB); // get physical gigs for that array
printf("wrote to %i GB\n",i+1);
}
return 0;
}
But I will not get even one:
allocated 10000 blocks //virtual memory (one block==1GB)
wrote to 1 GB //real memory (not really
Command terminated
Press ENTER or type command to continue
I know I have at leaset 4 gigs on my machine and I also know OS has some limit it needs to operate, however I cannot get even 2 GB for user space? That is strange. Can someone explain please?
$uname -a:
`4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) x86_64 GNU/Linux
EDIT:
It gets stranger:
Now if I run (after an hour from the previous output):
allocated 10000 blocks
wrote to 1 GB
wrote to 2 GB
Command terminated
Press ENTER or type command to continue
so it gets through 2 time in for loop, and allocates 2GB of real memory, but before an hour, I catch only 1GB and something (not full 2GB).
addition info:
$free -h:
total used free shared buff/cache available
Mem: 3.7Gi 1.1Gi 2.2Gi 236Mi 404Mi 2.2Gi
Swap: 0B 0B 0B
How can I take advantage from these information? So it is I said -> I have 4GB (total), but can get 2GB. That is maximum for user space?
Note: This answer does not solve the problem OP was having because it's based on fake code originally in the question, and still present in the question after the first request to correct fake code was made and acted upon. It does explain the question as originally asked.
If the code you posted is accurate now, your problem is that you are attempting to call functions without valid declarations for them (I see no #include directives). This is invalid C and the compiler should warn (or ideally error out) if you do it. Always add -Werror=implicit-function-declaration to get it to do that.
The particular mechanism of your crash is likely this: memset takes a size_t (unsigned long) as its third argument, but without a prototype, the function is being called as if the type of the function matched the argument types you provided (subject to default promotions). This produces undefined behavior. 1<<30 has type int, not unsigned long, and on x86_64 ABI, int is passed in the low 32 bits of a 64-bit register, with arbitrary junk allowed in the upper bits. So rather than passing 1 GB to memset, you're passing some astronomically large number.
Possibly a memory or swap limitation.
On a 4GB Ubuntu 18 VM, I get:
$ gcc -o alloc -Wall alloc.c
$ free -h
total used free shared buff/cache available
Mem: 3.9G 154M 3.1G 23M 572M 3.5G
Swap: 2.0G 0B 2.0G
$ uname -a
Linux ub18 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$./alloc
allocated 10000 blocks
wrote to 1 GB
wrote to 2 GB
wrote to 3 GB
wrote to 4 GB
wrote to 5 GB
Killed
$ free -h
total used free shared buff/cache available
Mem: 3.9G 127M 3.7G 1.6M 60M 3.6G
Swap: 2.0G 112M 1.9G
$
syslog says:
May 30 17:57:56 ub18 kernel: [ 240.723065] alloc invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
May 30 17:57:56 ub18 kernel: [ 240.723067] alloc cpuset=/ mems_allowed=0
May 30 17:57:56 ub18 kernel: [ 240.723072] CPU: 0 PID: 3062 Comm: alloc Not tainted 4.15.0-101-generic #102-Ubuntu
May 30 17:57:56 ub18 kernel: [ 240.723072] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
May 30 17:57:56 ub18 kernel: [ 240.723073] Call Trace:
May 30 17:57:56 ub18 kernel: [ 240.723088] dump_stack+0x6d/0x8e
May 30 17:57:56 ub18 kernel: [ 240.723094] dump_header+0x71/0x285
May 30 17:57:56 ub18 kernel: [ 240.723096] oom_kill_process+0x21f/0x420
May 30 17:57:56 ub18 kernel: [ 240.723097] out_of_memory+0x116/0x4e0
May 30 17:57:56 ub18 kernel: [ 240.723100] __alloc_pages_slowpath+0xa53/0xe00
May 30 17:57:56 ub18 kernel: [ 240.723102] __alloc_pages_nodemask+0x29a/0x2c0
May 30 17:57:56 ub18 kernel: [ 240.723104] alloc_pages_vma+0x88/0x1f0
May 30 17:57:56 ub18 kernel: [ 240.723107] __handle_mm_fault+0x8b7/0x1290
May 30 17:57:56 ub18 kernel: [ 240.723109] handle_mm_fault+0xb1/0x210
May 30 17:57:56 ub18 kernel: [ 240.723112] __do_page_fault+0x281/0x4b0
May 30 17:57:56 ub18 kernel: [ 240.723114] do_page_fault+0x2e/0xe0
May 30 17:57:56 ub18 kernel: [ 240.723115] ? page_fault+0x2f/0x50
May 30 17:57:56 ub18 kernel: [ 240.723116] page_fault+0x45/0x50
May 30 17:57:56 ub18 kernel: [ 240.723118] RIP: 0033:0x7f6d13bd7963
May 30 17:57:56 ub18 kernel: [ 240.723119] RSP: 002b:00007ffeede7e608 EFLAGS: 00010206
May 30 17:57:56 ub18 kernel: [ 240.723120] RAX: 00007f6b93b16010 RBX: 0000000000000000 RCX: 00007f6bb0993000
May 30 17:57:56 ub18 kernel: [ 240.723121] RDX: 00007f6bd3b16000 RSI: 0000000000000002 RDI: 00007f6b93b16010
May 30 17:57:56 ub18 kernel: [ 240.723122] RBP: 00007ffeede91eb0 R08: 0000000000000000 R09: 0000000000000000
May 30 17:57:56 ub18 kernel: [ 240.723122] R10: 0000000000000000 R11: 0000000000000246 R12: 0000559df7b44640
May 30 17:57:56 ub18 kernel: [ 240.723123] R13: 00007ffeede91f90 R14: 0000000000000000 R15: 0000000000000000
May 30 17:57:56 ub18 kernel: [ 240.723124] Mem-Info:
May 30 17:57:56 ub18 kernel: [ 240.723127] active_anon:780631 inactive_anon:158435 isolated_anon:0
May 30 17:57:56 ub18 kernel: [ 240.723127] active_file:13 inactive_file:10 isolated_file:1
May 30 17:57:56 ub18 kernel: [ 240.723127] unevictable:0 dirty:0 writeback:2 unstable:0
May 30 17:57:56 ub18 kernel: [ 240.723127] slab_reclaimable:4568 slab_unreclaimable:7013
May 30 17:57:56 ub18 kernel: [ 240.723127] mapped:196 shmem:364 pagetables:15033 bounce:0
May 30 17:57:56 ub18 kernel: [ 240.723127] free:21182 free_pcp:60 free_cma:0
May 30 17:57:56 ub18 kernel: [ 240.723130] Node 0 active_anon:3122524kB inactive_anon:633740kB active_file:52kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):4kB mapped:784kB dirty:0kB writeback:8kB shmem:1456kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
May 30 17:57:56 ub18 kernel: [ 240.723131] Node 0 DMA free:15736kB min:268kB low:332kB high:396kB active_anon:168kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
May 30 17:57:56 ub18 kernel: [ 240.723133] lowmem_reserve[]: 0 3421 3867 3867 3867
May 30 17:57:56 ub18 kernel: [ 240.723135] Node 0 DMA32 free:61252kB min:59536kB low:74420kB high:89304kB active_anon:2779728kB inactive_anon:629972kB active_file:20kB inactive_file:24kB unevictable:0kB writepending:48kB present:3653568kB managed:3566076kB mlocked:0kB kernel_stack:80kB pagetables:51412kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB
May 30 17:57:56 ub18 kernel: [ 240.723138] lowmem_reserve[]: 0 0 446 446 446
May 30 17:57:56 ub18 kernel: [ 240.723139] Node 0 Normal free:7740kB min:7772kB low:9712kB high:11652kB active_anon:342620kB inactive_anon:3748kB active_file:28kB inactive_file:8kB unevictable:0kB writepending:0kB present:524288kB managed:457424kB mlocked:0kB kernel_stack:2096kB pagetables:8720kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB
May 30 17:57:56 ub18 kernel: [ 240.723142] lowmem_reserve[]: 0 0 0 0 0
May 30 17:57:56 ub18 kernel: [ 240.723143] Node 0 DMA: 0*4kB 1*8kB (M) 1*16kB (M) 1*32kB (U) 3*64kB (UM) 1*128kB (U) 2*256kB (UM) 1*512kB (M) 2*1024kB (UM) 0*2048kB 3*4096kB (M) = 15736kB
May 30 17:57:56 ub18 kernel: [ 240.723149] Node 0 DMA32: 439*4kB (UME) 357*8kB (UME) 202*16kB (UME) 103*32kB (UE) 37*64kB (UME) 15*128kB (UME) 13*256kB (UME) 9*512kB (UE) 1*1024kB (E) 0*2048kB 9*4096kB (M) = 61252kB
May 30 17:57:56 ub18 kernel: [ 240.723156] Node 0 Normal: 621*4kB (UMEH) 285*8kB (UMEH) 110*16kB (UMEH) 34*32kB (UEH) 2*64kB (MH) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 7740kB
May 30 17:57:57 ub18 kernel: [ 240.723162] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
May 30 17:57:57 ub18 kernel: [ 240.723162] 3052 total pagecache pages
May 30 17:57:57 ub18 kernel: [ 240.723164] 2652 pages in swap cache
May 30 17:57:57 ub18 kernel: [ 240.723168] Swap cache stats: add 527798, delete 525146, find 859/1742
May 30 17:57:57 ub18 kernel: [ 240.723168] Free swap = 0kB
May 30 17:57:57 ub18 kernel: [ 240.723169] Total swap = 2104316kB
May 30 17:57:57 ub18 kernel: [ 240.723169] 1048462 pages RAM
May 30 17:57:57 ub18 kernel: [ 240.723170] 0 pages HighMem/MovableOnly
May 30 17:57:57 ub18 kernel: [ 240.723170] 38610 pages reserved
May 30 17:57:57 ub18 kernel: [ 240.723171] 0 pages cma reserved
May 30 17:57:57 ub18 kernel: [ 240.723171] 0 pages hwpoisoned
May 30 17:57:57 ub18 kernel: [ 240.723172] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
May 30 17:57:57 ub18 kernel: [ 240.723175] [ 396] 0 396 23715 11 217088 164 0 systemd-journal
May 30 17:57:57 ub18 kernel: [ 240.723177] [ 417] 0 417 24427 0 90112 51 0 lvmetad
May 30 17:57:57 ub18 kernel: [ 240.723178] [ 418] 0 418 11706 43 118784 590 -1000 systemd-udevd
May 30 17:57:57 ub18 kernel: [ 240.723179] [ 586] 62583 586 35483 0 184320 151 0 systemd-timesyn
May 30 17:57:57 ub18 kernel: [ 240.723181] [ 755] 100 755 20044 8 176128 176 0 systemd-network
May 30 17:57:57 ub18 kernel: [ 240.723182] [ 782] 101 782 17659 10 172032 157 0 systemd-resolve
May 30 17:57:57 ub18 kernel: [ 240.723183] [ 957] 0 957 17650 8 180224 172 0 systemd-logind
May 30 17:57:57 ub18 kernel: [ 240.723184] [ 959] 0 959 71564 14 200704 214 0 accounts-daemon
May 30 17:57:57 ub18 kernel: [ 240.723185] [ 961] 102 961 65760 0 163840 284 0 rsyslogd
May 30 17:57:57 ub18 kernel: [ 240.723186] [ 962] 0 962 7083 3 102400 49 0 atd
May 30 17:57:57 ub18 kernel: [ 240.723188] [ 963] 0 963 7507 0 102400 75 0 cron
May 30 17:57:57 ub18 kernel: [ 240.723189] [ 996] 0 996 42276 102 217088 1841 0 networkd-dispat
May 30 17:57:57 ub18 kernel: [ 240.723190] [ 997] 0 997 160115 231 233472 4850 -900 snapd
May 30 17:57:57 ub18 kernel: [ 240.723191] [ 1008] 0 1008 23885 0 77824 73 0 lxcfs
May 30 17:57:57 ub18 kernel: [ 240.723193] [ 1012] 103 1012 12571 1 139264 202 -900 dbus-daemon
May 30 17:57:57 ub18 kernel: [ 240.723194] [ 1094] 0 1094 101403 29 401408 641 0 NetworkManager
May 30 17:57:57 ub18 kernel: [ 240.723195] [ 1095] 0 1095 11308 6 122880 131 0 wpa_supplicant
May 30 17:57:57 ub18 kernel: [ 240.723196] [ 1110] 0 1110 108581 10 356352 344 0 ModemManager
May 30 17:57:57 ub18 kernel: [ 240.723197] [ 1167] 0 1167 72221 0 212992 208 0 polkitd
May 30 17:57:57 ub18 kernel: [ 240.723198] [ 1283] 0 1283 46486 96 249856 1879 0 unattended-upgr
May 30 17:57:57 ub18 kernel: [ 240.723200] [ 1450] 0 1450 3722 0 69632 33 0 agetty
May 30 17:57:57 ub18 kernel: [ 240.723201] [ 1531] 0 1531 18075 11 184320 177 -1000 sshd
May 30 17:57:57 ub18 kernel: [ 240.723202] [ 1571] 111 1571 79693 19 331776 484 -900 postgres
May 30 17:57:57 ub18 kernel: [ 240.723203] [ 1578] 111 1578 79781 4 335872 500 -900 postgres
May 30 17:57:57 ub18 kernel: [ 240.723204] [ 1603] 111 1603 79781 1 294912 500 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723205] [ 1604] 111 1604 79781 10 311296 500 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723207] [ 1605] 111 1605 79781 2 303104 507 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723208] [ 1606] 111 1606 79883 97 311296 521 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723209] [ 1607] 111 1607 43543 7 282624 509 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723210] [ 1608] 111 1608 79882 58 303104 528 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723211] [ 1611] 111 1611 79693 0 299008 501 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723212] [ 1612] 111 1612 79693 19 311296 495 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723213] [ 1613] 111 1613 79693 0 307200 506 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723214] [ 1614] 111 1614 79794 71 315392 530 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723216] [ 1615] 111 1615 43457 13 282624 511 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723217] [ 1616] 111 1616 79771 46 307200 532 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723218] [ 2896] 0 2896 26997 7 249856 245 0 sshd
May 30 17:57:57 ub18 kernel: [ 240.723219] [ 2908] 1000 2908 19166 0 180224 277 0 systemd
May 30 17:57:57 ub18 kernel: [ 240.723220] [ 2909] 1000 2909 48429 0 266240 593 0 (sd-pam)
May 30 17:57:57 ub18 kernel: [ 240.723221] [ 3026] 1000 3026 26997 11 245760 243 0 sshd
May 30 17:57:57 ub18 kernel: [ 240.723223] [ 3027] 1000 3027 5823 1 81920 461 0 bash
May 30 17:57:57 ub18 kernel: [ 240.723224] [ 3062] 1000 3062 2621451127 934324 93466624 504770 0 alloc
May 30 17:57:57 ub18 kernel: [ 240.723225] [ 3089] 111 3089 79921 271 319488 468 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723226] [ 3090] 111 3090 79921 331 319488 415 0 postgres
May 30 17:57:57 ub18 kernel: [ 240.723227] [ 3091] 0 3091 8646 38 98304 0 0 sshd
May 30 17:57:57 ub18 kernel: [ 240.723228] Out of memory: Kill process 3062 (alloc) score 951 or sacrifice child
May 30 17:57:57 ub18 kernel: [ 240.723268] Killed process 3062 (alloc) total-vm:10485804508kB, anon-rss:3737288kB, file-rss:8kB, shmem-rss:0kB
May 30 17:57:57 ub18 kernel: [ 240.916591] oom_reaper: reaped process 3062 (alloc), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
$

"strange" virtual memory chunks

I have an ARM 32bit C/C++ program running on Ubuntu 14.04. The max resident memory of the program is about 90MB. However, the virtual memory size of the program is huge - around 400MB. I use pmap with x switch to check the details, what I find and don't quite understand is there are about 20-30 virtual memory chunks, each chunk's size is 8188KB and mode is RWX. The actual resident size of each 8188KB chunk is very small, something like 8K, 12KB, 24KB. Below is a snapshot for your reference.
Address Kbytes RSS Dirty Mode Mapping
00010000 12 12 0 r-x-- theApp
00022000 4 4 4 r---- theApp
00023000 4 4 4 rw--- theApp
00024000 3556 3536 3536 rw--- [ anon ]
9941c000 4 0 0 ----- [ anon ]
9941d000 8188 8 8 rwx-- [ anon ]
99c1c000 4 0 0 ----- [ anon ]
99c1d000 8188 8 8 rwx-- [ anon ]
9a41c000 4 0 0 ----- [ anon ]
9a41d000 8188 8 8 rwx-- [ anon ]
9ac1c000 4 0 0 ----- [ anon ]
9ac1d000 8188 8 8 rwx-- [ anon ]
9b41c000 4 0 0 ----- [ anon ]
9b41d000 8188 8 8 rwx-- [ anon ]
9bc1c000 4 0 0 ----- [ anon ]
9bc1d000 8188 24 24 rwx-- [ anon ]
...
b57f6000 28 28 28 rw--- libcrypto.so.1.0.0
b57fd000 12 4 4 rw--- [ anon ]
b5800000 4 0 0 ----- [ anon ]
b5801000 8188 12 12 rwx-- [ anon ]
...
In GDB, there is no information after I run - info symbol "the chunk's address". I also dump(dump memory command) some chunks into a file, each bit is zero.
In the program, there is no direct call to mmap. It only explicitly calls malloc and new for heap memory allocation. And I don't find any memory allocation size is 8188KB. Actually, I could identify each heap allocation of my code from the output of pmap. Of course, the program also uses some 3rd party shared libraries which I don't know the internal implementation.
My questions are:
where may the memory chunks come from?
is there any other ways to trace them down?
if swap file of the system is disabled, will the program still running fine? As the actual physical RAM is less than 400MB.
Any input will be appreciated!
Thanks,
-Neil

application memory optimization

We have a project written in ANSI C. Generally the memory consumption was not a big concern, but now we have a request to fit our program into 256 KB of RAM. I don't have this exact platform on hands, so I compile my project under 32 bit x86 Linux (because it provides enough different tools to evaluate the memory consumption), optimize what I can, remove some features and eventually I have to have the conclusion: what features we need to sacrifice to be able to run on very small systems (if we're able at all). First of all I did a research what exactly a memory size in linux and it seems I have to optimize the RSS size, not VSZ. But in linux even a smallest program which prints "Hello world!" once a second consumes 285-320 KB in RSS:
#include <stdio.h>
#include <unistd.h>
#include <signal.h>
unsigned char cuStopCycle = 0;
void SigIntHandler(int signo)
{
printf("SIGINT received, terminating the program\n");
cuStopCycle = 1;
}
int main()
{
signal( SIGINT, SigIntHandler);
while(!cuStopCycle)
{
printf("Hello, World!\n");
sleep(1);
}
printf("Exiting...\n");
}
user#Ubuntu12-vm:~/tmp/prog_size$ size ./prog_size
text data bss dec hex filename
1456 272 12 1740 6cc ./prog_size
root#Ubuntu12-vm:/home/app# ps -C prog_size -o pid,rss,vsz,args
PID RSS VSZ COMMAND
22348 316 2120 ./prog_size
Obviously this program will perfectly run on small PLCs, with 64KB of RAM. It is just linux loads a lot of libs. I generate a map file for this program and all this data + bss comes from the CRT library. I need to mention that if I add some code to this project - 10,000 times "a = a + b" or manipulate arrays 2000 long int variables, I see the difference in code size, bss size but eventually the RSS size of the process is the same, it doesn't affected)
So I take this as a baseline, the point I want to reach (and which I will never reach, because I need more functions than just print a message once a second).
So here comes my project, where I removed all extra features, removed all auxiliary functions, removed everything except the basic functionality. There are some ways to optimize more, but not that much, what could be removed is already taken away:
root#Ubuntu12-vm:/home/app/workspace/proj_sizeopt/Cmds# ls -l App
-rwxr-xr-x 1 root root 42520 Jul 13 18:33 App
root#Ubuntu12-vm:/home/app/workspace/proj_sizeopt/Cmds# size ./App
text data bss dec hex filename
37027 404 736 38167 9517 ./App
So I have ~36KB of code and ~1KB of data. I do not call malloc inside of my project, I use a shared memory allocation with a wrapper library so I can control how much memory is allocated:
The total memory size allocated is 2052 bytes
Under the hood there are malloc calls obviously, if I substitute 'malloc' calls with my function which summarize all alloc requests I see that ~2.3KB of memory is allocated:
root#Ubuntu12-vm:/home/app/workspace/proj_sizeopt/Cmds# LD_PRELOAD=./override_malloc.so ./App
Malloc allocates 2464 bytes total
Now I run my project amd see that it consumes 600KB of RAM.
root#Ubuntu12-vm:/home/app/workspace/proj_sizeopt# ps -C App -o pid,rss,vsz,args
PID RSS VSZ COMMAND
22093 604 2340 ./App
I do not understand why it eats so much memory. The code size is small. There is not much memory allocated. The size of data is small. Why it takes so much memory? I tried to analyze the mapping of the process:
root#Ubuntu12-vm:/home/app/workspace/proj_sizeopt# pmap -x 22093
22093: ./App
Address Kbytes RSS Dirty Mode Mapping
08048000 0 28 0 r-x-- App
08052000 0 4 4 r---- App
08053000 0 4 4 rw--- App
09e6a000 0 4 4 rw--- [ anon ]
b7553000 0 4 4 rw--- [ anon ]
b7554000 0 48 0 r-x-- libpthread-2.15.so
b756b000 0 4 4 r---- libpthread-2.15.so
b756c000 0 4 4 rw--- libpthread-2.15.so
b756d000 0 8 8 rw--- [ anon ]
b7570000 0 300 0 r-x-- libc-2.15.so
b7714000 0 8 8 r---- libc-2.15.so
b7716000 0 4 4 rw--- libc-2.15.so
b7717000 0 12 12 rw--- [ anon ]
b771a000 0 16 0 r-x-- librt-2.15.so
b7721000 0 4 4 r---- librt-2.15.so
b7722000 0 4 4 rw--- librt-2.15.so
b7731000 0 4 4 rw-s- [ shmid=0x70000c ]
b7732000 0 4 4 rw-s- [ shmid=0x6f800b ]
b7733000 0 4 4 rw-s- [ shmid=0x6f000a ]
b7734000 0 4 4 rw-s- [ shmid=0x6e8009 ]
b7735000 0 12 12 rw--- [ anon ]
b7738000 0 4 0 r-x-- [ anon ]
b7739000 0 104 0 r-x-- ld-2.15.so
b7759000 0 4 4 r---- ld-2.15.so
b775a000 0 4 4 rw--- ld-2.15.so
bfb41000 0 12 12 rw--- [ stack ]
-------- ------- ------- ------- -------
total kB 2336 - - -
And it looks like the program size (in RSS) is only 28KB, the rest is consumed by shared libraries. BTW I do not use posix threads, I do not explicitly link to it, but somehow the linker anyway links this library I have no idea why (this is not really important). If we look at the mapping in more details:
root#Ubuntu12-vm:/home/app/workspace/proj_sizeopt# cat /proc/22093/smaps
08048000-08052000 r-xp 00000000 08:01 344838 /home/app/workspace/proj_sizeopt/Cmds/App
Size: 40 kB
Rss: 28 kB
Pss: 28 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 28 kB
Private_Dirty: 0 kB
Referenced: 28 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
...
09e6a000-09e8b000 rw-p 00000000 00:00 0 [heap]
Size: 132 kB
Rss: 4 kB
Pss: 4 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 4 kB
Referenced: 4 kB
Anonymous: 4 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
...
b7570000-b7714000 r-xp 00000000 08:01 34450 /lib/i386-linux-gnu/libc-2.15.so
Size: 1680 kB
Rss: 300 kB
Pss: 7 kB
Shared_Clean: 300 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 300 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
...
b7739000-b7759000 r-xp 00000000 08:01 33401 /lib/i386-linux-gnu/ld-2.15.so
Size: 128 kB
Rss: 104 kB
Pss: 3 kB
Shared_Clean: 104 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 104 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
...
bfb41000-bfb62000 rw-p 00000000 00:00 0 [stack]
Size: 136 kB
Rss: 12 kB
Pss: 12 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 12 kB
Referenced: 12 kB
Anonymous: 12 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
So I see that RSS size for my project is 40KB, but only 28 KB is used. Does it mean that this project will fit into 256 KB of RAM?
The heap size is 132KB but only 4 KB is used. Why is that? I'm sure it will be different on the small embedded platform.
The stack is 136KB but only 12KB is used.
GLIBC/LD obviously consume some memory, but what exactly memory will it be on the embedded platform?
I do not look at PSS because it doesn't make any sense in my case, I look only at RSS.
What conclusions can I draw from this picture? How exactly to evaluate memory consumption by the application? Look at the RSS size of the process? Or subtract from this size RSS of all mapped system libraries? What is about heap/stack size?
I would be very grateful for any advises, notes, memory consumption optimizations techniques, DOs and DON'Ts for platforms with extremely small amount of RAM (except obvious - keep amount of data and code to the very minimum).
I also will appreciate an explanation WHY the program with small amount of code and data (and which doesn't allocate much memory) still consumes a lot of RAM in RSS.
Thank you in advance
... fit our program into 256 KB of RAM. I don't have this exact platform on hands, so I compile my project under 32 bit x86 Linux..
And what you now see is that the Linux platform tools make reasonable assumptions of your possible need of stack and heap, given that it nows you run on a big machine, and links-in a reasonable set of library functions for your needs. Some you won't need, but it gives them to you "for free".
To fit in 256 Kb on your target platform, you must compile for your target platform and link with the target platform's libraries (and CRT) using the target platform's linker.
Those will make different assumptions, use possibly smaller linbrary footprints, make smaller assumptions on stack and heap space, etcetera. For example, create "Hello World" for the target platform and check its needs on that target platform. Or use a realistic simulator of the target platform and libraries (and not to forget, OS, whch partly dictates what the libraries must do).
And if it is then still too big, you have to re-write or tweak the whole CRT and all libraries....
the program needs to be compiled/linked with the embedded device in mind.
For best results use a makefile
use the 'rt' library written for the embedded device
use the start.s file, located, via the makefile, where execution begins.
use 'static' in the linker parameters
use the linker parameters to not include any libraries but what is specifically requested.
do not use libraries written for your development machine. Only use libraries written for the embedded device.
do NOT include stdio.h, etc. unless specifically written for the embedded device
do NOT call printf() inside a signal handler.
if possible, do not call printf() at all.
instead write a small char output function and have it perform the output through the uart.
do not use signals, instead use interrupts
the resulting application will not run on your PC., but, once loaded, will run on the 256k device
do not call sleep(), rather write your own function that uses a device timer peripheral, that sets the timer and puts the device into powerdown mode.
the time interrupt needs to bring the device out of the powerdown mode.
in the makefile, specifically set the size of the stack, heap, etc.
have the link step output a .map file. study that map file until you understand everything in it.
use a compiler/linker that is specific for the embedded device
you probably will need to include a function that initializes the peripherals on the device, like the clock, the uart, the timer, the watchdog, and any other built in peripherals that the code actually uses.
you will need a file that allocates the interrupt table, and a small function to handle each of the interrupts, even though most of those functions will do nothing beyond clearing the appropriate interrupt pending flag and returning from the interrupt
you will probably need a function to periodically refresh the watchdog, conditionally, depending on an indication that the main function is still cycling regularily. I.E the main function loop and the initialization function will refresh the watchdog

What address does the text section start?

I read in a book (can't recollect the name) that in a 32 bit system text section always starts at 0x0848000. But when I do readelf -S example_executable it does not reflect the same information. Why is that? Do other sections (bss,data,rodata etc) also start at fixed addresses? How can I find the alignment boundary for these sections?
There is a good explanation here of how Linux virtual memory works when allocating storage for a particular program.
ua alberta - linux memory allocation
The designers of the compiler/linker tool chain need to allocate an arbitary address for particular blocks of memory. In order to make it easier for other components of the tool chain like debuggers and profilers they always allocatate the same block to the same addresses. The actual addresses chosen are completely arbitrary.
When the program is loaded the virtual address will be mapped to some random piece of free memory (this is mostly done in hardware). This mapping is done on a per process basis to several programs can address virtual address x'0848000' but be pointed at different "real" memory addresses.
It all depends on the implementation on the particular machine.For a linux machine the behaviour will be different than that from windows machine.
Note however that the virtual memory addresses need to start at some fixed address in order to make life easier for debuggers.However the real addresses will be different depending on the pages available in RAM.
If you look the output of readelf -S more carefully you will notice you will notice subtracting the offset from the address indeed gives you 0x0848000.
As i had mentioned earlier this magic number 0x0848000 will depend on the type of executable format.
here is the output i get on my ubuntu 32 bit machine:
readelf -S ~/a.out
There are 29 section headers, starting at offset 0x1130:
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .interp PROGBITS 08048134 000134 000013 00 A 0 0 1
[ 2] .note.ABI-tag NOTE 08048148 000148 000020 00 A 0 0 4
[ 3] .note.gnu.build-i NOTE 08048168 000168 000024 00 A 0 0 4
[ 4] .gnu.hash GNU_HASH 0804818c 00018c 000020 04 A 5 0 4
[ 5] .dynsym DYNSYM 080481ac 0001ac 000050 10 A 6 1 4
[ 6] .dynstr STRTAB 080481fc 0001fc 00004c 00 A 0 0 1
[ 7] .gnu.version VERSYM 08048248 000248 00000a 02 A 5 0 2
[ 8] .gnu.version_r VERNEED 08048254 000254 000020 00 A 6 1 4
[ 9] .rel.dyn REL 08048274 000274 000008 08 A 5 0 4
[10] .rel.plt REL 0804827c 00027c 000018 08 A 5 12 4
[11] .init PROGBITS 08048294 000294 000030 00 AX 0 0 4
[12] .plt PROGBITS 080482c4 0002c4 000040 04 AX 0 0 4
[13] .text PROGBITS 08048310 000310 00018c 00 AX 0 0 16
[14] .fini PROGBITS 0804849c 00049c 00001c 00 AX 0 0 4
There is no consistent address between operating systems and architectures for the text section or any other section. Additionally position independent code and address space layout randomizations make these values even inconsistent on the same machine for some systems.

Resources