I need to get the start time of process using C code in userspace.
The process will run as root, So I can fopen /proc/PID/stat.
I saw implementation, e.g:
start time of a process on linux
or
http://brokestream.com/procstat.c
But they are invalid, Why they are invalid ? if the process 2nd parameter contains space, e.g:
[ilan#CentOS7286-64 tester]$ cat /proc/1077/stat
1077 (rs:main Q:Reg) S 1 1054 1054 0 -1 1077944384 21791 0 10 0 528 464 0 0 20 0 3 0 1056 321650688 1481 18446744073709551615 1 1 0 0 0 0 2146172671 16781830 1133601 18446744073709551615 0 0 -1 1 0 0 1 0 0 0 0 0 0 0 0 0 0
These solutions will not work.
Is there a better way retrieving a process start time other then parsing the /proc/PID/stat results ? I can do the following logic:
read long, first parameter is pid
read char, make sure that i finish reading only when hitting close ')'. - 2nd parameter is tcomm (filename of the executable)
read char - 3rd parameter process state.
In solaris, you simply read the result to psinfo_t struct.
You can simply use the stat(2) kernel call.
The creation time is not set by the proc file system. But you can use the modification time, because the modification time of a directory changes only, if files are added to or removed from the directory. And because the content of a directory in the proc filesystem changes only, if you replace the kernel module of the running kernel, you can be pretty sure, that the modification time is also the creation time.
Example:
$ stat -c %y /proc/1
2018-06-01 11:46:57.512000000 +0200
$ uptime -s
2018-06-01 11:46:57
Related
I have a simple C HTTP server. I close file descriptors for disk files and new connection fds returned by accept(...), but I noticed that I am getting new file descriptor numbers that are bigger than the previous numbers: for example file descriptor from accept return starts with 4, then 5, then 4 again and so on until file descriptor reaches max open file descriptor on a system.
I have set the value to 10,000 on my system but I am not sure why exactly file descriptor number jumps to max value. And I am kind of sure than my program is closing the file descriptors.
So I would like to know if there are not thousands of connections then how come file descriptor new number are increasing periodically: in around 24 hours I get message accept: too many open files. What is this message?
Also, does ulimit -n number value get reset automatically without system reboot?
as mentioned in the answer. The output of _2$ ps aux | grep lh is
dr-x------ 2 fawad fawad 0 Oct 11 11:15 .
dr-xr-xr-x 9 fawad fawad 0 Oct 11 11:15 ..
lrwx------ 1 fawad fawad 64 Oct 11 11:15 0 -> /dev/pts/3
lrwx------ 1 fawad fawad 64 Oct 11 11:15 1 -> /dev/pts/3
lrwx------ 1 fawad fawad 64 Oct 11 11:15 2 -> /dev/pts/3
lrwx------ 1 fawad fawad 64 Oct 11 11:25 255 -> /dev/pts/3
and the output of ls -la /proc/$$/fd is
root 49855 0.5 5.4 4930756 322328 ? Sl Oct09 15:58 /usr/share/atom/atom --executed-from=/home/fawad/Desktop/C++-work/lhparse --pid=49844 --no-sandbox
root 80901 0.0 0.0 25360 5952 pts/4 S+ 09:32 0:00 sudo ./lh
root 80902 0.0 0.0 1100852 2812 pts/4 S+ 09:32 0:00 ./lh
fawad 83419 0.0 0.0 19976 916 pts/3 S+ 11:27 0:00 grep --color=auto lh
I like to know what is pts/4 etc. column. is this the file descriptor number.
It's likely that the socket that is represented by the file descriptor is in close_wait or time_wait state. Which means the TCP stack holds the fd open for a bit longer. So you won't be able to reuse it immediately in this instance.
Once the socket is fully finished with and closed, the file descriptor number will then available for reuse inside your program.
See: https://en.m.wikipedia.org/wiki/Transmission_Control_Protocol
Protocol Operation and specifically Wait States.
To see what files are still open you can run
ls -la /proc/$$/fd
The output of this will also be of help.
ss -tan | head -5
LISTEN 0 511 *:80 *:*
SYN-RECV 0 0 192.0.2.145:80 203.0.113.5:35449
SYN-RECV 0 0 192.0.2.145:80 203.0.113.27:53599
ESTAB 0 0 192.0.2.145:80 203.0.113.27:33605
TIME-WAIT 0 0 192.0.2.145:80 203.0.113.47:50685
If a running process's executable is deleted, I've noticed fork fails where the child process is never executed.
For example, consider the code below:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(void) {
sleep(5);
pid_t forkResult;
forkResult = fork();
printf("after fork %d \n", forkResult);
return 0;
}
If I compile this and delete the resulting executable before fork is called, I never see fork return a pid of 0, meaning the child process never starts. I only have a Mac running Big Sur, so not sure if this repros on other OS's.
Does anyone know why this would be? My understanding is an executable should work just fine even if it's deleted while still running.
The expectation that the process should continue even if the binary was deleted is correct, however not fully correct in case of macOS. The example is tripping on a side-effect of the System Integrity Protection (SIP) mechanism inside the macOS kernel, however before explaining what is exactly going on, we need to make several experiments which will help us to better understand the whole scenario.
Modified example to better demonstrate the issue
To demonstrate what is going on, I had modified the example to count to 9, than do the fork, after the fork, the child will print a message "I am done", wait 1 second and exit by printing the 0 as the PID. The parent will continue to count to 14 and print the child PID. The code is as follows:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(void) {
for(int i=0; i <10; i++)
{
sleep(1);
printf("%i ", i);
}
pid_t forkResult;
forkResult = fork();
if (forkResult != 0) {
for(int i=10; i < 15; i++) {
sleep(1);
printf("%i ", i);
}
} else {
sleep(1);
printf("I am done ");
}
printf("after fork %d \n", forkResult);
return 0;
}
After compiling it, I have started the normal scenario:
╰> ./a.out
0 1 2 3 4 5 6 7 8 9 I am done after fork 0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 after fork 4385
So, the normal scenario works as expected. The fact that we see the count from 0 to 9 two times, is due to the copy of the buffers for stdout that was done in the fork call.
Tracing the failing example
Now is time to do the negative scenario, we will wait for 5 seconds after the start and remove the binary.
╰> ./a.out & (sleep 5 && rm a.out)
[4] 8555
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 after fork 8677
[4] 8555 done ./a.out
We see that the output is only from the parent. Since the parent had counted to 14, and shows valid PID for the child, however the child is missing, it never printed anything. So, the child creation failed after the fork() was performed, otherwise fork() would have received and error instead of a valid PID. Traces from ktrace reveal that the child was created under the pid and was waken up:
test5-ko.txt:2021-04-07 13:34:26.623783 +04 0.3 MACH_DISPATCH 1bc 0 84 4 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623783 +04 0.2 TMR_TimerCallEnter 9931ba49ead1bd17 0 330e7e4e9a59 41 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623783 +04 0.0(0.0) TMR_TimerCallEnter 9931ba49ead1bd17 0 330e7e4e9a59 0 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623783 +04 0.0 TMR_TimerCallEnter 9931ba49ead1bd17 0 330e7e4e9a59 0 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623854 +04 0.0 imp_thread_qos_and_relprio 88775d 20000 20200 6 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623854 +04 0.0 imp_update_thread 88775d 811200 140000100 1f 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623855 +04 0.1(0.8) imp_update_thread 88775d c15200 140000100 25 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623855 +04 0.0(1.1) imp_thread_qos_and_relprio 88775d 30000 20200 40 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623855 +04 0.0 imp_thread_qos_workq_override 88775d 30000 20200 0 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623855 +04 0.0 imp_update_thread 88775d c15200 140000100 25 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623855 +04 0.1(0.1) imp_update_thread 88775d c15200 140000100 25 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623855 +04 0.0(0.2) imp_thread_qos_workq_override 88775d 30000 20200 40 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623857 +04 1.3 TURNSTILE_turnstile_added_to_thread_heap 88775d 9931ba6049ddcc77 0 0 888065 2 a.out(8677)
test5-ko.txt:2021-04-07 13:34:26.623858 +04 1.0 MACH_MKRUNNABLE 88775d 25 0 5 888065 2 a.out(8677)
t
So the child's process was dispatched with MACH_DISPATCH and made runnable with MACH_MKRUNNABLE. This is the reason the parent got valid PID after the fork().
Further more the ktrace for the normal scenario shows that the process had issued BSC_exit and and imp_task_terminated system call occurred, which is the normal way for a process to exit. However, in the second scenario where we had deleted the file, the trace doesn't show BSC_exit. This means that the child was terminated by the kernel, not by a normal termination. And we know that the termination happend after the child was created properly, since the parent had received the valid PID and the PID was made runnable.
This bring us closer to the understanding of what is going on here. But, before we have the conclusion, let's show another even more "twisted" example.
Even more strange example
What if we replace the binary on the filesystem after we started the process?
Here is the test to answer this question: we will start the process, remove the binary and create an empty file with the same name on his place with touch.
╰> ./a.out & (sleep 5 && rm a.out; touch a.out)
[1] 6264
0 1 2 3 4 5 6 7 8 9 I am done after fork 0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 after fork 6851
[1] + 6722 done ./a.out
Wait a minute, this works!? What is going on here!?!?
This strange example gives us important clue that will help us to explain what is going on.
The root-cause of the issue
The reason why the third example works, while the second one is failing, reveals a lot of what is going on here. As mentioned on the beginning, we are tripping on a side-effect of SIP, more precisely on the runtime protection mechanism.
To protect the system integrity, SIP will examine the running processes for the system protection and special entitlement. From the apple documentation: ...When a process is started, the kernel checks to see whether the main executable is protected on disk or is signed with an special system entitlement. If either is true, then a flag is set to denote that it is protected against modification. Any attempt to attach to a protected process is denied by the kernel...
When we had removed the binary from the filesystem, the protection mechanism was not able to identify the type of process for the child nor the special system entitlements since the binary file was missing from the disk. This triggered the protection mechanism to treat this process as an intruder in the system and terminate it, hanse we had not seen the BSC_exit for the child process.
In the third example, when we created dummy entry on the file system with touch, the SIP was able to detect that this is not a special process nor it has special entitlements and allowed the process to continue. This is a very solid indication that we ware tripping on the SIP realtime protection mechanism.
To prove that this is the case, I have disabled the SIP which requires a restart in the recovery mode and executed the test
╰> csrutil status
System Integrity Protection status: disabled.
╰> ./a.out & (sleep 5 && rm a.out)
[1] 1504
0 1 2 3 4 5 6 7 8 9 I am done after fork 0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 after fork 1626
Conclusion
So, the whole issue was caused by the System Integrity Protection. More details can be fond in the documentation
All the SIP needed was to have a file on the filesystem with the process name, so the mechanism can run the verification and decide to allow the child to continue the execution. This is showing us that we are observing a side-effect, rather than designed behavior, since the empty file was not even a valid dwarf, yet the execution had proceed.
I'm trying to install a Informix Developer instance on a debian 7 server, to get to test a couple of programs with Informix as Database engine. I tried all solutions I could find googling the error messages, but it kept me going back and forth. Can any body see what my problem is?
The log:
14:46:46 Parameter's user-configured value was adjusted.
(DS_MAX_SCANS)
14:46:46 Parameter's user-configured value was adjusted. (ONLIDX_MAXMEM)
14:46:46 IBM Informix Dynamic Server Started.
14:46:46 Warning: The IBM IDS Developer Edition license restriction limits
14:46:46 the total shared memory size for this server to 1048576 KB.
14:46:46 The maximum allowable shared memory was reset to this size to start the database server.
14:46:46 Requested shared memory segment size rounded from 8308KB to 8840KB
Sat Dec 19 14:46:48 2015
14:46:48 Successfully added a bufferpool of page size 2K.
14:46:48 Event alarms enabled. ALARMPROG = '/etc/Informix/etc/alarmprogram.sh'
14:46:48 Booting Language <c> from module <>
14:46:48 Loading Module <CNULL>
14:46:48 Booting Language <builtin> from module <>
14:46:48 Loading Module <BUILTINNULL>
14:46:53 DR: DRAUTO is 0 (Off)
14:46:53 DR: ENCRYPT_HDR is 0 (HDR encryption Disabled)
14:46:53 Event notification facility epoll enabled.
14:46:53 CCFLAGS2 value set to 0x200
14:46:53 SQL_FEAT_CTRL value set to 0x8008
14:46:53 SQL_DEF_CTRL value set to 0x4b0
14:46:53 IBM Informix Dynamic Server Version 12.10.FC5DE Software Serial Number AAA#B000000
14:46:53 requested number of KAIO events (32768) exceeds limit (31437). using 31437.
14:46:54 Value of FULL_DISK_INIT has been changed to 0.
14:46:54 Performance Advisory: The physical log size is smaller than the recommended size for a
server configured with RTO_SERVER_RESTART.
14:46:54 Results: Fast recovery performance might not be optimal.
14:46:54 Action: For best fast recovery performance when RTO_SERVER_RESTART is enabled,
increase the physical log size to at least 66602 KB. For servers
configured with a large buffer pool, this might not be necessary.
14:46:54 IBM Informix Dynamic Server Initialized -- Complete Disk Initialized.
14:46:54 Started 1 B-tree scanners.
14:46:54 B-tree scanner threshold set at 5000.
14:46:54 B-tree scanner range scan size set to -1.
14:46:54 B-tree scanner ALICE mode set to 6.
14:46:54 B-tree scanner index compression level set to med.
14:46:54 Warning: Invalid (non-existent/blobspace/disabled) dbspace listed
in DBSPACETEMP: 'tempdbs'
14:46:55 The Master Daemon Died
14:46:55 invoke_alarm(): /bin/sh -c '/etc/Informix/etc/alarmprogram.sh 5 6 "Internal Subsystem failure: 'MT'" "The Master Daemon Died" "" 6069'
14:46:55 invoke_alarm(): mt_exec failed, status 32512, errno 0
14:46:55 PANIC: Attempting to bring system down
And if it helps the onconfig file the database
ROOTNAME rootdbs
ROOTPATH /home/informix_storage/rootdbs
ROOTOFFSET 0
ROOTSIZE 2000000
MIRROR 0
MIRRORPATH $INFORMIXDIR/tmp/demo_on.root_mirror
MIRROROFFSET 0
PHYSFILE 29792
PLOG_OVERFLOW_PATH $INFORMIXDIR/tmp
PHYSBUFF 512
LOGFILES 4
LOGSIZE 7168
DYNAMIC_LOGS 2
LOGBUFF 256
LTXHWM 70
LTXEHWM 80
MSGPATH /etc/Informix/informix_inst_01.log
CONSOLE $INFORMIXDIR/tmp/online.con
TBLTBLFIRST 0
TBLTBLNEXT 0
TBLSPACE_STATS 1
DBSPACETEMP tempdbs
SBSPACETEMP
SBSPACENAME sbspace
SYSSBSPACENAME
ONDBSPACEDOWN 2
SERVERNUM 0
DBSERVERNAME informix_inst_01
DBSERVERALIASES dr_informix_inst_01, lo_informix_inst_01
FULL_DISK_INIT 0
NETTYPE ipcshm,1,50,CPU
LISTEN_TIMEOUT 60
MAX_INCOMPLETE_CONNECTIONS 1024
FASTPOLL 1
NUMFDSERVERS 4
NS_CACHE host=900,service=900,user=900,group=900
MULTIPROCESSOR 1
VPCLASS cpu,num=1,noage
VP_MEMORY_CACHE_KB 0
SINGLE_CPU_VP 1
AUTO_TUNE 1
CLEANERS 2
DIRECT_IO 1
LOCKS 40000
DEF_TABLE_LOCKMODE page
RESIDENT 0
SHMBASE 0x44000000L
SHMVIRTSIZE 101376
SHMADD 3165
EXTSHMADD 8192
SHMTOTAL 0
SHMVIRT_ALLOCSEG 0,3
SHMNOACCESS
CKPTINTVL 300
RTO_SERVER_RESTART 60
BLOCKTIMEOUT 3600
CONVERSION_GUARD 2
RESTORE_POINT_DIR $INFORMIXDIR/tmp
TXTIMEOUT 300
DEADLOCK_TIMEOUT 60
HETERO_COMMIT 0
TAPEDEV /dev/null
TAPEBLK 32
TAPESIZE 0
LTAPEDEV /dev/null
LTAPEBLK 32
LTAPESIZE 0
BAR_ACT_LOG $INFORMIXDIR/tmp/bar_act.log
BAR_DEBUG_LOG $INFORMIXDIR/tmp/bar_dbug.log
BAR_DEBUG 0
BAR_MAX_BACKUP 0
BAR_RETRY 1
BAR_NB_XPORT_COUNT 20
BAR_XFER_BUF_SIZE 31
RESTARTABLE_RESTORE ON
BAR_PROGRESS_FREQ 0
BAR_BSALIB_PATH $INFORMIXDIR/lib/libbsapsm.so
BACKUP_FILTER
RESTORE_FILTER
BAR_PERFORMANCE 0
BAR_CKPTSEC_TIMEOUT 15
PSM_DBS_POOL DBSPOOL
PSM_LOG_POOL LOGPOOL
DD_HASHSIZE 31
DD_HASHMAX 10
DS_HASHSIZE 31
DS_POOLSIZE 127
PC_HASHSIZE 31
PC_POOLSIZE 127
PRELOAD_DLL_FILE
STMT_CACHE 0
STMT_CACHE_HITS 0
STMT_CACHE_SIZE 512
STMT_CACHE_NOLIMIT 0
STMT_CACHE_NUMPOOL 1
USEOSTIME 0
STACKSIZE 64
ALLOW_NEWLINE 0
USELASTCOMMITTED "NONE"
FILLFACTOR 90
MAX_FILL_DATA_PAGES 0
BTSCANNER num=1,threshold=5000,rangesize=-1,alice=6,compression=default
ONLIDX_MAXMEM 5242880
MAX_PDQPRIORITY 100
DS_MAX_QUERIES 5
DS_TOTAL_MEMORY 26214400
DS_MAX_SCANS 5
DS_NONPDQ_QUERY_MEM 1310720
DATASKIP
OPTCOMPIND 2
DIRECTIVES 1
EXT_DIRECTIVES 0
OPT_GOAL -1
IFX_FOLDVIEW 1
STATCHANGE 10
USTLOW_SAMPLE 1
BATCHEDREAD_TABLE 1
BATCHEDREAD_INDEX 1
EXPLAIN_STAT 1
IFX_EXTEND_ROLE 1
SECURITY_LOCALCONNECTION
UNSECURE_ONSTAT
ADMIN_USER_MODE_WITH_DBSA
ADMIN_MODE_USERS
SSL_KEYSTORE_LABEL
TLS_VERSION
PLCY_POOLSIZE 127
PLCY_HASHSIZE 31
USRC_POOLSIZE 127
USRC_HASHSIZE 31
SQL_LOGICAL_CHAR OFF
SEQ_CACHE_SIZE 10
ENCRYPT_HDR
ENCRYPT_SMX
ENCRYPT_CDR 0
ENCRYPT_CIPHERS
ENCRYPT_MAC
ENCRYPT_MACFILE
ENCRYPT_SWITCH
CDR_EVALTHREADS 1,2
CDR_DSLOCKWAIT 5
CDR_QUEUEMEM 4096
CDR_NIFCOMPRESS 0
CDR_SERIAL 0
CDR_DBSPACE
CDR_QHDR_DBSPACE
CDR_QDATA_SBSPACE
CDR_SUPPRESS_ATSRISWARN
CDR_DELAY_PURGE_DTC 0
CDR_LOG_LAG_ACTION ddrblock
CDR_LOG_STAGING_MAXSIZE 0
CDR_MAX_DYNAMIC_LOGS 0
GRIDCOPY_DIR $INFORMIXDIR
CDR_TSINSTANCEID 0
CDR_MAX_FLUSH_SIZE 50
CDR_AUTO_DISCOVER 0
CDR_MEM 0
DRAUTO 0
DRINTERVAL 0
HDR_TXN_SCOPE NEAR_SYNC
DRTIMEOUT 30
HA_ALIAS
HA_FOC_ORDER SDS,HDR,RSS
DRLOSTFOUND $INFORMIXDIR/etc/dr.lostfound
DRIDXAUTO 0
LOG_INDEX_BUILDS
SDS_ENABLE
SDS_TIMEOUT 20
SDS_TEMPDBS
SDS_PAGING
SDS_LOGCHECK 10
SDS_ALTERNATE NONE
SDS_FLOW_CONTROL 0
UPDATABLE_SECONDARY 0
FAILOVER_CALLBACK
FAILOVER_TX_TIMEOUT 0
TEMPTAB_NOLOG 0
DELAY_APPLY 0
STOP_APPLY 0
LOG_STAGING_DIR
RSS_FLOW_CONTROL 0
SMX_NUMPIPES 1
ENABLE_SNAPSHOT_COPY 0
SMX_COMPRESS 0
SMX_PING_INTERVAL 10
SMX_PING_RETRY 6
CLUSTER_TXN_SCOPE SERVER
ON_RECVRY_THREADS 2
OFF_RECVRY_THREADS 5
DUMPDIR $INFORMIXDIR/tmp
DUMPSHMEM 1
DUMPGCORE 0
DUMPCORE 0
DUMPCNT 1
ALARMPROGRAM $INFORMIXDIR/etc/alarmprogram.sh
ALRM_ALL_EVENTS 0
STORAGE_FULL_ALARM 600,3
SYSALARMPROGRAM $INFORMIXDIR/etc/evidence.sh
RAS_PLOG_SPEED 14896
RAS_LLOG_SPEED 0
EILSEQ_COMPAT_MODE 0
QSTATS 0
WSTATS 0
USERMAPPING OFF
SP_AUTOEXPAND 1
SP_THRESHOLD 0
SP_WAITTIME 30
AUTOLOCATE 0
DEFAULTESCCHAR \
MQSERVER
MQCHLLIB
MQCHLTAB
REMOTE_SERVER_CFG
REMOTE_USERS_CFG
S6_USE_REMOTE_SERVER_CFG 0
LOW_MEMORY_RESERVE 0
LOW_MEMORY_MGR 0
GSKIT_VERSION
INFORMIXCONTIME 60
INFORMIXCONRETRY 1
JVPPROPFILE $INFORMIXDIR/extend/krakatoa/.jvpprops
JVPLOGFILE $INFORMIXDIR/tmp/jvp.log
JVPARGS -Dcom.ibm.tools.attach.enable=no
JVPCLASSPATH INFORMIXDIR/extend/krakatoa/jdbc.jar
BUFFERPOOL default,memory='auto'
BUFFERPOOL size=2k,memory=500MB
NETTYPE onsoctcp,1,150,NET
NETTYPE drsoctcp,1,150,NET
AUTO_TUNE_SERVER_SIZE SMALL
Thank you for any help!
I am using MPI in C. I was able to distribute parts of an array to different processors. And the different processors did all the manipulation I wanted. Now I am at the point where I wanted to combine all the sub-arrays that are in all the processors into one big array. For example if the different processors had sub-arrrays as follows:
Processor 1:
0 1 1 0
0 0 1 0
Processor 2:
0 0 1 0
1 1 0 1
Processor 3:
1 1 0 0
1 1 1 1
...
I want to be able to combine, or "concatenate", all the sub-arrays together. For example I would want the large array to be:
0 1 1 0
0 0 1 0
0 0 1 0
1 1 0 1
1 1 0 0
1 1 1 1
...
I was trying to use MPI_Reduce but I couldn't find an operation to do what I wanted to do. Is there another MPI method I could use to achieve what I am looking for?
You are looking for MPI_Gather:
Each process (root process included) sends the contents of its send buffer to the root process. The root process receives the messages and stores them in rank order.
For documentation and examples, see here and here. The section 5.5 in the MPI 2.2 Standard also has examples.
I have a binary image in Matlab, and I need the binary array(0 and 1) in a C program,
How can I import it?
I can save the binary image in Matlab as ".mat" but I don't know how to read the mat file in C.
Also I thought maybe I can save the binary image in Matlab as ".bin" which is readable in "C" but again I don't know how to save an image in matlab as ".bin".
Any help would be greatly appreciated.
You can also write your data to a binary file, which will be about twice smaller then text.
img=rand(10)>.5; %# an array from #Amro's answer
fid = fopen('test.bin','w');
fwrite(fid,img,'int8')
fclose(fid);
Then I believe you can read it in C without problems. Remember, MATLAB will write data by columns, not by rows - first 1st column, then 2nd, etc. If you want to read it in C by rows, just transpose the array before writing:
fwrite(fid,img','int8') %# notice ' symbol after img
First of all, check the net using a searching engine. The place to search in first is of course Mathworks, where you can find docs like this. The second step, since it looks complicated to write from scratch a converter, is to search if your need is already someone else need. A link like this could be useful, but there's the requirements of a MATLAB itself (linking with one of its libraries, the MAT-file interface library I suppose). I believe it is however the simpler thing to do.
The next simple solution is to use this with a format we know we can read, If the input array is of class logical, imwrite assumes the data is a binary image and writes it to the file with a bit depth of 1, if the format allows it. BMP, PNG, or TIFF formats accept binary images as input arrays.
BMP images are not hard to be handled by custom routines; for PNG images, you can use png library, and for TIFF, I can't remember, but you can search or choose the other options.
Here other ways of saving data are explained; a plain ascii file could be also easy to be parsed by custom C routines.
>> img=rand(10)>.5
img =
1 0 1 1 0 0 1 1 0 0
1 1 0 0 0 1 0 0 1 0
0 1 1 0 1 1 1 1 1 1
1 0 1 0 1 0 1 0 1 1
1 1 1 0 0 0 1 1 1 1
0 0 1 1 0 0 1 0 0 0
0 0 1 1 0 1 1 0 1 1
1 1 0 0 1 0 0 0 1 0
1 1 1 1 1 1 0 1 0 0
1 1 0 0 1 0 0 0 1 0
>> dlmwrite('file.dat', img, 'delimiter',' ')
If you need to export array data from Matlab and import it into a C program, one easy option (assuming the array isn't gigantic) is to use a plain text file as in intermediate. From Matlab, write the array out as strings to a text file with each element separated by a comma (for example, 1,0,0,1,0,1,1,etc). Save this as a regular ASCII text file. Now, you can open the file in C with fopen, read in a character at a time with fgetc, and reconstruct the array.
For example, the C portion might look like:
FILE* fp;
char inp[3];
int array[PLENTY_OF_ROOM], index=0;
fp = fopen("exported_data.txt", "r");
while (fgets(inp, 3, fp) != NULL) {
if ((inp[0] == '0' || inp[0] == '1') && inp[1] == ',')
array[index++] = inp[0] - '0';
else {
fprintf(stderr, "Error: bad format\n");
exit(1);
}
}
On some platforms (like Windows) you have to specify "binary" when opening the file, otherwise read/written bytes can be interpreted as special characters and may not be read/written to the file as expected:
FILE* fin,fout;
...
fin = fopen("file1.bin", "rb"); // the 'b' indicates "binary"
fout = fopen("file1.bin", "wb"); // the 'b' indicates "binary"