I want change drive where I keep my data of postgresql on VPS server with Ubuntu 18.04.
Now it's default:
data_directory = '/var/lib/postgresql/10/main'
on /dev/sda. I want keep this on /dev/sdb. I have created partition /dev/sdb1 properly.
I find tutorial: https://www.tutorialspoint.com/how-to-change-postgresql-data-folder-location-on-ubuntu-16-04
I did the same steps like above, means:
Stop postgresql process
Copy all data from /var/lib/postgresql to my second drive to /second_drive/postgresql_data
Change 'data_directory' in my postgresql.conf
Start postgresql - and here I have problem. DB doesn't starting. I don't remember error, and I can't check this now because my db have to working whole weekend.
Anyone has a idea what's causes problem? Something another to change too? I looking info in google, but all tuts looking like than above.
EDIT:
I remembered that the problem was with the permissions for postgres user.
I tried now again change the permissions for directory with data, but chown not working - I mean, terminal not return any error/message, but the permissions/ownership not change. I find info that's problem is because I tried change permissions on mounted drive, but it's not possible on NTFS drives. I'm confused about that. My current data is on /dev/md3 to /disk_ssd_3
I putting info from my linux below:
/etc/fstab - I don't know where is my /disk_ssd_3
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/md3 / ext4 errors=remount-ro 0 1
/dev/md2 /boot ext4 errors=remount-ro 0 1
/dev/nvme0n1p4 swap swap defaults 0 0
/dev/nvme1n1p4 swap swap defaults 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs defaults 0 0
UUID=E368-416A /boot/efi vfat defaults 0 0
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 1.8T 0 disk
└─sda1 8:1 0 1.8T 0 part /disk_hdd_1
sdb 8:16 0 1.8T 0 disk
└─sdb1 8:17 0 1.8T 0 part /disk_hdd_2
nvme1n1 259:0 0 419.2G 0 disk
└─nvme1n1p1 259:6 0 419.2G 0 part /disk_ssd_3
nvme0n1 259:1 0 419.2G 0 disk
├─nvme0n1p1 259:2 0 511M 0 part /boot/efi
├─nvme0n1p2 259:3 0 511M 0 part
│ └─md2 9:2 0 511M 0 raid1 /boot
├─nvme0n1p3 259:4 0 417.7G 0 part
│ └─md3 9:3 0 417.7G 0 raid1 /
└─nvme0n1p4 259:5 0 511M 0 part [SWAP]
Related
I am trying to retrieve data from an Excel file format in a GitHub repository using the !curl command in Jupyter-Notebook, or alternatively from a file stored locally on my PC.
I created a new Jupyter Notebook and ran a command to access an http domain on GitHub but Notebook returned the following error - 'Not Found' and the display of an empty table of results. The aim is to try and follow instructions given in a course tutorial to see if I can create a functional notebook with access to data.
!curl https://github.com/lynstanford/JupyterBUTI/BUTI_Copy.xlsx
What have I done wrong? Do I need to save the file in .csv format?
Error Message returns: 'Not Found'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
100 9 0 9 0 0 2 0 --:--:-- 0:00:03 --:--:-- 2
100 9 0 9 0 0 2 0 --:--:-- 0:00:03 --:--:-- 2
I found this information on 'notebooks.azure.com'. I simply need to set up my Working Directory for Jupyter Notebooks correctly so the !curl command will access and read the file from the correct location. A Jupyter Notebook example of the command.
about a week ago, I deleted all files from my hard drive using rm -rf * and rm -rf .*, thinking that all important files would be protected, but it seems like I was wrong.
After reboot, it wasn't able to mount the drive anymore.. I tried a couple more things to make it work (fill the disk with zeros, with the dd command), but I fear I only made things worse.
Now, lsblk command returns:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 3,9G 0 disk
sdb 8:16 0 465,8G 0 disk
├─sdb1 8:17 0 448,6G 0 part /
└─sdb2 8:18 0 17,2G 0 part [SWAP]
The disk is supposed to be 1Tb, so there's a problem here. All of the partition or file system tools I tried get stuck when scanning the drive (I guess), including sfdisk, wipefs, fsck, and the gui KDE disk manager.
I'm under Manjaro, KDE, but that shouldn't be relevant...
Thanks!
For those reaching here; Unfortunately I could not recover the data, after various tries and reproducing the problem it was too costy to keep trying, so we just used a past backup to recreate the information needed
A human error broke an 150G UFS filesystem (Solaris).
When trying to do a backup of the filesytem (c0t0d0s3) the ufsdump(1M) hasn't been correctly used.
I will explain the background that led to this ...
The admin used:
# ufsdump 0f /dev/dsk/c0t0d0s3 > output_1
root#ats-br000432 # ufsdump 0f /dev/dsk/c0t0d0s3 > output_1
Usage: ufsdump [0123456789fustdWwnNDCcbavloS [argument]] filesystem
This is a bad usage, so it created a file called output_1 with 0 bytes:
# ls -la output_1
-rw-r--r-- 1 root root 0 abr 12 14:12 output_1
Then, the syntax used was:
# ufsdump 0f /dev/rdsk/c0t0d0s3 output_1
Which wrote that 0 bytes file output_1 to /dev/rdsk/c0t0d0s3 - which was the partition slice
Now, interestingly, due to being a 0 bytes file, we thought that this would cause no harm to the filesystem, but it did.
When trying to ls in the mountpoint, the partition claimed there was an I/O error, when umounting and mounting again, the filesystem showed no contents, but the disk space was still showing as used just like it was previously.
I assume, at some point, the filesystem 'header' was affected, right? Or was it the slice information?
A small fsck try brings up this:
** /dev/rdsk/c0t0d0s3
** Last Mounted on /database
** Phase 1 - Check Blocks and Sizes
INCORRECT DISK BLOCK COUNT I=11 (400 should be 208)
CORRECT?
Disk block count / I=11
this seems that the command broke filesystem information regarding its own contents, right?
When we tried to fsck -y -F ufs /dev/dsk.. various files have been recovered, but not the dbf files we are after (which are GB sized)
What can be done now? Should I try every superblock information from newfs -N ?
EDIT: new information regarding partition
newfs output showing superblock information
# newfs -N /dev/rdsk/c0t0d0s3
Warning: 2826 sector(s) in last cylinder unallocated
/dev/rdsk/c0t0d0s3: 265104630 sectors in 43149 cylinders of 48 tracks, 128 sectors
129445,6MB in 2697 cyl groups (16 c/g, 48,00MB/g, 5824 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
98464, 196896, 295328, 393760, 492192, 590624, 689056, 787488, 885920,
Initializing cylinder groups:
.....................................................
super-block backups for last 10 cylinder groups at:
264150944, 264241184, 264339616, 264438048, 264536480, 264634912, 264733344,
264831776, 264930208, 265028640
How do I check with C if a port on my local machine (if required by passing an IP or interface, too), is in listen state? I don't want to connect to this port for checking because I don't want to irritate the service behind this port.
I want to use this to add the missing net.tcp.listen item to Zabbix.
EDIT - THIS IS THE REAL ANSWER:
The correct way is to read the socket tables:
/proc/net/tcp
/proc/net/tcp6
They contain lines like:
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:1F40 00000000:0000 0A 00000000:00000000 00:00000000 00000000 101 0 4083927 1 f5d15240 750 0 0 2 -1
1: 00000000:2742 00000000:0000 0A 00000000:00000000 00:00000000 00000000 1002 0 6100 1 decd76c0 750 0 0 2 -1
and can easily parsed for listening sockets (dst:00000000:0000). An strace on netstat shows that netstat works the same way.
Just try to open the port for listening. You will get an error.
There is no way you can steal a port from another process, so this will be safe, and of course easy to implement as it requires no additional code other than proper error handling.
The correct way is to read the socket tables:
/proc/net/tcp
/proc/net/tcp6
They contain lines like:
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:1F40 00000000:0000 0A 00000000:00000000 00:00000000 00000000 101 0 4083927 1 f5d15240 750 0 0 2 -1
1: 00000000:2742 00000000:0000 0A 00000000:00000000 00:00000000 00000000 1002 0 6100 1 decd76c0 750 0 0 2 -1
and can easily parsed for listening sockets (dst:00000000:0000). An strace on netstat shows that netstat works the same way.
Something like netstat ?
`netstat -anp`
Will list all the ports and should do the trick.
Since this is open-source software, you can probably take some inspiration in it. And i believe it is written in C.
As per your query it seems to be that you want to check open port on your server, but on sure which command need to be run. No need to worries for example we try to found out if port 80 open on your server.
First login into server as root user
root#[~]# whoami
root
Now run following command.
root#[~]# netstat -anp | grep :80
It will give following output if port 80 open.
tcp 0 0 78.129.128.40:80 207.46.12.20:62746 TIME_WAIT -
tcp 0 0 78.129.128.40:80 74.53.3.132:47003 TIME_WAIT -
tcp 0 0 78.129.128.40:80 85.9.80.110:63325 FIN_WAIT2 -
And if port 80 blocked in your server you will not receive any output.
You can also use following URL to check open ports on your server.
http://www.yougetsignal.com/tools/open-ports/
Regards,
Gunjan
I don't know if you just want to see if your ports are open properly or if you want to integrate this into an application somehow. If yours is the prior case
nmap localhost
should handle it for you
Is there a Linux library that will enable me to tell what IP sockets are owned by what processes? I guess I'm looking for the programmatic equivalent of lsof -i. Ultimately, I want to correlate packets seen through libpcap to processes.
UPDATE: A couple of people have suggested using /proc/<pid>/net/tcp and udp, but on my system, the same data is shown for every process, so it doesn't help.
I think you first have to look through the open fds in /proc/*/fd, e.g.
4 -> socket:[11147]
and then look for the referenced sockets (by the inode) in /proc/net/tcp (or /proc/net/udp), e.g.
12: B382595D:8B40 D5C43B45:0050 01 00000000:00000000 00:00000000 00000000 1000 0 11065 1 ffff88008bd35480 69 4 12 4 -1
To determine sockets owned by a process you can just use netstat. Here's an example w/output (shortened) of netstat with options that will do what you want.
$ sudo netstat -apeen
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
tcp 0 0 127.0.0.1:8118 0.0.0.0:* LISTEN 138 744850 13248/privoxy
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 117 9612 2019/postgres
udp 0 0 127.0.0.1:51960 127.0.0.1:51960 ESTABLISHED 117 7957 2019/postgres
udp 0 0 0.0.0.0:68 0.0.0.0:* 0 7740 1989/dhclient
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node PID/Program name Path
unix 2 [ ACC ] STREAM LISTENING 7937 2019/postgres /var/run/postgresql/.s.PGSQL.5432
unix 2 [ ACC ] STREAM LISTENING 958058 8080/emacs /tmp/emacs1000/server
unix 2 [ ACC ] STREAM LISTENING 6969 1625/Xorg /tmp/.X11-unix/X0
unix 2 [ ] DGRAM 9325 1989/dhclient
unix 3 [ ] STREAM CONNECTED 7720 1625/Xorg #/tmp/.X11-unix/X0
Make sure you run netstat as root otherwise you'll get this message:
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
An explanation of the -apeen options from the netstat manpage:
-a, --all
Show both listening and non-listening sockets. With the
--interfaces option, show interfaces that are not up
-p, --program
Show the PID and name of the program to which each socket
belongs.
-e, --extend
Display additional information. Use this option twice for
maximum detail.
--numeric , -n
Show numerical addresses instead of trying to determine symbolic host, port or user names.
--numeric-hosts
shows numerical host addresses but does not affect the resolution of port or user names.
--numeric-ports
shows numerical port numbers but does not affect the resolution of host or user names.
--numeric-users
shows numerical user IDs but does not affect the resolution of host or port names.
The /proc filesystem provides details on each process, including networking information. Open socket information is listed in /proc/net/tcp. The IPv6 sockets are listed separately in the tcp6 file. The socket information includes information such as the local and remote ports, and the socket inode number, which can be mapped back to the process by parsing the /proc/{pid}/fd/* information.
If you aren't familiar with the /proc filesystem, it is basically a virtual filesystem that allows the kernel to publish all sorts of useful information to user-space. The files are normally simple structured text files that are easy to parse.
For example, on my Ubuntu system I used netcat for testing, and ran nc -l -p 8321 to listen on port 8321. Looking at the tcp socket information:
$ cat /proc/net/tcp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:2081 00000000:0000 0A 00000000:00000000 00:00000000 00000000 1000 0 26442 1 de0c8e40 300 0 0 2 -1
1: 0100007F:0277 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 7019 1 de0c84c0 300 0 0 2 -1
The first line shows it is listening on all addresses to point 8321 (0x2081). The inode number is 26442, which we can use to look up the matching pid in /proc/{pid}/fd/*, which consists of a bunch of symlinks from the file handle number to the device. So if we look up the pid for netcat, and check its fd mapping:
$ ls -l /proc/7266/fd
total 0
lrwx------ 1 gavinb gavinb 64 2009-12-31 09:10 0 -> /dev/pts/1
lrwx------ 1 gavinb gavinb 64 2009-12-31 09:10 1 -> /dev/pts/1
lrwx------ 1 gavinb gavinb 64 2009-12-31 09:10 2 -> /dev/pts/1
lrwx------ 1 gavinb gavinb 64 2009-12-31 09:10 3 -> socket:[26442]
And there we see that file descriptor 3 in this process is mapped to the socket with inode 26442, just as we expect.
So obviously to build a complete map of sockets, you will need to first enumerate all the /proc/**/fd/* files, look up the socket symlinks, then match the socket inode against the tables from /proc/net/tcp which has the endpoint information.
This is the way the lsof tool works (see lsof/dialects/linux/dsocket.c for the implementation).
Wikipedia on procfs
The Linux /proc filesystem as a Programmer's Tool
/proc/<pid>/net is equivalent to /proc/net for all processes in the same network namespace as you – in other words, it's "global" information.
You can do what lsof and fuser do, which is to iterate through both /proc/<pid>/fd/* and /proc/net/* looking for matching inodes. Quick demonstration:
#!/bin/sh
pgrep "$#" | while read pid; do
for fd in /proc/$pid/fd/*; do
name=$(readlink $fd)
case $name in
socket:\[*\])
ino=${name#*:}
for proto in tcp:10 tcp6:10 udp:10 udp6:10 unix:7; do
[[ ! -e /proc/net/${proto%:*} ]] ||
awk "
\$${proto##*:} == ${ino:1:${#ino}-2} {
print \"${proto%:*}:\", \$0
exit 1
}
" /proc/net/${proto%:*} || break
done
;;
esac
done
done
You can extend this to other protocols (I see ax25, ipx, packet, raw, raw6, udplite, udp6lite in /proc/net/ too) or rewrite in a language of your choosing.
You can read them from proc filesystem. The 'files' you probably want to look at are found in
/proc/<pid>/net (namely tcp, udp, unix)
Here's some examples on using the proc filesystem
You could try running lsof with strace and see just which files in /proc it gets data from.
I'd go to the source:
http://ubuntuforums.org/showthread.php?t=1346778