Most frequent ps command example to monitor linux process and troubleshooting

by Anish

Posted on Friday September 13

The process status (ps)

The ps utility displays a header line, followed by lines containing information about all of your processes that have controlling terminal. The ps command only produces a snapshot of processes, meaning the information produced is static

This is the first command where most of the system administrator uses to troubleshoot the environment so in this article, we will quickly discuss ps related tips that users may find useful.

By simply typing ps will display the process for the current shell

# ps
PID TTY  TIME CMD
20048 pts/0  00:00:00 sudo
20049 pts/0  00:00:00 su
20050 pts/0  00:00:00 bash
20241 pts/0  00:00:00 ps

Note the time format is 00:00:00 indicates no CPU time has been given by the kernel

The output headers and their meaning

• PID: Process ID
• TTY: full name of control terminal
• TIME: accumulated CPU time, user + system (alias cputime)
• CMD: Command and Arguments

Display information about other users’ processes

The ps -A will display information about other users’ processes, including those without controlling terminals.

 # ps -A 
  PID TTY          TIME CMD
    1 ?        00:01:15 systemd
    2 ?        00:00:00 kthreadd
    3 ?        00:35:29 ksoftirqd/0
    .....

ps -e Identical to ps -A. many people prefer ps -e

Display all Information

the ps -l will display information associated with the following keywords: uid, pid, ppid, flags, cpu, pri, nice, vsz=SZ, rss, wchan, state=S, paddr=ADDR, tty, time, and command=CMD

# ps -l 
F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
4 S     0 20048 20009  0  80   0 - 13233 poll_s pts/0    00:00:00 sudo
4 S     0 20049 20048  0  80   0 - 13071 wait   pts/0    00:00:00 su
4 S     0 20050 20049  0  80   0 -  5340 wait   pts/0    00:00:00 bash
0 R     0 20489 20050  0  80   0 -  7229 -      pts/0    00:00:00 ps

The output headers and their meaning

• F: Flag value (1=forked but didn’t exec, 4=used super-user privileges)
• S: STAT refer below.
• UID : effective user ID
• PID : process ID
• PPID : parent process ID
• WCHAN The address in the system on which a process waits.
• ADDR: swap address
• TTY: full name of control terminal
• TIME: accumulated CPU time, user + system (alias cputime)
• CMD: Command and Arguments

the ps -u will display information associated with the following keywords: user, pid, %cpu, %mem, vsz, rss, tt, state, start, time, and command

# ps -u 
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      1048  0.0  0.0  15936  1500 tty1     Ss+  Jul13   0:00 /sbin/agetty --noclear tty1 linux
root     20048  0.0  0.0  52932  3996 pts/0    S    12:22   0:00 sudo su
root     20049  0.0  0.0  52284  3568 pts/0    S    12:22   0:00 su
root     20050  0.0  0.0  21360  3832 pts/0    S    12:22   0:00 bash

The output headers and their meaning

• USER: The username from uid.
• %CPU: percentage CPU usage
• %MEM: percentage memory usage
• STAT: symbolic process state
  • R running or runnable (on run queue)
  • D uninterruptible sleep (usually IO)
  • S interruptible sleep (waiting for an event to complete)
  • Z defunct/zombie, terminated but not reaped by its parent
  • T stopped, either by a job control signal or because it is being traced

For BSD formats and when the stat keyword is used, additional characters may be displayed:

• BSD formats

  • < high-priority (not nice to other users)
  • N low-priority (nice to other users)
  • L has pages locked into memory (for real-time and custom IO)
  • s is a session leader
  • l is multi-threaded (using CLONE_THREAD, like NPTL pthreads do)
  • + is in the foreground process group

• START : time started

Display every process on the system

ps -e
ps -ef
ps -eF
ps -ely

Display process tree:

ps -ejH
ps axjf

Display info about threads:

ps -eLf
ps axms

Display processes owned by specific users

ps -u ansible,www-data

Display security info

This will attach an extra header called LABEL , and show security context for SELinux

ps -eo euser,ruser,suser,fuser,f,comm,label
ps axZ

ps -eM
/usr/sbin/mysqld (enforce)  19926 ?  78-11:31:17 mysqld
unconfined  20005 ?  00:00:00 sshd
unconfined  20009 pts/0  00:00:00 bash
unconfined  20048 pts/0  00:00:00 sudo

Display process running as root

ps -U root -u root u
USER PID %CPU %MEM  VSZ RSS TTY  STAT START TIME COMMAND
root 1  0.0  0.1  37772  5176 ?  Ss Jul13 1:15 /sbin/init
root 2  0.0  0.0  0 0 ?  S  Jul13 0:00 [kthreadd]
root 3  0.0  0.0  0 0 ?  S  Jul13  35:31 [ksoftirqd/0]

The output headers and their meaning

• %MEM: percentage memory usage
• VSZ : virtual size in Kbytes.
• RSS : Resident set size

Display every process with a user-defined format:

Using the -o or -format options to build user-defined output formats as shown below.

ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
ps -Ao pid,tt,user,fname,tmout,f,wchan

Display process by command name

ps -C mysqld
PID TTY  TIME CMD
19926 ?  78-10:23:55 mysqld

Display the process ID of mysqld

ps -C mysqld -o pid=

Display the process name of PID 19926:

ps -q  19926  -o comm=
mysqld

Display children of the process

 ps -aef --forest
 UID  PID  PPID  C STIME TTY  TIME CMD
 root 2 0  0 Jul13 ?  00:00:00 [kthreadd]
 root 3 2  0 Jul13 ?  00:35:31  \_ [ksoftirqd/0]
 root 5 2  0 Jul13 ?  00:00:00  \_ [kworker/0:0H]
 root 7 2  0 Jul13 ?  01:49:34  \_ [rcu_sched]
 root 8 2  0 Jul13 ?  00:00:00  \_ [rcu_bh]

To find children of the process pass the PID of the process -p <pid> in the pstree command

pstree -p 19926
mysqld(19926)─┬─{mysqld}(19933)
├─{mysqld}(19934)
├─{mysqld}(19935)
├─{mysqld}(19936)
├─{mysqld}(19937)
├─{mysqld}(19938)

Display threads

use the -L command line option to display all the threads attached with that processid

ps -L 19926
PID LWP TTY  STAT TIME COMMAND
19926 19926 ?  Ssl  2990:32 /usr/sbin/mysqld
19926 19933 ?  Ssl  0:00 /usr/sbin/mysqld
19926 19934 ?  Ssl  1:59 /usr/sbin/mysqld

The output headers and their meaning

• LWP : Light Weight Process.

Display full command executed by the process

ps -efww

root 900 1  0 Jul13 ?  00:00:02 /usr/lib/policykit-1/polkitd --no-debug
root 906 1  0 Jul13 ?  00:00:00 /sbin/dhclient -1 -v -pf /run/dhclient.ens3.pid -lf /var/lib/dhcp/dhclient.ens3.leases -I -df /var/lib/dhcp/dhclient6.ens3.leases ens3
root 927 1  0 Jul13 ?  00:00:00 /sbin/mdadm --monitor --pid-file /run/mdadm/monitor.pid --daemonise --scan --syslog
.....
.....

Display processes attached to device

the -T will display information about processes attached to the device associated with the standard input.

the t Display information about processes attached to the specified terminal devices.

Troubleshoot with ps

Sort process by Start time

ps -ef --sort=start_time

Sort process by cpu usage

ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head

PID  PPID CMD %MEM %CPU
19926 1 /usr/sbin/mysqld 9.9  125
7374  5367 /usr/lib/jvm/java-8-openjdk 26.1 32.1
27969 27959 python lsat.py 1.0 20.0
25821 1 Xtightvnc :2 -desktop X -au  3.0  3.0

Sort process by memory usage

ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head

PID  PPID CMD %MEM %CPU
7374  5367 /usr/lib/jvm/java-8-openjdk 26.1 32.1
19926 1 /usr/sbin/mysqld 9.9  125
25821 1 Xtightvnc :2 -desktop X -au  3.0  3.0
27969 27959 python lsat.py 1.0 20.0
25884 1 xfdesktop  0.8  0.0

Real time monitoring

using with watch command , to find which process is causing high cpu

watch -n 1 'ps -aux | sort -nrk 4 | head'

-n is to specify the time interval, execute the line every 1 second. Ctrl+c to quit watch.

or

watch -n 1 'ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu'

which process is eating memory

watch -n 1 'ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem'

Watching specific command for the troubleshoot

 ps u -C mysqld

USER PID %CPU %MEM  VSZ RSS TTY  STAT START TIME COMMAND
mysql  19926  125  9.9 1676224 404556 ?  Ssl  Jul13 113018:30 /usr/sbin/mysqld

with watch command every 1 second.

watch -n 1 "ps u -C  mysqld"

Every 1.0s: ps u -C  mysqld Fri Sep 14 15:01:33 2018
USER PID %CPU %MEM  VSZ RSS TTY  STAT START TIME COMMAND
mysql  19926  125  9.9 1676224 404556 ?  Rsl  Jul13 113020:52 /usr/sbin/mysqld

Display how many process is created by each user

ps -ef | cut -d " " -f 1 |  sort -n  | uniq -c | sort -nr

121 root
28 ansible
5 www-data
...

Display all Zombie process

ps aux |grep "defunct"

Note: Zombie processes can not be killed with the normal commands like kill -9 PID , you will have to reboot the system to kill them

Display process stuck in D state IOWait

ps aux | awk '{if ($8 ~ "D") print $0}'

Through watch command

watch -d -n 1 "(ps aux | awk '\$8 ~ /D/ { print \$0 }')"

Count number of thread created by a given process

# ps -T 7374  | cut -d " " -f 2  | wc -l
18

ps command to output specific column

This command will output command and arguments returned by the ps command

ps ax | awk -v p='COMMAND' 'NR==1 {n=index($0, p); next} {print substr($0, n)}'

/sbin/init
[kthreadd]
[ksoftirqd/0]
[kworker/0:0]
[kworker/0:0H]
...
....

Things to know

ps -e and ps e is not same. Try out in your shell
ps -ef and ps aux is same, there is historical divergences between POSIX and BSD systems

A note on pgrep/pkill

pgrep & pkill looks through the currently running processes and lists the process IDs which match the selection criteria to stdout

for example this will only list the processes called sshd AND owned by root

# pgrep -u root sshd
973
19905

and pgrep -u root,daemon will list the processes owned by root or daemon.

Killing the process

To kill a process you need the processid and then using SIGKILL or option 9 will terminate the process

for example to kill the mysql process, first find the PID and then kill the process.

kill -9 $(pgrep -f mysql)

or

kill -9 <pid_of_mysql>

if no option is specified , then it will Send the default signal, SIGTERM, to all those processes.

kill pid

To get all the available signal use kill -l, OMG there are 64 signals, each signal carry a specific meaning by the OS.

 kill -l
 
1) SIGHUP    2) SIGINT   3) SIGQUIT  4) SIGILL   5) SIGTRAP
 6) SIGABRT  7) SIGBUS   8) SIGFPE   9) SIGKILL 10) SIGUSR1
11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP
21) SIGTTIN 22) SIGTTOU 23) SIGURG  24) SIGXCPU 25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF 28) SIGWINCH    29) SIGIO   30) SIGPWR
31) SIGSYS  34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX    

For example Taking Java thread dumps

In jRE kill -3 option is used thread dump is sent to the standard error stream.

kill -3 <pid>
or 
kill -QUIT <pid>

Make syslog reread its configuration file

pkill -HUP syslogd
or 
kill -1 syslogd

The killall commands sends a signal to all processes running any of the specified commands. If no signal name is specified, SIGTERM is sent.

Next Reading : How to Monitoring All Executed Commands in Linux

Thanku for reading !!! Give a Share for Support

Your Support Matters!

Instead of directly asking for donations, I'm thrilled to offer you all nine of my books for just $9 on leanpub By grabbing this bundle you not only help cover my coffee, beer, and Amazon bills but also play a crucial role in advancing and refining this project. Your contribution is indispensable, and I'm genuinely grateful for your involvement in this journey!

Any private key value that you enter or we generate is not stored on this site, this tool is provided via an HTTPS URL to ensure that private keys cannot be stolen, for extra security run this software on your network, no cloud dependency