How to Monitoring All Executed Commands in Linux

by Anish

Posted on Tuesday August 21



To safeguards the production environment requires additional control. This can be achieve through logging the user activity and this will be rescued in the scenario like investigating suspicious activity happening on that environment.

The Linux kernel can record a lot of information about processes as they exit. In this article we will go though one of command that can help for audit trail and detective work.

The accton command enables process accounting, and specifies the file used for the audit trail i.e /var/account/pacct .

This file must already exist, so manually create an empty file first if necessary, carefully restricting access to prevent public viewing of the sensitive accounting data.

If the filename is omitted, then the accton command disables process accounting.

  • Prepare the file

       umask 077
      touch /var/account/pacct
  • Install psacct or acct

       RHEL :  yum install psacct
      Ubuntu : sudo apt-get install acct
  • Check the status of psacct if it’s not started then start it

       systemctl status psacct
      psacct.service - Kernel process accounting
      Loaded: loaded (/usr/lib/systemd/system/psacct.service; disabled; vendor preset: disabled)
      Active: inactive (dead)
  • Start the psacct service

    systemctl start psacct
    systemctl status psacct
    psacct.service - Kernel process accounting
    Loaded: loaded (/usr/lib/systemd/system/psacct.service; enabled; vendor preset: disabled)
    Active: **active (exited)** since Tue 2018-08-21 05:17:41 EDT; 56min ago
    Main PID: 9124 (code=exited, status=0/SUCCESS)
    CGroup: /system.slice/psacct.service
    Aug 21 05:17:41 host-192-0-2-74 systemd[1]: Starting Kernel process accounting...
    Aug 21 05:17:41 host-192-0-2-74 accton[9124]: Turning on process accounting, file set to '/var/account/pacct'.
    Aug 21 05:17:41 host-192-0-2-74 systemd[1]: Started Kernel process accounting.


       /etc/init.d/psacct start
      /etc/init.d/acct start    # In Ubuntu
  • Enable this changes in Linux runlevel

       chkconfig psacct on
  • Note: Logrotate file for psacct, other wise your system will fill-up immediately. In centos/rhel the daily rotation specified in the file /etc/logrotate.d/psacct

    /var/account/pacct {
        rotate 31
        create 0600 root root
           if /usr/bin/systemctl --quiet is-active psacct.service ; then
               /usr/sbin/accton /var/account/pacct | /usr/bin/grep -v "Turning on process accounting, file set to '/var/account/pacct'." | /usr/bin/cat

Summarizes accounting (SA)

The Linux sa summarizes information about previously executed commands as recorded in the acct file. In addition, it condenses this data into a summary file named savacct which contains the number of times the command was called and the system resources used. The information can also be summarized on a per-user basis;

Files Used

fileName Description
acct The raw system wide process accounting file
savacct A summary of system process accounting sorted by command
usracct A summary of system process accounting sorted by user ID

sa output summary

label description
cpu sum of system and user time in cpu minutes
re Elapsed time in minutes
k cpu-time averaged core usage, in 1k units
avio average number of I/O operations per execution
tio total number of I/O operations
k*sec cpu storage integral (kilo-core seconds)
u user cpu time in cpu seconds
s system time in cpu seconds

Command Options

  • If no arguments are specified, sa will print information about all of the commands in the acct file

       # sa
      294     428.58re       0.01cp         0avio     23947k
      29     277.34re       0.00cp         0avio     14085k   ***other*
      14       0.00re       0.00cp         0avio     29616k   troff
      14      14.45re       0.00cp         0avio     29712k   man
      98       0.00re       0.00cp         0avio     29770k   man*
      14      14.44re       0.00cp         0avio     27584k   less
      14       0.01re       0.00cp         0avio     28288k   nroff
      14       0.01re       0.00cp         0avio      3190k   groff
      14       0.01re       0.00cp         0avio     29200k   grotty
      14       0.00re       0.00cp         0avio      3198k   tbl
  • User Summary sa -m
    this command will print the number of processes and number of CPU minutes on a per-user basis

    # sa -m 
                                   330     446.58re       0.01cp         0avio     24026k
         root                      330     446.58re       0.01cp         0avio     24026k
  • Show userid and command name executed by each user with sa -u

       sa -u 
      ansible    0.00 cpu    28832k mem      0 io bash            *
      ansible    0.00 cpu    26992k mem      0 io touch           
      ansible    0.00 cpu    28864k mem      0 io bash            
      root       0.01 cpu    31264k mem      0 io su              
      root       0.00 cpu     1624k mem      0 io sa              
      root       0.00 cpu    34960k mem      0 io useradd         *
      root       0.00 cpu    34960k mem      0 io useradd         *
      root       0.00 cpu    34960k mem      0 io useradd         *
      root       0.00 cpu    34960k mem      0 io useradd         *
      root       0.00 cpu    34960k mem      0 io useradd         
      test1      0.00 cpu     3842k mem      0 io id              
      test1      0.00 cpu     2920k mem      0 io bash            *
      test1      0.00 cpu     1626k mem      0 io hostname        
  • last but not least check the help sa -h to explore all the available options

Print statistics about users (AC)

The linux ac commands prints out a report of connect time (in hours) based on the logins/logouts in the current wtmp file. A total is also printed out.

This accounting file wtmp is maintained by init(8) and login(1). Neither ac nor login creates the wtmp if it doesn't exist, no accounting is done. To begin accounting, create the file with a length of zero.

NOTE: The wtmp file can get really big, really fast. You might want to trim it every once and a while.

Some use-case about ac comand line

  • print statistics of user day wise ac -d
    Print totals for each day rather than just one big total at the end. The output looks like this:

       ac -d
      Aug  3  total 1.17
      Today  total  5.64
  • Print total time for each users ac -p

    Print time totals for each user in addition to the usual everything-lumped-into-one value. It looks like:

       ac -p 
      bob 8.06
      goff  0.60
      maley 7.37
      root  0.12
      total  16.15


lastcomm prints out information about previously executed commands, it uses the file acct

  • Find the commands executed by the user test1

        # lastcomm test1
      bash S test1  pts/1  0.00 secs Tue Aug 21 06:09
      touch  test1  pts/1  0.00 secs Tue Aug 21 06:09
      bash  F  test1  pts/1  0.00 secs Tue Aug 21 06:09
      consoletype  test1  pts/1  0.00 secs Tue Aug 21 06:09
      grep test1  pts/1  0.00 secs Tue Aug 21 06:09
  • If no arguments are specified, lastcomm will print info about all of the commands in acct(the record file)

  • finding with lastcomm which user has added the new user, here you can see after few fail attempts the user is created by the root

       # lastcomm adduser
      adduser  root pts/1  0.00 secs Tue Aug 21 06:24
      adduser S  root pts/1  0.00 secs Tue Aug 21 06:24
      adduser F  root pts/1  0.00 secs Tue Aug 21 06:24
      adduser F  root pts/1  0.00 secs Tue Aug 21 06:24
      adduser F  root pts/1  0.00 secs Tue Aug 21 06:24


Last searches back through the file /var/log/wtmp and displays a list of all users logged in (and out) since that file was created.

 # last
root pts/1 Tue Aug 21 04:23 still logged in
root pts/0 Tue Aug 21 03:09 - 05:22  (02:12)
root tty1  Tue Aug 21 03:09 still logged in
reboot system boot  3.10.0-862.el7.x Tue Aug 21 03:08 - 06:29  (03:21)
root tty1  Thu Jul  5 06:59 - 07:00  (00:01)
reboot system boot  3.10.0-862.el7.x Thu Jul  5 06:58 - 07:00  (00:02)
wtmp begins Thu Jul  5 06:58:40 2018


This is not command it's the file on the Linux, Solaris, and BSD operating systems that keeps a history of all logins and logouts. On Linux systems. it is located at /var/log/wtmp. Various commands access wtmp to report login statistics, like who , last, lastb commands.

Monitor changes in File

In production environment certain file integrity needs to be maintained and should be reported for any accidental or purpose fully changes

The linux watch commands allows you to watch the program output change over time. By default, the program is run every 2 seconds. By default, watch will run until interrupted.

 watch -n 10 -d ls -l /etc/shadow

inotifywait from inotify-tools is useful if you want to run a command every time a file (or any files in a directory) change. For example:

 inotifywait -r -m -e modify /var/log | 
   while read path _ file; do 
       echo $path$file modified

Next Reading : Most frequent ps command example to monitor linux process and troubleshooting

Thanku for reading !!! Give a Share for Support

Your Support Matters!

Instead of directly asking for donations, I'm thrilled to offer you all nine of my books for just $9 on leanpub By grabbing this bundle you not only help cover my coffee, beer, and Amazon bills but also play a crucial role in advancing and refining this project. Your contribution is indispensable, and I'm genuinely grateful for your involvement in this journey!

Any private key value that you enter or we generate is not stored on this site, this tool is provided via an HTTPS URL to ensure that private keys cannot be stolen, for extra security run this software on your network, no cloud dependency

python Cryptography Topics
For Coffee/ Beer/ Amazon Bill and further development of the project Support by Purchasing, The Modern Cryptography CookBook for Just $9 Coupon Price

Kubernetes for DevOps

Hello Dockerfile

Cryptography for Python Developers

Cryptography for JavaScript Developers

Go lang ryptography for Developers