INCIDENT 2026-08-29 23:01:52 trigger: php_rss=1857MB ================================================================== load: 1.71 1.06 0.97 10/675 1584981 ncpu:10 mem_avail: 7058MB/13777MB (51.2%) swap_used:1786MB swapout:0/s blocked(D):0 running:13 iowait:0.0% steal:0.0% php: 13 procs / 1857MB heaviest user: madalice (10 procs, 1782MB) apache: 32 procs tcp_estab: 45 --- RSS by user (all processes) --- USER PROCS RSS_MB mysql 1 2615 madalice 10 1780 root 253 1012 nobody 31 561 netdata 7 196 mailman 20 147 cpanellogin 1 62 kurri 4 55 corpkicks 1 52 dovenull 6 48 evolution 11 47 wp-toolkit 3 34 aaa 2 29 lowrange 2 28 zabbix 1 21 dovecot 3 20 named 1 15 infinium 1 14 candyfabcom 1 14 mailnull 1 6 --- php workers by user --- USER PROCS RSS_MB madalice 10 1800 corpkicks 1 56 root 1 23 evolution 1 13 --- vm tuning / memory pressure --- vm.swappiness 30 vm.vfs_cache_pressure 100 vm.min_free_kbytes 67584 vm.overcommit_memory 0 vm.overcommit_ratio 50 Cached: 5223 MB Active(file): 3375 MB Inactive(file): 2165 MB Dirty: 0 MB Writeback: 0 MB SReclaimable: 571 MB CommitLimit: 10984 MB Committed_AS: 8523 MB --- backup / dump / scan processes (page-cache evictors) --- PID USER ELAPSED RSS_KB COMMAND 489033 root 1-20:43:12 6372 /usr/bin/imunify-agent-proxy 1568942 root 01:51 1244 /bin/sh -c bash -c 'sleep $((RANDOM % 1800))' ;imunify-antivirus imunify-patch subscriptions refresh > /dev/null 2>&1 --- top 60 by RSS --- PID PPID USER RSS %CPU STAT ELAPSED COMMAND 506671 1 mysql 2678400 8.4 Ssl 1-20:42:41 /usr/sbin/mariadbd 1584950 510022 madalice 216784 113 R 00:01 php-fpm: pool madalice_com_au 1584951 510022 madalice 204228 107 R 00:01 php-fpm: pool madalice_com_au 508924 1 netdata 179720 1.9 Ssl 1-20:42:10 /usr/sbin/netdata -P /run/netdata/netdata.pid -D 1584949 510022 madalice 175724 112 R 00:01 php-fpm: pool madalice_com_au 83639 1451906 root 168724 0.0 S 07:04:18 spamd child 1583902 510022 madalice 163408 20.2 R 01:07 php-fpm: pool madalice_com_au 1584878 510022 madalice 161460 49.5 R 00:08 php-fpm: pool madalice_com_au 1502904 1451906 root 161432 0.0 S 20:25:30 spamd child 1584760 510022 madalice 159916 37.2 S 00:18 php-fpm: pool madalice_com_au 1584879 510022 madalice 159844 47.0 S 00:08 php-fpm: pool madalice_com_au 1584875 510022 madalice 159616 40.1 S 00:09 php-fpm: pool madalice_com_au 1584874 510022 madalice 159612 41.1 S 00:09 php-fpm: pool madalice_com_au 1584876 510022 madalice 159608 41.7 S 00:09 php-fpm: pool madalice_com_au 509266 1 root 118872 0.0 Ss 1-20:42:07 /usr/lib/systemd/systemd-journald 1451906 1 root 114528 0.0 Ss 20:38:45 /usr/local/cpanel/3rdparty/perl/542/bin/perl -T -w /usr/local/cpanel/3rdparty/bin/spamd --allowed-ips=127.0.0.1,::1 --max-children=5 -- 1584952 1583039 corpkicks 69608 0.0 R 00:00 /opt/cpanel/ea-php83/root/usr/bin/php-cgi 1534856 1353308 cpanellogin 63700 0.0 S 13:06 /usr/local/cpanel/base/show_template.stor cpanel_locale docroot /usr/local/cpanel/base server_locale en default_login_theme cpanel 509236 509098 root 59224 0.0 Sl 1-20:42:07 /opt/saltstack/salt/bin/python3.10 /usr/bin/salt-minion MultiMinionProcessManager MinionProcessManager 1353308 1 root 42044 0.1 Ss 38-20:20:33 cpsrvd (SSL) - waiting for connections 509640 508992 root 41224 0.0 Sl 1-20:42:05 /usr/libexec/netdata/plugins.d/ebpf.plugin 1 1451116 1 root 31796 0.0 Ss 20:39:38 lfd - sleeping 510022 1 root 24212 0.0 Rs 1-20:42:03 php-fpm: master process (/opt/cpanel/ea-php83/root/etc/php-fpm.conf) 1568644 506832 nobody 23516 0.0 S 02:15 /usr/sbin/httpd -k start 1567997 506832 nobody 23320 0.2 S 03:33 /usr/sbin/httpd -k start 1567551 506832 nobody 22732 0.0 S 04:45 /usr/sbin/httpd -k start 1568651 506832 nobody 22416 0.2 S 02:15 /usr/sbin/httpd -k start 1568620 506832 nobody 22332 0.0 S 02:17 /usr/sbin/httpd -k start 1583067 506832 nobody 22332 0.0 S 01:37 /usr/sbin/httpd -k start 1583039 506832 nobody 22312 0.0 S 01:38 /usr/sbin/httpd -k start 483898 1 zabbix 22264 0.2 Ssl 1-20:43:24 /usr/sbin/zabbix_agent2 -c /etc/zabbix/zabbix_agent2.conf 1583040 506832 nobody 21572 0.2 S 01:38 /usr/sbin/httpd -k start 1583122 506832 nobody 21100 0.3 S 01:36 /usr/sbin/httpd -k start 1583107 506832 nobody 21040 0.0 S 01:36 /usr/sbin/httpd -k start 1568652 506832 nobody 20692 0.0 S 02:15 /usr/sbin/httpd -k start 1583110 506832 nobody 20508 0.5 S 01:36 /usr/sbin/httpd -k start 1469 1 root 20456 0.1 Ss 61-12:52:02 cpdavd - accepting connections on: 2079, 2080, 2091, 2077, 2078 1463 1 root 20376 0.0 Ss 61-12:52:02 tailwatchd 938582 1 wp-toolkit 20312 0.0 Ss 1-18:49:44 /usr/bin/sw-engine /usr/local/cpanel/3rdparty/wp-toolkit/bin/run-script background-tasks-executor.php 1584891 506832 nobody 19304 0.1 S 00:07 /usr/sbin/httpd -k start 1584897 506832 nobody 19244 0.3 S 00:06 /usr/sbin/httpd -k start 1584890 506832 nobody 19228 0.1 S 00:07 /usr/sbin/httpd -k start 1584883 506832 nobody 19220 0.1 S 00:08 /usr/sbin/httpd -k start 1584896 506832 nobody 19216 0.1 S 00:06 /usr/sbin/httpd -k start 1584762 506832 nobody 19036 0.0 S 00:18 /usr/sbin/httpd -k start 1583108 506832 nobody 18860 0.0 S 01:36 /usr/sbin/httpd -k start 1583109 506832 nobody 18348 0.2 S 01:36 /usr/sbin/httpd -k start 1584899 506832 nobody 17192 0.0 S 00:06 /usr/sbin/httpd -k start 1584910 506832 nobody 16792 0.0 S 00:05 /usr/sbin/httpd -k start 1584913 506832 nobody 16760 0.0 S 00:05 /usr/sbin/httpd -k start 1584914 506832 nobody 16760 0.0 S 00:05 /usr/sbin/httpd -k start 1584911 506832 nobody 16712 0.0 S 00:05 /usr/sbin/httpd -k start 1584915 506832 nobody 16712 0.0 S 00:05 /usr/sbin/httpd -k start 1584912 506832 nobody 16620 0.0 S 00:05 /usr/sbin/httpd -k start 1566568 1471 root 16604 0.0 S 06:16 cPhulkd - dbprocessor 1584898 506832 nobody 16580 0.0 S 00:06 /usr/sbin/httpd -k start 1568641 506832 nobody 16400 0.0 S 02:15 /usr/sbin/httpd -k start 509671 508992 netdata 15808 2.1 Rl 1-20:42:05 /usr/libexec/netdata/plugins.d/apps.plugin 1 508634 1 named 15604 0.0 Ssl 1-20:42:15 /usr/sbin/pdns_server --socket-dir=/run/pdns --guardian=no --daemon=no --disable-syslog --log-timestamp=no --write-pid=no 1471 1 root 15528 0.0 Ss 61-12:52:02 cPhulkd - processor --- processes in D state (uninterruptible IO) --- --- mariadb processlist (non-sleeping) --- +---------+------+------+---------+------+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ID | USER | DB | COMMAND | TIME | STATE | q | +---------+------+------+---------+------+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 4223310 | root | NULL | Query | 0 | Filling schema table | SELECT ID,USER,DB,COMMAND,TIME,STATE,LEFT(INFO,300) q FROM information_schema.PROCESSLIST WHERE COMMAND<>'Sleep' ORDER BY TIME DESC LIMIT 40 | +---------+------+------+---------+------+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ --- lock waits --- --- vhost traffic RIGHT NOW (last 3 min, deduped by inode) --- VHOST LOG REQ/3min evolution/app.evolutionerp.com.au-ssl_log 1077 corpkicks/2025.corporatekicks.com.au 93 bigdeals/bigdeals.com.au-ssl_log 92 infinium/infinium.net.au-ssl_log 91 corpkicks/2025.corporatekicks.com.au-ssl_log 84 lowrange/lowrange.com.au-ssl_log 55 evolution/portal.evolutionerp.com.au-ssl_log 29 madalice/madalice.com.au-ssl_log 16 alison/alisonarts.com.au-ssl_log 8 aaa/aaadownunder.com.au-ssl_log 5 customc/customcentral.com.au-ssl_log 3 (total vhosts taking traffic: 11) --- top client IPs across all vhosts (last 3 min) --- 1077 40.115.94.245 172 20.63.81.20 92 20.220.10.235 87 108.167.132.89 55 52.139.37.240 29 203.54.180.238 4 34.55.160.83 4 163.227.6.243 4 118.127.40.55 3 67.223.118.185 2 36.152.7.77 2 35.231.130.18 2 162.241.60.164 2 114.119.143.14 2 114.119.140.234 --- top request paths across all vhosts (last 3 min) --- 135 GET /api/getItem/LWPG2001KG 135 GET /api/getItem/LWPB2001KG 135 GET /api/getItem/LWHG2001KG 135 GET /api/getItem/LWFBW2001KG 135 GET /api/getItem/LWFB2001KG 64 GET /wp-login.php 29 GET /heartbeat.php 23 GET /wp-json/wp/v2/users/me 12 POST /wp-login.php 9 GET /api/getItem/NA 6 GET / 4 GET /api/getItem/PWFBW3151KG 3 POST /wp-cron.php 3 POST / 3 GET /zoo1.php 3 GET /xty.php 3 GET /ws85.php 3 GET /wp-content/plugins/hellopress/wp_filemanager.php 3 GET /this_is_a_new_hello_world.php 3 GET /sf.php --- busiest 3 vhosts: top paths + last 10 requests --- == evolution/app.evolutionerp.com.au-ssl_log == -- top paths -- 135 GET /api/getItem/LWPG2001KG 135 GET /api/getItem/LWPB2001KG 135 GET /api/getItem/LWHG2001KG 135 GET /api/getItem/LWFBW2001KG 135 GET /api/getItem/LWFB2001KG 9 GET /api/getItem/NA 4 GET /api/getItem/PWFBW3151KG 2 GET /api/getItem/SHSG8989508 2 GET /api/getItem/SHSG8989358 2 GET /api/getItem/SHSG8989258 2 GET /api/getItem/SHSG8989208 2 GET /api/getItem/SHSG7575508 -- last 10 requests -- 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/RGW120 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/RGW160 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/RGW117T HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/RGW80 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/GP150x200x8 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/K125MSGB HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/253-40SP HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:17 +1000] "GET /api/getItem/300558 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:18 +1000] "GET /api/getItem/SGC-100 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [29/Aug/2026:23:01:18 +1000] "GET /api/getItem/IPC2020 HTTP/1.1" 429 62 "-" "RestSharp/106.13.0.0" == corpkicks/2025.corporatekicks.com.au == -- top paths -- 2 GET /goods.php 1 GET /zoo2.php 1 GET /zoo1.php 1 GET /xty.php 1 GET /x.php 1 GET /wsd.php 1 GET /ws85.php 1 GET /ws55.php 1 GET /wp.php 1 GET /w.php 1 GET /wp-content/uploads/ 1 GET /wp-content/plugins/hellopress/wp_filemanager.php -- last 10 requests -- 20.63.81.20 - - [29/Aug/2026:23:01:32 +1000] "GET /indo.php HTTP/1.1" 301 291 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:34 +1000] "GET /sign.php?p= HTTP/1.1" 301 294 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:36 +1000] "GET /wnnjpsby.php HTTP/1.1" 301 295 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:38 +1000] "GET /gigi.php HTTP/1.1" 301 291 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:40 +1000] "GET /info.php/new.php HTTP/1.1" 301 299 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:41 +1000] "GET /wp-content/uploads/ HTTP/1.1" 301 302 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:43 +1000] "GET /w.php HTTP/1.1" 301 288 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:45 +1000] "GET /x.php HTTP/1.1" 301 288 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:45 +1000] "GET /ssla.php HTTP/1.1" 301 291 "-" "-" 20.63.81.20 - - [29/Aug/2026:23:01:47 +1000] "GET /wp-admin/maint/wp-is.php HTTP/1.1" 301 307 "-" "-" == bigdeals/bigdeals.com.au-ssl_log == -- top paths -- 1 GET /ztv.php 1 GET /zoz.php 1 GET /Zeiss.php 1 GET /zafir1.php 1 GET /yyy.php 1 GET /X7T6.php 1 GET /x23.php 1 GET /ww3.php 1 GET /ww2.php 1 GET /wsz.php 1 GET /ws66.php 1 GET /wp-wlx.php -- last 10 requests -- 20.220.10.235 - - [29/Aug/2026:22:59:45 +1000] "GET /str.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:46 +1000] "GET /mega.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:46 +1000] "GET /ww3.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:47 +1000] "GET /sko.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:47 +1000] "GET /zoz.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:47 +1000] "GET /mmm.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:48 +1000] "GET /advanced.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:48 +1000] "GET /blox.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:49 +1000] "GET /kcs.php HTTP/1.1" 401 36 "-" "-" 20.220.10.235 - - [29/Aug/2026:22:59:49 +1000] "GET /wsz.php HTTP/1.1" 401 36 "-" "-" --- last 30 samples (run-up to this incident) --- 2026-08-29 22:59:30 L1=0.60 run=1 blk=0 avail=8718 (63.3%) swpO=0 conn=1 myRSS=2615 php=3 phpRSS=172 ap=20 tcp=31 topuser=madalice 2026-08-29 22:59:32 L1=0.60 run=1 blk=0 avail=8837 (64.1%) swpO=0 conn=1 myRSS=2615 php=2 phpRSS=37 ap=20 tcp=28 topuser=root 2026-08-29 22:59:40 L1=0.66 run=2 blk=0 avail=8428 (61.2%) swpO=0 conn=1 myRSS=2615 php=4 phpRSS=314 ap=48 tcp=46 topuser=alison 2026-08-29 22:59:42 L1=0.66 run=1 blk=0 avail=8556 (62.1%) swpO=0 conn=1 myRSS=2615 php=3 phpRSS=169 ap=46 tcp=31 topuser=madalice 2026-08-29 22:59:50 L1=0.56 run=3 blk=0 avail=8602 (62.4%) swpO=0 conn=1 myRSS=2615 php=4 phpRSS=215 ap=38 tcp=34 topuser=madalice 2026-08-29 22:59:52 L1=0.56 run=3 blk=0 avail=8502 (61.7%) swpO=0 conn=1 myRSS=2615 php=6 phpRSS=391 ap=36 tcp=39 topuser=vinesit 2026-08-29 23:00:00 L1=0.55 run=3 blk=0 avail=8689 (63.1%) swpO=0 conn=1 myRSS=2615 php=4 phpRSS=202 ap=27 tcp=36 topuser=madalice 2026-08-29 23:00:02 L1=0.55 run=4 blk=0 avail=8592 (62.4%) swpO=0 conn=1 myRSS=2615 php=6 phpRSS=326 ap=26 tcp=32 topuser=evolution 2026-08-29 23:00:10 L1=0.69 run=3 blk=0 avail=8666 (62.9%) swpO=0 conn=1 myRSS=2615 php=4 phpRSS=238 ap=21 tcp=33 topuser=madalice 2026-08-29 23:00:12 L1=0.69 run=2 blk=0 avail=8681 (63.0%) swpO=0 conn=1 myRSS=2615 php=5 phpRSS=279 ap=21 tcp=33 topuser=madalice 2026-08-29 23:00:20 L1=1.41 run=2 blk=0 avail=8563 (62.2%) swpO=0 conn=1 myRSS=2615 php=4 phpRSS=146 ap=48 tcp=35 topuser=evolution 2026-08-29 23:00:22 L1=1.41 run=6 blk=0 avail=8265 (60.0%) swpO=0 conn=1 myRSS=2615 php=6 phpRSS=494 ap=47 tcp=36 topuser=alison 2026-08-29 23:00:31 L1=1.35 run=3 blk=0 avail=8544 (62.0%) swpO=0 conn=1 myRSS=2615 php=5 phpRSS=274 ap=38 tcp=32 topuser=madalice 2026-08-29 23:00:32 L1=1.35 run=5 blk=0 avail=8559 (62.1%) swpO=0 conn=1 myRSS=2615 php=5 phpRSS=259 ap=37 tcp=30 topuser=madalice 2026-08-29 23:00:41 L1=1.44 run=2 blk=0 avail=8618 (62.6%) swpO=0 conn=1 myRSS=2615 php=5 phpRSS=255 ap=28 tcp=34 topuser=madalice 2026-08-29 23:00:42 L1=1.44 run=3 blk=0 avail=8486 (61.6%) swpO=0 conn=4 myRSS=2615 php=6 phpRSS=434 ap=27 tcp=32 topuser=alison 2026-08-29 23:00:51 L1=1.76 run=7 blk=0 avail=8663 (62.9%) swpO=0 conn=1 myRSS=2615 php=5 phpRSS=233 ap=21 tcp=34 topuser=madalice 2026-08-29 23:00:52 L1=1.76 run=3 blk=0 avail=8641 (62.7%) swpO=0 conn=4 myRSS=2615 php=5 phpRSS=309 ap=21 tcp=36 topuser=madalice 2026-08-29 23:01:01 L1=1.88 run=2 blk=0 avail=8698 (63.1%) swpO=0 conn=2 myRSS=2615 php=4 phpRSS=219 ap=19 tcp=29 topuser=madalice 2026-08-29 23:01:02 L1=1.88 run=5 blk=0 avail=8643 (62.7%) swpO=0 conn=4 myRSS=2615 php=4 phpRSS=278 ap=19 tcp=31 topuser=madalice 2026-08-29 23:01:11 L1=1.66 run=3 blk=0 avail=8611 (62.5%) swpO=0 conn=2 myRSS=2615 php=4 phpRSS=322 ap=18 tcp=31 topuser=madalice 2026-08-29 23:01:12 L1=1.66 run=4 blk=0 avail=8618 (62.6%) swpO=0 conn=4 myRSS=2615 php=3 phpRSS=282 ap=18 tcp=30 topuser=madalice 2026-08-29 23:01:21 L1=1.65 run=2 blk=0 avail=8690 (63.1%) swpO=0 conn=2 myRSS=2615 php=3 phpRSS=215 ap=18 tcp=33 topuser=madalice 2026-08-29 23:01:22 L1=1.65 run=2 blk=0 avail=8623 (62.6%) swpO=0 conn=4 myRSS=2615 php=4 phpRSS=309 ap=18 tcp=34 topuser=madalice 2026-08-29 23:01:31 L1=1.39 run=1 blk=0 avail=8705 (63.2%) swpO=0 conn=2 myRSS=2615 php=3 phpRSS=192 ap=18 tcp=31 topuser=madalice 2026-08-29 23:01:32 L1=1.39 run=1 blk=0 avail=8706 (63.2%) swpO=0 conn=4 myRSS=2615 php=3 phpRSS=192 ap=18 tcp=30 topuser=madalice 2026-08-29 23:01:41 L1=1.49 run=2 blk=0 avail=8551 (62.1%) swpO=0 conn=2 myRSS=2615 php=4 phpRSS=341 ap=19 tcp=33 topuser=madalice 2026-08-29 23:01:42 L1=1.49 run=2 blk=0 avail=8541 (62.0%) swpO=0 conn=2 myRSS=2615 php=4 phpRSS=353 ap=19 tcp=36 topuser=madalice 2026-08-29 23:01:51 L1=1.71 run=2 blk=0 avail=7791 (56.5%) swpO=0 conn=2 myRSS=2615 php=10 phpRSS=1170 ap=33 tcp=37 topuser=madalice 2026-08-29 23:01:52 L1=1.71 run=13 blk=0 avail=7058 (51.2%) swpO=0 conn=9 myRSS=2615 php=13 phpRSS=1857 ap=32 tcp=45 topuser=madalice