INCIDENT 2026-09-01 04:13:57 trigger: php_rss=1570MB ================================================================== load: 0.67 0.83 0.97 4/686 112270 ncpu:10 mem_avail: 6533MB/13777MB (47.4%) swap_used:2267MB swapout:491/s blocked(D):0 running:4 iowait:0.1% steal:0.0% php: 12 procs / 1570MB heaviest user: madalice (10 procs, 1525MB) apache: 55 procs tcp_estab: 41 --- RSS by user (all processes) --- USER PROCS RSS_MB mysql 1 3319 madalice 10 1541 root 248 1118 nobody 54 964 netdata 7 219 infinium 5 129 wp-toolkit 3 127 cpanellogin 1 68 mailman 20 67 dovenull 6 46 lowrange 2 29 evolution 7 29 kurri 2 26 mailnull 2 22 zabbix 1 20 dovecot 3 18 aaa 1 15 named 1 11 polkitd 1 4 nscd 1 4 --- php workers by user --- USER PROCS RSS_MB madalice 10 1550 root 1 34 evolution 1 10 --- vm tuning / memory pressure --- vm.swappiness 30 vm.vfs_cache_pressure 100 vm.min_free_kbytes 67584 vm.overcommit_memory 0 vm.overcommit_ratio 50 Cached: 5095 MB Active(file): 4005 MB Inactive(file): 1390 MB Dirty: 0 MB Writeback: 0 MB SReclaimable: 650 MB CommitLimit: 10984 MB Committed_AS: 8843 MB --- backup / dump / scan processes (page-cache evictors) --- PID USER ELAPSED RSS_KB COMMAND 60845 root 13:57 1360 /bin/sh -c bash -c 'sleep $((RANDOM % 1800))' ;imunify-antivirus imunify-patch subscriptions refresh > /dev/null 2>&1 3926885 root 01:51:28 6768 /usr/bin/imunify-agent-proxy --- top 60 by RSS --- PID PPID USER RSS %CPU STAT ELAPSED COMMAND 506671 1 mysql 3398816 9.0 Ssl 4-01:54:48 /usr/sbin/mariadbd 112237 3929444 madalice 244124 42.0 R 00:03 php-fpm: pool madalice_com_au 508924 1 netdata 198648 1.9 Ssl 4-01:54:17 /usr/sbin/netdata -P /run/netdata/netdata.pid -D 111271 3929444 madalice 186960 12.4 R 01:38 php-fpm: pool madalice_com_au 3963221 3930335 root 185920 0.0 S 01:43:40 spamd child 3930335 1 root 175620 0.0 Ss 01:50:46 /usr/local/cpanel/3rdparty/perl/542/bin/perl -T -w /usr/local/cpanel/3rdparty/bin/spamd --allowed-ips=127.0.0.1,::1 --max-children=5 -- 112224 3929444 madalice 170288 53.2 S 00:04 php-fpm: pool madalice_com_au 3963222 3930335 root 168156 0.0 S 01:43:40 spamd child 112171 3929444 madalice 164244 46.8 S 00:11 php-fpm: pool madalice_com_au 112225 3929444 madalice 159860 66.2 S 00:04 php-fpm: pool madalice_com_au 112228 3929444 madalice 140328 37.7 S 00:04 php-fpm: pool madalice_com_au 112226 3929444 madalice 140292 38.2 S 00:04 php-fpm: pool madalice_com_au 112229 3929444 madalice 137220 36.0 S 00:04 php-fpm: pool madalice_com_au 112231 3929444 madalice 137188 48.3 S 00:03 php-fpm: pool madalice_com_au 112230 3929444 madalice 137176 36.0 S 00:04 php-fpm: pool madalice_com_au 509266 1 root 95744 0.0 Ss 4-01:54:14 /usr/lib/systemd/systemd-journald 509640 508992 root 75556 0.0 Sl 4-01:54:12 /usr/libexec/netdata/plugins.d/ebpf.plugin 1 3929420 1 wp-toolkit 73752 0.0 Ss 01:51:19 /usr/bin/sw-engine /usr/local/cpanel/3rdparty/wp-toolkit/bin/run-script background-tasks-executor.php 41166 1353308 cpanellogin 69880 0.0 S 23:28 /usr/local/cpanel/base/show_template.stor default_login_theme cpanel server_locale en docroot /usr/local/cpanel/base cpanel_locale 509236 509098 root 59960 0.0 Sl 4-01:54:14 /opt/saltstack/salt/bin/python3.10 /usr/bin/salt-minion MultiMinionProcessManager MinionProcessManager 112219 1353308 infinium 55500 1.8 S 00:07 webmaild - serving 1.41.252.211 3929564 1 wp-toolkit 47408 0.0 Ss 01:51:18 /usr/bin/sw-engine /usr/local/cpanel/3rdparty/wp-toolkit/bin/run-script scheduled-tasks-executor.php 1353308 1 root 42008 0.1 Ss 41-01:32:40 cpsrvd (SSL) - waiting for connections 3929532 1 root 35412 0.0 Ss 01:51:18 lfd - sleeping 3929444 1 root 35160 0.0 Ss 01:51:19 php-fpm: master process (/opt/cpanel/ea-php83/root/etc/php-fpm.conf) 3929469 1 root 35080 0.0 Ss 01:51:19 sw-engine-fpm: master process (/etc/sw-engine/sw-engine-fpm.conf) 41169 3929397 infinium 33768 0.0 S 23:28 php-fpm: pool user_infinium 1469 1 root 22196 0.1 Ss 63-18:04:09 cpdavd - accepting connections on: 2079, 2080, 2091, 2077, 2078 94348 506832 nobody 21468 0.0 S 05:39 /usr/sbin/httpd -k start 93428 506832 nobody 20924 0.0 S 06:46 /usr/sbin/httpd -k start 111546 506832 nobody 20868 0.1 S 01:16 /usr/sbin/httpd -k start 111203 506832 nobody 20820 0.0 S 01:41 /usr/sbin/httpd -k start 483898 1 zabbix 20736 0.2 Ssl 4-01:55:31 /usr/sbin/zabbix_agent2 -c /etc/zabbix/zabbix_agent2.conf 111279 506832 nobody 20504 0.0 S 01:38 /usr/sbin/httpd -k start 111152 506832 nobody 20424 0.0 S 01:49 /usr/sbin/httpd -k start 111542 506832 nobody 20412 0.0 S 01:16 /usr/sbin/httpd -k start 110405 506832 nobody 20272 0.1 S 02:38 /usr/sbin/httpd -k start 111621 506832 nobody 20252 0.1 S 01:13 /usr/sbin/httpd -k start 95008 506832 nobody 20220 0.0 S 04:24 /usr/sbin/httpd -k start 111171 506832 nobody 20112 0.0 S 01:47 /usr/sbin/httpd -k start 111593 506832 nobody 20104 0.0 S 01:14 /usr/sbin/httpd -k start 111159 506832 nobody 19972 0.0 S 01:49 /usr/sbin/httpd -k start 111263 506832 nobody 19936 0.0 S 01:39 /usr/sbin/httpd -k start 111274 506832 nobody 19784 0.0 S 01:38 /usr/sbin/httpd -k start 111588 506832 nobody 19752 0.0 S 01:14 /usr/sbin/httpd -k start 95093 506832 nobody 19708 0.0 S 04:13 /usr/sbin/httpd -k start 1463 1 root 19660 0.0 Ss 63-18:04:09 tailwatchd 111277 506832 nobody 19572 0.0 S 01:38 /usr/sbin/httpd -k start 111155 506832 nobody 19548 0.0 S 01:49 /usr/sbin/httpd -k start 111606 506832 nobody 19392 0.0 S 01:14 /usr/sbin/httpd -k start 111575 506832 nobody 19372 0.0 S 01:15 /usr/sbin/httpd -k start 111620 506832 nobody 19372 0.0 S 01:13 /usr/sbin/httpd -k start 111574 506832 nobody 19360 0.0 S 01:15 /usr/sbin/httpd -k start 111564 506832 nobody 19340 0.0 S 01:15 /usr/sbin/httpd -k start 111584 506832 nobody 19332 0.0 S 01:14 /usr/sbin/httpd -k start 111599 506832 nobody 19312 0.0 S 01:14 /usr/sbin/httpd -k start 111609 506832 nobody 19296 0.0 S 01:14 /usr/sbin/httpd -k start 111614 506832 nobody 19280 0.0 S 01:13 /usr/sbin/httpd -k start 111592 506832 nobody 19236 0.0 S 01:14 /usr/sbin/httpd -k start 111597 506832 nobody 19216 0.0 S 01:14 /usr/sbin/httpd -k start --- processes in D state (uninterruptible IO) --- --- mariadb processlist (non-sleeping) --- +----------+------+------+---------+------+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ID | USER | DB | COMMAND | TIME | STATE | q | +----------+------+------+---------+------+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 10119417 | root | NULL | Query | 0 | Filling schema table | SELECT ID,USER,DB,COMMAND,TIME,STATE,LEFT(INFO,300) q FROM information_schema.PROCESSLIST WHERE COMMAND<>'Sleep' ORDER BY TIME DESC LIMIT 40 | +----------+------+------+---------+------+----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ --- lock waits --- --- vhost traffic RIGHT NOW (last 3 min, deduped by inode) --- VHOST LOG REQ/3min evolution/app.evolutionerp.com.au-ssl_log 140 tubesupplies/tubesupplies.com.au-ssl_log 121 olytis/tis.com.au-ssl_log 111 otter2/otterfencing.net.au-ssl_log 71 cosgrovecreative/cosgrovecreative.com.au-ssl_log 35 evolution/evolutionerp.com.au-ssl_log 30 evolution/portal.evolutionerp.com.au-ssl_log 28 madalice/madalice.com.au-ssl_log 22 infinium/infinium.net.au-ssl_log 13 aaa/aaadownunder.com.au-ssl_log 11 (total vhosts taking traffic: 10) --- top client IPs across all vhosts (last 3 min) --- 140 40.115.94.245 119 20.104.49.130 111 68.155.159.216 71 20.220.227.7 35 20.151.10.161 30 158.23.184.117 28 203.54.180.238 9 162.19.8.222 5 32.195.67.60 4 118.127.40.55 3 51.178.99.32 2 69.49.241.71 2 50.87.144.101 2 209.141.52.196 2 2001:1aef:19b:1bd2:397f:ffd9:a8b2:365c --- top request paths across all vhosts (last 3 min) --- 50 GET /api/getItem/LWFBW2001KG 28 GET /heartbeat.php 9 POST /wp-login.php 8 GET /wp-login.php 7 GET / 3 POST /wp-cron.php 3 GET /new.php 3 GET /av.php 2 GET /xmlrpc.php 2 GET /wp-trackback.php 2 GET /wp-includes/block-bindings/ 2 GET /wp-content/themes/index.php 2 GET /wp-blog.php 2 GET /wp-admin/maint/index.php 2 GET /wp-admin/images/admin.php 2 GET /wp-admin/css/colors/blue/ 2 GET /wp-admin/css/ 2 GET /t.php 2 GET /tool.php 2 GET /m.php --- busiest 3 vhosts: top paths + last 10 requests --- == evolution/app.evolutionerp.com.au-ssl_log == -- top paths -- 50 GET /api/getItem/LWFBW2001KG 2 GET /api/getItem/IPC2525 2 GET /api/getItem/H8025D 2 GET /api/getItem/GEB12-DL 1 GET /api/getItem/SPHDG15040 1 GET /api/getItem/SGC-100 1 GET /api/getItem/S6 1 GET /api/getItem/S5 1 GET /api/getItem/S4 1 GET /api/getItem/S3 1 GET /api/getItem/S2 1 GET /api/getItem/S1 -- last 10 requests -- 40.115.94.245 - - [01/Sep/2026:04:13:11 +1000] "GET /api/getItem/RGW117T HTTP/1.1" 404 109 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:12 +1000] "GET /api/getItem/RGW80 HTTP/1.1" 200 1604 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:12 +1000] "GET /api/getItem/GP150x200x8 HTTP/1.1" 200 1522 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:12 +1000] "GET /api/getItem/K125MSGB HTTP/1.1" 200 1451 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:13 +1000] "GET /api/getItem/253-40SP HTTP/1.1" 200 1312 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:13 +1000] "GET /api/getItem/300558 HTTP/1.1" 200 1327 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:14 +1000] "GET /api/getItem/SGC-100 HTTP/1.1" 200 1354 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:14 +1000] "GET /api/getItem/IPC2020 HTTP/1.1" 200 1352 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:14 +1000] "GET /api/getItem/IPC2525 HTTP/1.1" 200 1621 "-" "RestSharp/106.13.0.0" 40.115.94.245 - - [01/Sep/2026:04:13:15 +1000] "GET /api/getItem/IPC25NB HTTP/1.1" 200 1468 "-" "RestSharp/106.13.0.0" == tubesupplies/tubesupplies.com.au-ssl_log == -- top paths -- 1 POST /wp-login.php 1 POST /wp-cron.php 1 GET /zoo2.php 1 GET /yawa.php 1 GET /xwx1.php 1 GET /xwpg.php 1 GET /xmini4.php 1 GET /xa.php 1 GET /x1da.php 1 GET /wsd.php 1 GET /ws85.php 1 GET /ws78.php -- last 10 requests -- 20.104.49.130 - - [01/Sep/2026:04:12:51 +1000] "GET /ws78.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:51 +1000] "GET /ws58.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:52 +1000] "GET /ws45.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:53 +1000] "GET /ortasekerli1.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:53 +1000] "GET /t.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:54 +1000] "GET /wp-good.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:54 +1000] "GET /as.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:55 +1000] "GET /mh.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:55 +1000] "GET /press.php HTTP/1.1" 404 5750 "-" "-" 20.104.49.130 - - [01/Sep/2026:04:12:56 +1000] "GET /edit.php HTTP/1.1" 404 5750 "-" "-" == olytis/tis.com.au-ssl_log == -- top paths -- 1 GET /z.php 1 GET /zoom1.php 1 GET /xx.php 1 GET /xmlrpc.php 1 GET /xda.php 1 GET /ww2.php 1 GET /ws.php 1 GET /ws80.php 1 GET /ws77.php 1 GET /ws37.php 1 GET /wp/wp-admin/includes/ 1 GET /wp-trackback.php -- last 10 requests -- 68.155.159.216 - - [01/Sep/2026:04:11:45 +1000] "GET /1index.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:46 +1000] "GET /autoload_classmap/function.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:46 +1000] "GET /dvve.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:46 +1000] "GET /wp-includes/PHPMailer/ HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:47 +1000] "GET /ws77.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:47 +1000] "GET /.well-known/acme-challenge/index.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:48 +1000] "GET /admin/function.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:48 +1000] "GET /wp-admin/css/index.php HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:48 +1000] "GET /wp-includes/css/ HTTP/1.1" 401 36 "-" "-" 68.155.159.216 - - [01/Sep/2026:04:11:49 +1000] "GET /xx.php HTTP/1.1" 401 36 "-" "-" --- last 30 samples (run-up to this incident) --- 2026-09-01 04:11:28 L1=0.68 run=1 blk=0 avail=7893 (57.3%) swpO=0 conn=1 myRSS=3336 php=3 phpRSS=201 ap=20 tcp=33 topuser=madalice 2026-09-01 04:11:36 L1=0.73 run=1 blk=0 avail=7891 (57.3%) swpO=0 conn=2 myRSS=3336 php=4 phpRSS=240 ap=20 tcp=34 topuser=madalice 2026-09-01 04:11:38 L1=0.73 run=1 blk=0 avail=7902 (57.4%) swpO=2 conn=1 myRSS=3336 php=3 phpRSS=201 ap=19 tcp=32 topuser=madalice 2026-09-01 04:11:46 L1=0.77 run=1 blk=0 avail=7891 (57.3%) swpO=2 conn=2 myRSS=3336 php=4 phpRSS=240 ap=19 tcp=33 topuser=madalice 2026-09-01 04:11:48 L1=0.77 run=1 blk=0 avail=8036 (58.3%) swpO=0 conn=1 myRSS=3336 php=2 phpRSS=45 ap=19 tcp=33 topuser=root 2026-09-01 04:11:56 L1=0.65 run=1 blk=0 avail=7926 (57.5%) swpO=0 conn=2 myRSS=3336 php=4 phpRSS=218 ap=18 tcp=33 topuser=madalice 2026-09-01 04:11:58 L1=0.65 run=4 blk=0 avail=7852 (57.0%) swpO=0 conn=1 myRSS=3336 php=3 phpRSS=268 ap=18 tcp=34 topuser=madalice 2026-09-01 04:12:06 L1=0.62 run=2 blk=0 avail=7831 (56.8%) swpO=0 conn=2 myRSS=3336 php=4 phpRSS=240 ap=22 tcp=41 topuser=madalice 2026-09-01 04:12:08 L1=0.62 run=1 blk=0 avail=7869 (57.1%) swpO=0 conn=1 myRSS=3336 php=3 phpRSS=201 ap=26 tcp=51 topuser=madalice 2026-09-01 04:12:16 L1=0.60 run=1 blk=0 avail=7791 (56.6%) swpO=0 conn=2 myRSS=3336 php=4 phpRSS=241 ap=35 tcp=54 topuser=madalice 2026-09-01 04:12:18 L1=0.60 run=2 blk=0 avail=7737 (56.2%) swpO=0 conn=1 myRSS=3336 php=3 phpRSS=258 ap=37 tcp=58 topuser=madalice 2026-09-01 04:12:26 L1=0.80 run=2 blk=0 avail=7520 (54.6%) swpO=0 conn=2 myRSS=3336 php=6 phpRSS=496 ap=43 tcp=29 topuser=madalice 2026-09-01 04:12:28 L1=0.80 run=1 blk=0 avail=7537 (54.7%) swpO=0 conn=1 myRSS=3336 php=5 phpRSS=457 ap=42 tcp=30 topuser=madalice 2026-09-01 04:12:36 L1=0.91 run=3 blk=0 avail=7389 (53.6%) swpO=0 conn=2 myRSS=3336 php=8 phpRSS=762 ap=33 tcp=35 topuser=madalice 2026-09-01 04:12:38 L1=0.91 run=2 blk=0 avail=7609 (55.2%) swpO=0 conn=1 myRSS=3336 php=5 phpRSS=458 ap=32 tcp=65 topuser=madalice 2026-09-01 04:12:46 L1=0.93 run=1 blk=0 avail=6825 (49.5%) swpO=0 conn=2 myRSS=3336 php=6 phpRSS=554 ap=125 tcp=125 topuser=madalice 2026-09-01 04:12:48 L1=0.93 run=1 blk=0 avail=6767 (49.1%) swpO=0 conn=1 myRSS=3336 php=5 phpRSS=591 ap=124 tcp=108 topuser=madalice 2026-09-01 04:12:57 L1=0.94 run=3 blk=0 avail=6929 (50.3%) swpO=2 conn=2 myRSS=3336 php=6 phpRSS=560 ap=115 tcp=32 topuser=madalice 2026-09-01 04:12:58 L1=0.94 run=3 blk=0 avail=6941 (50.4%) swpO=2 conn=1 myRSS=3336 php=5 phpRSS=519 ap=114 tcp=31 topuser=madalice 2026-09-01 04:13:07 L1=0.80 run=3 blk=0 avail=6920 (50.2%) swpO=0 conn=2 myRSS=3336 php=7 phpRSS=687 ap=105 tcp=36 topuser=madalice 2026-09-01 04:13:08 L1=0.80 run=3 blk=0 avail=6783 (49.2%) swpO=0 conn=1 myRSS=3336 php=8 phpRSS=850 ap=104 tcp=38 topuser=madalice 2026-09-01 04:13:17 L1=0.68 run=2 blk=0 avail=7231 (52.5%) swpO=0 conn=2 myRSS=3336 php=4 phpRSS=357 ap=95 tcp=28 topuser=madalice 2026-09-01 04:13:18 L1=0.68 run=3 blk=0 avail=7238 (52.5%) swpO=0 conn=1 myRSS=3336 php=5 phpRSS=380 ap=94 tcp=29 topuser=madalice 2026-09-01 04:13:27 L1=0.57 run=1 blk=0 avail=7449 (54.1%) swpO=0 conn=2 myRSS=3336 php=3 phpRSS=202 ap=85 tcp=30 topuser=madalice 2026-09-01 04:13:28 L1=0.57 run=1 blk=0 avail=7457 (54.1%) swpO=0 conn=1 myRSS=3336 php=3 phpRSS=202 ap=84 tcp=28 topuser=madalice 2026-09-01 04:13:37 L1=0.48 run=1 blk=0 avail=7520 (54.6%) swpO=0 conn=2 myRSS=3336 php=3 phpRSS=201 ap=75 tcp=27 topuser=madalice 2026-09-01 04:13:38 L1=0.48 run=2 blk=0 avail=7528 (54.6%) swpO=0 conn=1 myRSS=3336 php=3 phpRSS=201 ap=74 tcp=26 topuser=madalice 2026-09-01 04:13:47 L1=0.41 run=2 blk=0 avail=7421 (53.9%) swpO=0 conn=2 myRSS=3336 php=4 phpRSS=395 ap=65 tcp=30 topuser=madalice 2026-09-01 04:13:48 L1=0.41 run=2 blk=0 avail=7487 (54.3%) swpO=0 conn=1 myRSS=3336 php=4 phpRSS=340 ap=64 tcp=32 topuser=madalice 2026-09-01 04:13:57 L1=0.67 run=4 blk=0 avail=6533 (47.4%) swpO=491 conn=4 myRSS=3319 php=12 phpRSS=1570 ap=55 tcp=41 topuser=madalice