Re: [PLUG] >32K concurrent processes

On Fri, Apr 15, 2016 at 5:44 PM, Rohit Mehta <ro@paper-mill.com> wrote:

I've never tried to do this, but do you have to increase file-max as well? I'm not sure if there are other tweaks that need to be made.

On Fri, Apr 15, 2016 at 5:40 PM, K.S. Bhaskar <bhaskar@bhaskars.com> wrote:
I am trying to crank up more than 32K concurrent processes (the processes themselves hang and do a teeny bit of activity every 10 seconds). But the OS (64-bit Debian 8 - Jessie) stubbornly refuses to crank up beyond 32K-ish processes. pid_max is set to a very large number (1M), so that's not it. Any suggestions on what limits to look for appreciated. Thank you very much.

atop output below (x1234567 substituted for actual user id).

-- Bhaskar

ATOP - shrug 2016/04/15 17:27:00 ------ 1m54s elapsed
PRC | sys 6m41s | user 10m06s | #proc 32191 | #trun 25769 | #tslpi 6489 | #tslpu 1 | #zombie 0 | #exit 4 |
CPU | sys 349% | user 392% | irq 0% | idle 58% | wait 1% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 45% | user 50% | irq 0% | idle 5% | cpu006 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 42% | user 52% | irq 0% | idle 6% | cpu002 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 48% | user 45% | irq 0% | idle 6% | cpu005 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 43% | user 50% | irq 0% | idle 6% | cpu003 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 43% | user 50% | irq 0% | idle 6% | cpu000 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 48% | user 45% | irq 0% | idle 7% | cpu004 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 41% | user 50% | irq 0% | idle 9% | cpu001 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
cpu | sys 38% | user 49% | irq 0% | idle 12% | cpu007 w 0% | guest 0% | curf 4.00GHz | curscal 100% |
CPL | avg1 19343 | avg5 6390 | avg15 2266 | | csw 204165e3 | intr 6428113 | | numcpu 8 |
MEM | tot 31.5G | free 214.9M | cache 50.0M | dirty 0.0M | buff 5.3M | slab 1.7G | | |
SWP | tot 28.0G | free 21.9G | | | | | vmcom 60.6G | vmlim 43.8G |
PAG | scan 1833e3 | | stall 0 | | | swin 3 | | swout 0 |
LVM | vg0-extra2 | busy 13% | read 509 | write 9026 | KiB/w 4 | MBr/s 0.02 | MBw/s 0.31 | avio 1.49 ms |
LVM | md-swap | busy 0% | read 3 | write 0 | KiB/w 0 | MBr/s 0.00 | MBw/s 0.00 | avio 100.0 ms
LVM | vg0-library | busy 0% | read 5 | write 0 | KiB/w 0 | MBr/s 0.00 | MBw/s 0.00 | avio 0.00 ms |
MDD | md127 | busy 0% | read 3 | write 0 | KiB/w 0 | MBr/s 0.00 | MBw/s 0.00 | avio 0.00 ms |
DSK | sda | busy 13% | read 271 | write 1807 | KiB/w 10 | MBr/s 0.01 | MBw/s 0.16 | avio 6.78 ms |
DSK | sdb | busy 0% | read 396 | write 2004 | KiB/w 9 | MBr/s 0.05 | MBw/s 0.17 | avio 0.10 ms |
NET | transport | tcpi 28 | tcpo 24 | udpi 12 | udpo 12 | tcpao 0 | tcppo 0 | tcprs 0 |
NET | network | ipi 45 | ipo 39 | ipfrw 0 | deliv 45 | | icmpi 3 | icmpo 0 |
NET | eth0 0% | pcki 55 | pcko 49 | si 0 Kbps | so 0 Kbps | erri 0 | erro 0 | drpo 0 |
root@shrug:~#
PID RUID EUID THR SYSCPU USRCPU VGROW RGROW RDDSK WRDSK ST EXC S CPUNR CPU CMD 1/2671
2086 root root 1 2.23s 0.65s 0K 0K 0K 0K -- - R 4 3% atop
58 root root 1 0.12s 0.00s 0K 0K 0K 0K -- - R 6 0% kswapd0
74211 x1234567   x1234567 1 0.02s 0.04s 0K 0K 0K 0K -- - R 1 0% mumps
63966 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 5 0% mumps
64000 x1234567 x1234567   1 0.04s 0.02s 0K 0K 0K 0K -- - R 0 0% mumps
63876 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 7 0% mumps
63884 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 3 0% mumps
63796 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 4 0% mumps
64160 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 5 0% mumps
63318 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 6 0% mumps
64114 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 6 0% mumps
63396 x1234567 x1234567   1 0.02s 0.04s 0K 0K 0K 0K -- - R 3 0% mumps

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug

___________________________________________________________________________
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug