Tomcat server hangs troubleshooting

Find reasons why tomcat server hangs inexplicably on Linux

The company’s service was inexplicably hung up. At first, I thought that the server was attacked by someone. It started up and hung up again. I checked it and found that the system memory was too full and automatically killed the tomcat process!

Both are common Out of memory problems. This is usually caused by a large amount of memory requested by an application at a certain time, resulting in insufficient system memory. This usually triggers the Out of Memory (OOM) killer in the Linux kernel. The OOM killer kills a process to free up memory for the system. , So that the system does not crash immediately.

Looking at the /var/log/messages file, it is found that the kernel killed by the system is out of memory: Out of memory: Kill process 15983 (java) score 149 or sacrifice child linux system memory is full and automatically kills unprotected threads. Java needs to be removed. Process joins into protected process

View the /var/log/messages file, you can see the following information, found that the java process is killed by the operating system. The oom killer is a protection process of the Linux system. It is triggered when the remaining memory space of the Linux system is insufficient to meet the normal operation of the system. When oomkiller is executed, it will find the pid with the highest score value of all threads in the system and then kill it.

May 14 03:14:02 mqnameserver2 kernel: [7628] 0 7628 1483624 538 0 0 0 java

May 14 03:14:02 mqnameserver2 kernel: Out of memory: Kill process 7628 (java) score 109 or sacrifice child

May 14 03: 14:02 mqnameserver2 kernel: Killed process 7628, UID 0, (java) total-vm:5934496kB, anon-rss:0kB, file-rss:2152kB

Solution:

The solution is to increase the memory configuration for tomcat, or add tomcat to the protected process and not let the system kill it, but the root cause is that the server memory itself is too small, so it is meaningless to change these configurations. The only thing to do is to get a monitoring mechanism, memory overflow automatically restarts, or send an email reminder

Thanks for reading this post.

Pankaj K.