Pulling a Thread
Introduction
Some web applications require a lot of attention. If there are a lot of concurrent users, complex SQL queries, and the frequent need to create large batches of records in ACID compliant transactions, efficient resource management is crucial. The web application that has characteristics like those just mentioned, if it is going to bring deliver responsive and reliable services to its user community, must be managed with high intention. Among the things that must be managed are threads. (Self-managing web applications for large user bases is a nice idea. But at the time of this writing, my peers in the software development industry are not declaring victories in this area.) In Java-based web applications, threads play a pivotal role in handling concurrent requests, performing background tasks, and ensuring that the application remains responsive under high loads.
When thread management is either ignored or handled improperly, issues ensue. What kind of issues?
- Thread / Memory Leaks — Thread leaks occur when threads are not properly terminated or returned to the thread pool. Does it matter? Yes! If the threads are not being properly terminated or returned to the thread pool, and they are being created as the application runs, threads are leaking out. This will lead to a depletion of available threads, causing the application to hang or crash. (Eventually there are no threads are left to handle incoming requests.) How does this happen? This could be due to spend thrifty creation of threads in our code. Each thread uses a chunk of memory. Excessive creation of threads without proper management will exhaust the JVM heap space and lead to
OutOfMemoryErrorexceptions. (Practically speaking, not too long after you see these exceptions in log files, your application is going to crash or become unstable.) - Thread Starvation and Increased Latency — This happens when lower-priority threads are perpetually prevented from executing because higher-priority threads monopolize the CPU. These low prior threads are not altogether unimportant. But when they are forced to wait on resource hogging threads, their unimportant work eventually becomes urgent. Making them wait invariably leads to unresponsive services and missed deadlines for critical tasks. Users may see the symptoms in slow application response times.
- Unexpected Behavior — Poorly managed threads can cause the application to behave unpredictably under different load conditions. This makes it difficult to ensure consistent performance and reliability, especially in a production environment. Inconsistent State: If a
ThreadLocalvariable is not properly initialized for each thread, different threads may see different states, leading to inconsistent behavior in the application. (I experienced this woe while working on an application written for a government agency. The mismanagement ofThreadLocalvariables in Liferay kept manifesting itself in application users getting sessions that actually belonged to someone else. The menus of the application were determined by the security role associated with the user’s account. The security role was stored in the user’s session. Without warning a user would suddenly have the menu for another user with more or less privileges than themselves. No, I did not write that code. But I still had to fix it.) Inheritance Issues: UsingInheritableThreadLocalcan lead to unexpected behavior if child threads inadvertently inherit values from parent threads when such inheritance is not intended.
This post delves into the essentials of thread management in Java-based web applications that run on Apache Tomcat. In the discussion that follows we will give special attention to (1) enumerating threads, (2) checking for threads that require cleanup or termination, and (3) special issues associated with Threadlocal variables.
Enumerating the Threads in Your Web Application
Getting a list of all threads in a web application is very simple. I will show you two ways to do it. I will demonstrate getting the list of threads in a Java-based web application running in a standard servlet container. (This solution will run in Jetty, Apache Tomcat, Apache JBoss (WildFly), GlassFish, IBM WebSphere, or Oracle WebLogic.)
Step 1: Create a Thread-Enumerating Servlet
By creating the enumeration solution as a servlet you will make it easier to get the results. (Yes, you could write it to catalina.out. However, you may have to ask an administrator for the file. And the contents may contain a lot more than you need, want, or have permission to see.)
package org.roderickbarnes.servlet;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Map;
@WebServlet("/enumerateThreads")
public class ThreadEnumerationServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws ServletException, IOException {
// Tell the calling client what type of content will be returned (This is the mime type).
httpServletResponse.setContentType("text/plain");
PrintWriter printWriter = httpServletResponse.getWriter();
// Get all stack traces
Map<Thread, StackTraceElement[]> mapOfThreadToStackTraces = Thread.getAllStackTraces();
for (Map.Entry<Thread, StackTraceElement[]> mapEntry : mapOfThreadToStackTraces .entrySet()) {
Thread thread = mapEntry.getKey();
StackTraceElement[] arrayOfStackTraceElement = mapEntry.getValue();
printWriter.println("Thread Name: " + thread.getName());
printWriter.println("Thread ID: " + thread.getId());
printWriter.println("Thread State: " + thread.getState());
printWriter.println("Thread Priority: " + thread.getPriority());
printWriter.println("Is Daemon: " + thread.isDaemon());
printWriter.println("Stack Trace:");
for (StackTraceElement stackTraceElement : arrayOfStackTraceElement) {
out.println("\t" + stackTraceElement);
}
printWriter.println("--------------------------------------------------");
}
printWriter.close();
}
}
Listing 1 – Servlet for listing threads in a web application.
The doGet method in the servlet fetches all live threads and their stack traces using Thread.getAllStackTraces(). It then prints the details of each thread to the servlet response. This step is an easy one. You can create your own glorious code. But move your thread management goals forward faster by copying the code above and pasting it into your IDE of choice.
Step 2: Registering and Deploying the Thread-Enumerating Servlet
Unless you are using annotations, update theweb.xmlof your web application or servlet container. An example is provided in Listing 2.
<servlet>
<servlet-name>ThreadEnumerationServlet</servlet-name>
<servlet-class>com.example.ThreadEnumerationServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>ThreadEnumerationServlet</servlet-name>
<url-pattern>/enumerateThreads</url-pattern>
</servlet-mapping>
Listing 2 – Servlet deployment XML required for the web.xml file.
Deploy the web Application by doing the following:
- Package your web application (WAR file) and deploy it to your servlet container server.
- Ensure the servlet is correctly placed in the appropriate package (
org.roderickbarnes.servletin this case).
Step 3: Call the Servlet from Your Browser
Once deployed, you can access the servlet by navigating to http://your-server:port/your-app/enumerateThreads in your web browser. This will output the details of all active threads in your Tomcat application.
Pulling the Hanging Threads
Pulling Unstopped Threads
If you are creating threads manually, consider using ExecutorService from the java.util.concurrent package. This provides a better abstraction for managing threads, and you can easily shut down the executor service during application shutdown. Here are some of the benefits of using ExecutorService in your web application:
- Thread Management –
ExecutorServiceprovides a clean and efficient way to manage a pool of threads, avoiding the overhead and risks associated with manually creating and managing threads. That is what this whole article is about. Using this service is a major move toward better thread management. - Graceful Shutdown – Proper handling of thread shutdown ensures that tasks are completed or terminated correctly, preventing resource leaks.
ExecutorServicefacilitates graceful shutdown of your threads. - Concurrency Control – For those of us with a tendency toward micromanagement, the
ExecutorServiceis a friend. You can control the number of concurrent tasks by configuring the thread pool size.
How would we use this service. Here below I provide you with the steps required to employ ExecutorService in the management of threads for your web application.
Step 1: Create a Thread Management Servlet that Uses ExecutorService
package org.roderickbarnes.servlet;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.annotation.WebListener;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
@WebServlet("/executeTask")
public class TaskExecutorServlet extends HttpServlet {
private ExecutorService executorService;
private void setExecutorService(ExecutorService executorServiceNew) {
this.executorService = executorServiceNew;
}
private ExecutorService getExecutorService() {
return this.executorService;
}
@Override
public void init() {
// Initialize the ExecutorService with a fixed thread pool
executorServiceNew = Executors.newFixedThreadPool(17);
this.setExecutorService(executorServiceNew);
}
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse httpServletResponse) throws IOException {
// Submit a task to the executor service
this.getExecutorService().submit(() -> {
try {
// Simulate a task
System.out.println("Task executed by thread: " + Thread.currentThread().getName());
TimeUnit.SECONDS.sleep(2); // Simulate work with a sleep
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
e.printStackTrace();
}
});
httpServletResponse.getWriter().write("Task submitted");
}
@Override
public void destroy() {
// Shutdown the ExecutorService when the servlet is destroyed
this.getExecutorService().shutdown();
try {
if (!this.getExecutorService().awaitTermination(60, TimeUnit.SECONDS)) {
executorService.shutdownNow();
if (!this.getExecutorService().awaitTermination(60, TimeUnit.SECONDS)) {
System.err.println("ExecutorService did not terminate");
}
}
} catch (InterruptedException ie) {
this.getExecutorService().shutdownNow();
Thread.currentThread().interrupt();
}
}
}
Listing 3 – Demonstrating the use of ExecutorService to manage threads
Step 2: Add a ServletContextListener
To ensure proper cleanup and initialization, it’s a good idea to use a ServletContextListener that handles the lifecycle of the ExecutorService. Note: This is not required but is recommended.
package org.roderickbarnes.servlet;
@WebListener
public class ExecutorServiceListener implements ServletContextListener {
private ExecutorService executorService;
private void setExecutorService(ExecutorService executorServiceNew) {
this.executorService = executorServiceNew;
}
private ExecutorService getExecutorService() {
return this.executorService;
}
@Override
public void contextInitialized(ServletContextEvent sce) {
// Initialize the ExecutorService when the web application starts
executorServiceNew = Executors.newFixedThreadPool(10);
this.setExecutorService(executorServiceNew);
sce.getServletContext().setAttribute("executorService", this.getExecutorService());
}
@Override
public void contextDestroyed(ServletContextEvent sce) {
// Shutdown the ExecutorService when the web application stops
ExecutorService executorServiceTemp = (ExecutorService) sce.getServletContext().getAttribute("executorService");
if (executorServiceTemp != null) {
executorServiceTemp.shutdown();
try {
if (!executorServiceTemp.awaitTermination(60, TimeUnit.SECONDS)) {
executorServiceTemp.shutdownNow();
if (!executorService.awaitTermination(60, TimeUnit.SECONDS)) {
System.err.println("ExecutorService did not terminate");
}
}
} catch (InterruptedException ie) {
executorServiceTemp.shutdownNow();
Thread.currentThread().interrupt();
}
}
}
}
Listing 4 – Demonstrating the use of ExecutorService to manage threads
Why is this a good practice? The ServletContextListener provides a centralized place (1) to initialize the ExecutorService when the web application starts and (2) to properly shut it down when the application stops. This ensures that the ExecutorService is consistently managed throughout the application’s lifecycle.
Step 3: Update your web.xml if you are not using annotations.
<web-app>
<servlet>
<servlet-name>TaskExecutorServlet</servlet-name>
<servlet-class>org.roderickbarnes.TaskExecutorServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>TaskExecutorServlet</servlet-name>
<url-pattern>/executeTask</url-pattern>
</servlet-mapping>
<listener>
<listener-class>org.roderickbarnes.ExecutorServiceListener</listener-class>
</listener>
</web-app>
Listing 5 – Demonstrating the use of ExecutorService to manage threads
But how does all this work. Here, let me explain. There are four parts worth noting.
The Servlet Initialization Part – When the servlet is initialized (init method), an ExecutorService is created with a fixed thread pool of 17 threads. This means that up to 17 tasks can be executed concurrently.
The Handling Requests Part – Each time a request is made to the servlet’s /executeTask endpoint, a new task is submitted to the ExecutorService. The task simulates some work by sleeping for 2 seconds. In your web application you would have something that needs to be c concurrent and is germane to the user community or application’s needs.
The Servlet Destruction Part – When the servlet is destroyed (e.g., when the application is undeployed or the server is shut down), the ExecutorService is properly shut down. This ensures that all threads are terminated gracefully, avoiding potential memory leaks. Did you get that part?
The Context Listener Part (Optional) – The ExecutorServiceListener manages the lifecycle of the ExecutorService at the application level, ensuring it is available throughout the application’s lifecycle and cleaned up properly when the application is stopped.
Pulling Threads Hanging Due to ThreadLocal Variables Not Being Cleaned Up
Wherever you use ThreadLocal variables, make sure to call ThreadLocal.remove() when the variable is no longer needed, especially in the cleanup code of your application (e.g., in ServletContextListener or a similar lifecycle hook).
ThreadLocal<MyObject> threadLocalOfMyObject = new ThreadLocal<MyObject>();
try {
// Setup the ThreadLocal object.
threadLocalOfMyObject.set(new MyObject());
// Use the ThreadLocal object.
} finally {
threadLocalOfMyObject .remove();
}
Listing 6 – Using a Try-Catch Block to Ensure Removal of ThreadLocal Variables
Conclusion
This article has introduced you to some ways you can remediate thread issues in your web application. The ideas suggested here are not panaceas by any means. They are a few of many means for getting threads under control and bringing high intentionality to the management of resources. Among the things that should be considered are using the ExecutorService and ensuring that ThreadLocal variables are properly cleaned up. Before getting started with any plan to change how threads are being handled, use a tool like the one provided in Listing 1 to get your thread inventory.
In His grip by His grace,
Roderick L. Barnes, Sr.