How to Scale ASP.NET Core Using ThreadPool

In this article, we will learn about how to scale our ASP.NET Core applications using Threadpool. We will specifically look at how to scale ASP.NET Core Application using the ThreadPool configuration for the number of threads.

It is always advisable to implement asynchronous programming techniques in your application i.e. there are no blocking calls in your application so that your applications can scale better with the increasing load and there is no need to change the default values in the ThreadPool. But sometimes implementation of asynchronous programming techniques might require time as the impact can be high due to code changes required and full regression testing of the application.

In this scenario where implementing asynchronous programming techniques in your code is not a quick task but you have a performance issue on production that needs to be addressed in that case you need will some quick fix to handle high load conditions till the time you are able to improve your code by implementing asynchronous programming techniques. If you think that thread starvation could be the reason behind your performance issue then you can try this technique of increasing the minimum threads in ThreadPool and try to scale ASP.NET Core application

ThreadPool Overview

ThreadPool provides a pool of threads that can be used to execute requests involving tasks, asynchronous I/O, blocking synchronous tasks, etc. The thread pool is an efficient implementation of threads in any application as that enables an application to use threads more efficiently by providing a pool of worker threads that are managed by the system and not by the application.

There is one thread pool per process so there is a pool of worker threads for each process i.e. each instance of ASP.NET Core application. By default, the number of initial threads in the pool for any application depends on the number of cores in the CPU on the machine on which the application process is running

On my machine CPU configuration shows a total of 6 Core with 12 logical processors i.e. 2 threads per core i.e. a total of 12 threads at the hardware level. So ASP.NET Core applications on my machine should start with 12 default minimum threads.

To check the default value of the number of threads in the ThreadPool in the ASP.NET Core application I created an ASP.NET Core WebAPI application and added a controller to return the min & max values for the worker’s threads from the ThreadPool and it showed that by default 12 minimum worker threads and a max of 32767 worker threads were configured for the application process at startup. Below are the sample code & results for the same.

Below is the model to hold the values for the default Thread count in ThreadPool

public class ThreadCount
{
    public int MinWorkerThreads { get; set; }
    public int MinCompletionPortThreads { get; set; }
    public int MaxWorkerThreads { get; set; }
    public int MaxCompletionPortThreads { get; set; }
}

Below is the API controller code with Action to read the default values from the ThreadPool for thread count & return the same.

[Route("api/[controller]")]
[ApiController]
public class ThreadsCountController : ControllerBase
{
    [HttpGet]
    public ThreadCount Get()
    {
        ThreadCount threadCount = new();

        ThreadPool.GetMinThreads(out int workerThreads, out int completionPortThreads);
        threadCount.MinWorkerThreads = workerThreads;
        threadCount.MinCompletionPortThreads = completionPortThreads;
            
        ThreadPool.GetMaxThreads(out workerThreads, out completionPortThreads);
        threadCount.MaxWorkerThreads = workerThreads;
        threadCount.MaxCompletionPortThreads = completionPortThreads;
            
        return threadCount;
    }
}

Below is the screenshot for the CPU configuration as available from the Task Manager on my machine & the result from the ThreadCount action which shows that 12 min worker threads & 32767 max worker threads are available by default in the thread pool at the startup of the application.

For demonstration purposes, we will create an ASP.NET Core Web API that will container with both Sync & Async Get methods and to simulate long-running tasks in those methods we will make use of the delay and sleep methods available in C#. Then we will run the application, simulate load for the application & measure performance first without change in ThreadPool configuration and then with the change in ThreadPool configuration for default threads.

From the above details of the default number of threads count it is clear that there will be no need to change the default value for max workers threads as it is already very high so we will increase the value of min worker threads based on the results of performance measurement without change in max worker threads.

Now you will have a question like as already the number of max worker threads & completion port threads is so high then why should we increase the min threads value as it can increase threads till max value based on the load on the application. To answer this question this is because the ThreadPool has a throttling mechanism wherein when there is a new request it will look for a free thread in the thread pool and if all threads are busy then it will wait for some time before it will spawn a new thread for processing of the new request that is waiting for thread allocation.

So as in our case default threads are 12 which means the number of threads in the thread pool is 12 and when the 13th request comes it will look for a free thread and if all the threads are busy then it will spawn a new thread but will wait for some 0.5 seconds before it spawns a new thread. So if there is a sudden burst of 100 requests at the same time then ThreadPool will allocate 12 threads immediately but for the rest of 88 requests, it will wait for 0.5 seconds for each request i.e. for the 88th request it might have to wait for (88 * 0.5) 44 seconds for a new thread allocation.

So for this reason we need to increase the minimum number of worker threads so that the minimum specified number are threads are spawned without any delay. So if we configure minimum threads as 100 then till 100 all the threads will be spawned without any delay but from the 101st thread, it will wait for 0.5 seconds before a new thread is spawned.

Implementation to Scale ASP.NET Core Application

For demonstration application development purposes, we will make use of Visual Studio Community 2022 Version 17.0.6 with .NET 6.0 SDK

Overall Approach for Demonstration Application

For the demonstration of how to scale ASP.NET Core Application, we will take the following approach

1. We will first create an application of type ASP.NET Core Web Api and name it as ProCodeGuide.Samples.ThreadPoolConfig
2. To the default available WeatherForecast Controller we will add 2 Get action method GetSync & GetAsync
3. In GetSync as the name suggests it will be the synchronous call i.e. thread will be blocked using Thread.Sleep and after sleep, we will return the default WeatherForecast data
4. In GetAsync as the name suggests this will be an asynchronous call i.e. thread will not be blocked instead we will introduce delay by using await Task.Delay and after delay then we will return the default WeatherForecast data
5. We will load both the action methods in the application using 100 threads in Web Surge for 60 Seconds and check the performance of both actions Sync & Async under the same load conditions for both the actions.
6. We will increase min threads in Thread Pool and then load the application again with the same load conditions and check the performance of both actions Sync & Async under the same load conditions for both

When you specify 100 threads, you’re telling WebSurge to run 100 simultaneous Sessions, each running through the Session of its individual requests repeatedly, until the test is done. This gives you a simulation of 100 simultaneous Web clients using your application.

Create ASP.NET Core WebAPI Project

Create a new project of type ASP.NET Core Web API as per the screenshots shown below with the name as ProCodeGuide.Samples.ThreadPoolConfig

Now that the project has been created and it will contain the default WeatherForecast controller to return the weather forecast related data. We will modify this controller to add Sync & Async versions of actions to return the weather-related data.

Also, the ThreadCount Model & ThreadsCountController has been added to the project as per the code shown in the ThreadPool Overview section to check the default values of Min & Max threads of the ThreadPool when the application starts.

Modify Controller

Next, we will modify the default Weather Forecast Controller to add the GetSync & GetAsync action methods to the controller. Both the action methods will return the same set of data after a simulated delay of 5 seconds but one will be the synchronous method and the other will be an asynchronous method.

We will modify the Controllers\WeatherForecastController.cs as per the code shown below

[ApiController]
[Route("api/[controller]")]
public class WeatherForecastController : ControllerBase
{
    private static readonly string[] Summaries = new[]
    {
    "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
    };

    private readonly ILogger<WeatherForecastController> _logger;

    public WeatherForecastController(ILogger<WeatherForecastController> logger)
    {
        _logger = logger;
    }

    [HttpGet("GetSync")]
    public IEnumerable<WeatherForecast> GetSync()
    {
        Thread.Sleep(5000);
        return Enumerable.Range(1, 5).Select(index => new WeatherForecast
        {
            Date = DateTime.Now.AddDays(index),
            TemperatureC = Random.Shared.Next(-20, 55),
            Summary = Summaries[Random.Shared.Next(Summaries.Length)]
        })
        .ToArray();
    }

    [HttpGet("GetAsync")]
    public async Task<IEnumerable<WeatherForecast>> GetAsync()
    {
        await Task.Delay(5000);
        return Enumerable.Range(1, 5).Select(index => new WeatherForecast
        {
            Date = DateTime.Now.AddDays(index),
            TemperatureC = Random.Shared.Next(-20, 55),
            Summary = Summaries[Random.Shared.Next(Summaries.Length)]
        })
        .ToArray();
    }
}

We have added the above code where GetSync is the synchronous method and GetAsync is an asynchronous method. In GetSync Thread.Sleep has been used to introduce a delay of 5 seconds & in the GetAsync Task.Delay has been used to introduce a delay of 5 seconds. Let’s understand the difference between them both

Difference between Thread.Sleep & Task.Delay

Thread.Sleep is the classic way of suspending the executing thread. This method will block the thread until the specified time limit has elapsed. i.e. Thread will not be released back to the thread pool for executing other pending requests, if any, in the queue.

Task.Delay will also block the execution of the program until the specified time limit has elapsed but it differs from Thread.Sleep in a way that it will not block the thread instead it will release the thread back to Thread Pool and will again request a thread from the thread pool after the specified time is elapsed. When a thread is released back to the thread pool it is available to execute other pending requests, if any, in the queue.

Let’s run & verify the code

After running the code you should see below screen

As shown in the above screen You should get 2 controllers ThreadsCount & WeatherForecast

api/ThreadsCount in ThreadsCount controller will return the below JSON. This result can vary for you as this default number of threads available for any application at startup depends on the number of cores you have on your machine on which this code is executed.

{
  "minWorkerThreads": 12,
  "minCompletionPortThreads": 12,
  "maxWorkerThreads": 32767,
  "maxCompletionPortThreads": 1000
}

Both the actions api/WeatherForecast/GetSync & api/WeatherForecast/GetAsync in the WeatherForecast controller should return the same JSON result but after a delay of 5 seconds.

In a single request both the Get action methods will take the same time i.e. approximately 5 seconds so we won’t be able to make out the efficiency of asynchronous over synchronous method. To check the performance of the application we will have to load the application with concurrent requests for a stipulated time.

Since we have a default of a minimum of 12 threads so our load should be more than 12 concurrent requests to simulate a condition where all the threads are requested from the thread pool and are busy executing the request.

Measure Application Performance Under Load with default values for ThreadPool

To simulate load we will make use of West Wind WebSurge and use 100 concurrent threads for a period of 60 seconds. This is for demonstration purposes only so we will be using a free version of Web Surge.

First, we will check the performance of both the action methods GetSync & GetAsync with the default initial values for the number of threads in the ThreadPool

Below is the load test results from the GetSync action method

Below is the load test results from the GetAsync action method

As we can see from the comparison of the above results that GetAsync action method has performed far better as compared to the GetSync action method under the same load conditions. In the case of the asynchronous action method, we were able to process more requests as compared to the synchronous action method.

Yes, now we know that asynchronous is a better way to code but you have a situation where you have only synchronous actions methods and converting to asynchronous code will take time. But you are facing an issue when there is a surge in the number of requests i.e. hundreds or thousands of requests are coming at the same and your ASP.NET Core application is not able to handle the load.

So the question is how to scale ASP.NET Core application in this situation? To answer this question we will conduct the same load test after increasing the values for the default number of minimum threads in the ThreadPool and check the results whether we are able to improve the performance of the synchronous action method or not.

Measure Application Performance Under Load with modified values for ThreadPool

Next, we will increase the initial value for Min Threads in ThreadPool and again simulate the same load condition to check the performance of both the action methods i.e. GetSync & GetAsync

Add below the line of code in Program.cs to change the default value for the number of min threads i.e. worker threads and completion port threads

ThreadPool.SetMinThreads(100, 100);

The above code will set the default value of 100 for the number of worker threads and the number of completion port threads for the minimum value of threads. We are not changing the values for the max number of threads in the thread pool as it is not required.

Below is the load test results from the GetSync action method

Below is the load test results from the GetAsync action method

As we can see from the above load test results that after changing the default values for min threads the performance of the GetSync action method has improved and it is at par with the GetAsync action method. So by increasing the value of min threads in the thread pool we are able to scale ASP.NET Core application.

By specifying 100 min threads as the default in ThreadPool we specify that if there is a new request and all the threads in the thread pool are busy then it can instantly spawn up to 100 new threads without any delay to better scale ASP.NET Core application.

There is no change in the performance of the GetAsync action method as it is already optimized using asynchronous programming techniques.

Now everything looks fine as both the actions method GetSync & GetAsync are performing well with this increase in the min threads in the thread pool so you will think that there is no need to change the code as per asynchronous programming technique. Everything is fine in the given load condition but what happens when there is a sudden burst of incoming requests and requests become double from 100 to 200, will our application be able to handle that load?

Let’s run one more load test scenario where the configured minimum threads in the application remain the same i.e. 100 but in Web Surge, we will increase the number of threads from 100 to 200

Below is the load test results from the GetSync action method

As we can see from the above test results that the performance of the synchronous action method has degraded as we increased the number of concurrent sessions\threads in the Web Surge from 100 to 200. We can now see that there is an increase in the maximum as well as average time taken for the request. You can again increase the default number of minimum threads in the thread pool but you will have to keep on doing this as and when the load in your application increases.

Below is the load test results from the GetAsync action method

From the above results, we can see that performance of our GetAsync action method is unchanged even if the load on the application increases. We increased the number of threads in Web Surge from 100 to 200 still our GetAsync performed the same way as with 100 threads.

So far we saw how we make use of ThreadPool threads configuration of the minimum worker threads & minimum completion port threads to scale ASP.NET Core application.

Summary

We learned about how to scale ASP.NET Core Application using ThreadPool. You can apply this technique when your application is facing performance issues or is not able to scale due to a sudden surge in the number of requests.

This technique to increase the minimum threads in the ThreadPool is like a patchwork and should not be considered as a permanent solution as we don’t want so many threads running when the CPU can only process 12 threads (my machine number) at any given time.

Also, an increase in the number of threads is an expensive process and should be done with lots of caution as if we end up creating lots of threads then it can consume a lot of memory and can bring an application or system itself to a halt.