Managing concurrency

One simple way to handle concurrent requests is to choose a specific number of threads in advance and never change this number. This method is easy to design and implement, but the number of threads might have no relationship to the client needs. At times, the clients might be waiting because all the threads are busy. At other times, the threads might remain inactive, wasting system resources.

An alternative technique is to start the server with one thread and create a new thread for new requests each time the server receives a request to process. This way, the server always has one thread available for a new request. You delete each thread as the request completes processing, always retaining a minimum of one thread.

With this method, a client never needs to wait for a thread and there are always enough threads to handle client resources. You need to keep in mind that thread creation is relatively expensive, and in this model the server is creating threads constantly. Without an upper limit on the number of threads the server can have at one time, system performance can degrade and available memory can become an issue. You can enhance this model by limiting the total number of threads, which means that a client might have to wait for a thread to unblock. Also, you can cache threads and avoid creating more until the server exhausts the cache.