Using Semaphore to control a resource pool

The Semaphore in effect manages a set of "permissions". A thread asks to be granted a permission before continuing to access the pool. When it has finished, it releases the permission. If when a thread asks for a permission, the maximum number have already been given out, then the requesting thread blocks until permission becomes available, up to some maximum time specified by the programmer. So our resource pool idiom looks something like this:

public class ResourcePool<T> {
  private final Semaphore sem =
     new Semaphore(MAX_RESOURCES, true);
  private final Queue<T> resources =
     new ConcurrentLinkedQueue<T>();
  
  public T getResource(long maxWaitMillis)
     throws InterruptedException, ResourceCreationException {

    // First, get permission to take or create a resource
    sem.tryAcquire(maxWaitMillis, TimeUnit.MILLISECONDS);

    // Then, actually take one if available...
    T res = resources.poll();
    if (res != null)
      return res;

    // ...or create one if none available
    try {
      return createResource();
    } catch (Exception e) {
      // Don't hog the permit if we failed to create a resource!
      sem.release();
      throw new ResourceCreationException(e);
    }
  }
  public void returnResource(T res) {
    resources.add(res);
    sem.release();
  }
}

The true parameter passed into the constructor declares the semaphore as fair: that is, threads waiting for a resource will be given the next free one on a more or less first-come-first-served basis. For locks in many situations, this isn't what we want, because the code that holds a lock is generally a quick in-and-out operation, so even in the unfair case, waiters won't on average wait too long, and we'll get better throughput. In the case of access to something like a database connection, threads may well hold on to them for several milliseconds (or even several hundred), so a thread could wait an unacceptable amount of time if other threads "jump the queue". Of course, this is something that you must decide, depending on how long threads will be holding on to the particular resources in question.

Note that Semaphore controls purely the number of accesses to the pool and waiting for/interrupting access to the pool; it doesn't control how we manage synchronization when actually taking a resource from the list or putting it back. In this example, we use a ConcurrentLinkedQueue, which allows us to concurrently add to the end of the queue and take from the beginning.

Notice another subtlety in the above example: if during resource allocation we fail to create a resource, then we must immediately release the permit that we acquired. Just because we failed to create a resource shouldn't generally prevent another thread (or even our thread) from trying again later!