How Caching Mechanism WORKS in ASP.NET Core?

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddMemoryCache(); // Add this line

        services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
    }
}

2. Naive implementation

For the sake of this tutorial, we’ll use a free external API such as reqres.in/api/users. Let’s imagine we want to cache the response of the API. For simplicity, I have used the example code provided by Microsoft Docs.

// Code removed for brevity
...

// Look for cache key.
if (!_cache.TryGetValue(CacheKeys.Entry, out cacheEntry))
{
     // Key not in cache, so get data.
     cacheEntry = DateTime.Now;

     // Set cache options.
     var cacheEntryOptions = new MemoryCacheEntryOptions()
         // Keep in cache for this time, reset time if accessed.
         .SetSlidingExpiration(TimeSpan.FromSeconds(3));

     // Save data in cache.
     _cache.Set(CacheKeys.Entry, cacheEntry, cacheEntryOptions);
}

Explanation

The code is pretty straightforward. We first check whether we have the value for the given key present in our in-memory cache store. If not, we do the request to get the data and store in our cache. What SetSlidingExpiration does is that as long as no one accesses the cache value, it will eventually get deleted after 10 seconds. But if someone accesses it, the expiration will get renewed.

Here’s my implementation with a bit of code re-structuring:

var users = _cacheProvider.GetFromCache<IEnumerable<User>>(cacheKey);
if (users != null) return users;

// Key not in cache, so get data.
users = await func();

var cacheEntryOptions = new MemoryCacheEntryOptions()
    .SetSlidingExpiration(TimeSpan.FromSeconds(10));

_cacheProvider.SetCache(cacheKey, users, cacheEntryOptions);

return users;

await func() will wait and return the response from our external API endpoint (provided that it’s passed into our method) so that we can use that value to store in our cache.

3. Refactoring our code to use locking

Now, let’s assume that we have several users accessing our service which means there could be multiple accessing our in-memory cache. One way to make sure that no two different users get different results is by utilising .Net locks. Refer the following scenario:

Let’s breakdown the sequence of requests and responses:

1. User A makes a request to our web service
2. In-memory cache doesn’t have a value in place, it enters in to lock state and makes a request to the Users Service
3. User B makes a request to our web service and waits till the lock is released
4. This way, we can reduce the number of calls being made to the external web service. returns the response to our web service and the value is cached
5. Lock is released, User A gets the response
6. User B enters the lock and the cache provides the value (as long it’s not expired)
7. User B gets the response

The above depiction is a very high-level abstraction over all the awesome stuff that happens under the covers. Please use this as a guide only. Let’s implement this!

...
var users = _cacheProvider.GetFromCache<IEnumerable<User>>(cacheKey);

if (users != null) return users;
try
{
    await semaphore.WaitAsync();

    // Recheck to make sure it didn't populate before entering semaphore
    users = _cacheProvider.GetFromCache<IEnumerable<User>>(cacheKey);

    if (users != null) return users;
    users = await func();

    var cacheEntryOptions = new MemoryCacheEntryOptions()
        .SetSlidingExpiration(TimeSpan.FromSeconds(10));

    _cacheProvider.SetCache(cacheKey, users, cacheEntryOptions);
}
finally
{
    // It's important to do this, otherwise we'll be locked forever
    semaphore.Release();
}

return users;

Explanation

Same as in our previous example we first check our cache for the presence of the value for a key provided. if not, we then asynchronously wait to enter the Semaphore. Once our thread has been granted access to the Semaphore, we recheck if the value has been populated previously for safety. If we still don’t have a value, we then call our external service and store the value in the cache.

Demo

Hope you enjoyed this tutorial. Happy coding!