You may occasionally encounter strange errors when using Microsoft’s websites, such as Bad Request, Request Too Long, or Connection Rejected. Cookies are mostly to blame for these errors. I’ll outline the most frequent reasons for these problems in this post, along with advice on how to keep your websites free of them.

Sessions

One of the simplest—and, in PHP, arguably the most popular—methods for keeping data for a user’s browsing session is through sessions. It generates one cookie and sends it to the user; typically, it has a session-specific unique identifier that is connected to the server-stored data. File system storage is the most popular method for storing session data in PHP. By default, an in-memory cache handles sessions in ASP.NET Core.

You will strive for a stateless architecture when coding contemporary websites, which denotes that no state is kept directly on the server. Because stateless architecture does not store data on web workers, it enables you to scale your service quickly and easily. Naturally, sessions make the web stateful by default because data must be preserved somewhere, usually in the filesystem or server memory.

Using an external storage for the sessions—implemented in ASP.NET Core as a distributed cache—is one way to detach the session storage from the web workers. Another way to handle this in PHP is to use a persistent shared storage across web workers, which is very common for shared web hosts. The storage is typically handled by Redis or Memcached, but session data can also be stored in SQL.

Individual cookies

Data that is directly stored in cookies is the “stateless” method of storing it for the user. Even though it seems so simple, this presents many difficulties on its own.

Storing large data in cookies

What would happen if you needed to store a lot of data in the cookie? Let’s say the user’s complete identity in ASP.NET Core? Generally speaking, it will produce a cookie that is larger than 4096 bytes (sometimes much larger—for instance, if you store all id_token claims in it or utilize it as an ADAL token cache). In fact, ASP.NET Core is very clever about this; it chunks the resultant cookie into pieces (and then serializes them back).

Request size

The resulting request size with cookies is another crucial component that developers frequently ignore.

Let’s assume that the total size of your cookies is 4kB. The website downloads extra resources (such as CSS, JS, images, etc.) when it loads. You will be making much larger requests than usual because the cookie header will be added to all requests if they are coming from the same domain.

On PCs with strong internet connections, we don’t really need to worry about this, but an increasing percentage of Internet users use mobile devices. Although the number of people using mobile devices is increasing quickly, many regions of the world still require EDGE or comparable speeds due to poor operator coverage. All kilobytes are counted when using EDGE speeds, not to mention that FUP limits are enforced in many nations.

Server limits

We are now returning to the initial pain point errors of a comparable nature:

The real server limits are the source of these. In reality, the majority of servers impose different restrictions on the requests (such as maximum post size, header size, maximum request length, and others). Of course, the maximum header size limit—which varies depending on the server being used—affects cookies.

– Internet Information Services (WS2016) – 16kB per header

• These restrictions can be further set up to accommodate up to 16MB of header space each. You can find instructions for configuring here.

– Nginx – 8kB altogether

• The limit can be changed as per instructions here.

– Apache (2.2) – 8kB altogether

• The limit can be changed as per instructions here.

– Kestrel – 32kB altogether, maximum of 100 headers

• KestrelServerLimits is where the limit can be adjusted.

You can see that different servers implement different maximum header sizes based on the numbers above. It is generally safe to state that the default configuration for your cookies should not allow them to be larger than 4kB.

When you surpass the server limit, an error will be returned to you; either the server will show an error page and end the request, or it will end the request immediately without providing a response.

Solution

You are probably aware of the issues associated with over-using cookies on your websites by now. We will now investigate solutions that cater to both developers and users.

Users

When you encounter such a problem, the user solution is very straightforward: delete your cookies for that particular website. You can accomplish this through the settings or the Developer Tools in your browser. More information about that is available here.

Developers

The user portion was simple; the developer portion is now…

Open the Network tab in the Developer Tools and see how much data is transferred back and forth without requiring that the data be present at all. This can be a really entertaining thing to do sometimes.

Use HTML5 for local storage

Use a different type of storage than cookies when you need to store data in the browser and don’t need to access it on the server-side. LocalStorage is a good option.

Use Session Storage when necessary

Use of sessions when required is safe if you don’t need access to the client’s cookie data. By overriding the SessionStore variable, you can implement ITicketStore and use it to store the Cookie Authentication claims.

You can start small with a straightforward in-memory implementation and subsequently go larger with a distributed cache implementation, but doing so will come at the expense of having a stateful application.

Split cookies per subdomain and site paths

The problem lies not in the quantity of cookie data that is kept, but rather in the incorrect targeting of the cookies. Path and domain attributes are useful. You can also read about how crucial it is to set the right attributes in terms of security.

Another option is to set aside a subdomain for static resources, which, if the domain is configured properly, will not receive any cookies at all.