Response and Status Codes

It's time to discuss the response part of our cURL output, that is:

← HTTP/1.1 200 OK
← Date: Wed, 23 Aug 2023 13:13:32 GMT
← Connection: keep-alive
← Keep-Alive: timeout=5
← Content-Length: 11
← * Connection #0 to host localhost left intact
Hello world%

This is the response sent by the server, which in case is our simple Node.js HTTP server. Let's go through this line by line:

← HTTP/1.1 200 OK

This line indicates the HTTP protocol version, i.e HTTP/1.1 and the response status code (we'll look at status code in a bit). In this case, the response status code is 200, which means OK or that the request was successful. The server has successfully processed the request and is sending back the requested resource.

Imagine you're requesting a webpage from a server, and you get a 200 OK status code. It's like receiving a green light from the server, indicating that the page you requested has been found and is being sent back to you, and everything went OK.

← Date: Wed, 23 Aug 2023 13:13:32 GMT

This shows the date and time when the response was generated on the server.

← Connection: keep-alive

This line specifies the Connection HTTP header that is used to indicate that the connection between the client (our curl command) and the server will not be closed after this request-response cycle.

Instead, it will be kept alive so that subsequent requests can be made without the overhead of establishing a new connection each time. By keeping the connection open, the client and server can exchange data more efficiently and with less latency, which can lead to improved performance and faster response times.

The value of keep-alive is particularly useful for applications that require frequent requests to be made to a server, such as real-time web applications or streaming services. But for bi-directional communication we often use Connection header with the value Upgrade.

The Connection header can also have a value of upgrade. This means the server might want to change the way it communicates with the client. The new way might be better for certain types of actions, like sending a lot of data at once.

For example, we can upgrade from HTTP 1.1 to HTTP 2.0, or from an HTTP or HTTPS connection into a WebSocket.

← Keep-Alive: timeout=5

Keep-Alive is an another header, which is closely related to the Connection header that we talked about previously. It has a value, that has an attribute named timeout with a value of 5.

It specifies that the server will close the connection if no new requests are made within 5 seconds. This is particularly useful in preventing idle connections from hogging server resources unnecessarily.

Imagine if a server had to keep a connection open indefinitely, even when no data is being transferred. This would lead to a significant waste of resources, which could have been better allocated to active connections. This ensures that the server can efficiently manage its resources, leading to optimal performance and a better user experience.

There is no "standard" value for the timeout duration, but here are some recommended guidelines that I've been following for my applications:

1. Short-Lived Requests: If the application involves short-lived requests and responses, where clients frequently make requests and receive responses within a short time frame, a relatively low timeout value might be suitable. A value between 1 to 10 seconds could work in this case.

   For example, a chat application where messages are sent frequently between users might benefit from a lower timeout value.

2. Long-Polling or Streaming: If the application involves long-polling or streaming, where clients maintain a connection for an extended period to receive real-time updates, a longer timeout value would be appropriate. Values between 30 seconds to a few minutes might be reasonable.

   For eg. a stock market monitoring application that provides real-time updates to traders might use a longer timeout to keep the connection open while waiting for market changes.

3. Low Traffic and Resource Constraints: If your server has limited resources and serves a low volume of requests, you might want to use a shorter timeout to free up resources more quickly. A value around 5 to 15 seconds could be considered.

4. High Traffic and Scalability: In scenarios with high traffic and the need to maximize server efficiency, a slightly longer timeout value might be chosen. This allows clients to reuse connections more frequently, reducing the overhead of connection establishment. Values around 15 to 30 seconds could be appropriate.

Remember that the chosen timeout value should strike a balance between keeping connections open long enough to benefit from connection reuse and not tying up server resources unnecessarily.

← Content-Length: 11

We talked about Content-Length earlier, to revisit: It's the length of the response body in bytes. The body in our case is the Hello world text.

←

This line represents a blank line that separates the response headers from the response body. It indicates the end of the header section and the start of the actual content being sent back in the response.

* Connection #0 to host localhost left intact

After the cURL completes the HTTP request and receives the response from the server, it keeps the network connection open and in a "keep-alive" state. This means that the TCP connection established for the request-response cycle is maintained and not terminated immediately. The connection is "left intact" to allow for the possibility of reusing it for subsequent requests to the same host.

This behavior aligns with the idea of connection reuse that we talked about previously, where keeping the connection open reduces the overhead of TCP's connection establishment and termination.

Connection: close in action

Here's the current code for our HTTP server, that we wrote in the chapter 5.0:

// file: index.js

const http = require("node:http");

function handle_request(request, response) {
  response.end("Hello world");
}

const server = http.createServer(handle_request);

server.listen(3000, "localhost");

Let's add two more headers: Content-Type and Connection in our response.

// file: index.js

...

function handle_request(request, response) {
    /** Set the "Content-Type" header to "text/plain" */
    response.setHeader("Content-Type", "text/plain");

    /** Set the "Connection" header to "close" */
    response.setHeader("Connection", "close")

    /** Write the response body */
    response.end(`
Request method: ${request.method}
Request URL: ${request.url}
Request headers: ${JSON.stringify(request.headers, null, 2)}
HTTP Version: ${request.httpVersion}
HTTP Major Version: ${request.httpVersionMajor}
HTTP Minor Version: ${request.httpVersionMinor}
`);
}

...

Let's re-start our server using node index, and try to send an HTTP request again.

❯ curl http://localhost:3000 -v
*   Trying 127.0.0.1:3000...
* Connected to localhost (127.0.0.1) port 3000 (#0)
→ GET / HTTP/1.1
→ Host: localhost:3000
→ User-Agent: curl/7.87.0
→ Accept: */*
→
* Mark bundle as not supporting multiuse
← HTTP/1.1 200 OK
← Content-Type: text/plain
← Connection: close
← Date: Wed, 30 Aug 2023 01:16:14 GMT
← Content-Length: 156
←

Request method: GET
Request URL: /
Request headers: {
  "host": "localhost:3000",
  "user-agent": "curl/7.87.0",
  "accept": "*/*"
}
HTTP Version: 1.1 1.1
* Closing connection 0

That's too much to look at. Let's look at some key points.

Since we manually specified the Content-Type header in the handle_request function, it shows up as a part of the response, it did not previously.

The headers property of the request object provides access to all key-value pairs of the headers. Additionally, you can access the complete HTTP version, as well as the major and minor versions. This is useful if you want to upgrade an HTTP/1.1 connection to a WebSocket connection or HTTP/2.

The last line now says * Closing connection 0. This is because we set the Connection: close header, which instructs the client to immediately close the connection and not use it again for subsequent requests.

It also indicates that the HTTP method is GET, which is somewhat strange since we did not specify it ourselves. We instructed cURL to make a request to the http://localhost:3000 URL. Why was it automatically added?

Well turns out, by default every request is assumed to be a GET request even if we do not specify a method. Let's try to change the method to POST. With cURL you can set methods like this:

curl -X POST http://localhost:3000 -v

## or

curl --request POST http://localhost:3000 -v

By sending a POST request, our response output changed to:

...

Request method: POST
Request URL: /
Request headers: {
  "host": "localhost:3000",
  "user-agent": "curl/7.87.0",
  "accept": "*/*"
}
HTTP Version: 1.1 1.1
* Closing connection 0

...

The code seems to repeat the setHeader method call two times. What if we have 10-12 response headers that we wish to set? Is there a better way? Yes, there is: response.writeHead()

Let's look at the function signature:

writeHead(
    statusCode: number,
    headers?: OutgoingHttpHeaders | OutgoingHttpHeader[]
): this;

This is written in Typescript. Don't worry if you do not know TypeScript. I'll explain it to you in easy words.

In Typescript, you have to specify the type of every parameter for a function. In this case, the first parameter - statusCode argument has a type of number. The second parameter header has a type of either OutgoingHttpHeaders or OutgoingHttpHeader[]. Before looking at these types, let's talk about ? in headers?.

The ? specifies that the parameter is optional. So, if you do not provide any argument for headers, it will work fine and not throw any error.

OutgoingHttpHeaders is a type representing a collection of outgoing HTTP headers. Each header is defined as a property with the header name as the property key and the header value as the property value. This type provides a structured way to specify headers.

OutgoingHttpHeader[] is an array of headers, where each header is defined as an object containing a header name (string) and one or more corresponding header values (string[]).

This provides a lot of flexibility in how you provide headers when calling the writeHead function. But the preferred way is to use an object with keys. Let's update our code snippet to use response.writeHead instead of response.setHeader

// file: index.js

...

function handle_request(request, response) {
    response.writeHead(200, {
        "Content-Type": "text/plain",
        "Connection": "close"
    })

    ...
}

...

Now the code looks much more easier to reason about. One more thing that could be an issue if you're working with any Node.js project that we talked about earlier.

The editor has no way to identify the type of request and response arguments in the handle_request method, and because of that you can not enjoy the intellisense or autocompletion while trying to access properties of request or response.

If we were using a callback approach, like this:

const server = http.createServer(function handle_request(request, response) {
   ...
});

you would've got the autocompletes when you tried to access any property or method of request or response object. This might look okay, but it's always better to have reusability in mind. To fix this, let's just use jsdoc styled comments:

// file: index.js

/**
 *
 * @param {http.IncomingMessage} request
 * @param {http.ServerResponse} response
 */
function handle_request(request, response) { .. }

Try to access a property of request by typing request followed by a period:

This works fine! But how do we actually find the type of the arguments in the first place?

It's quite simple.

1. In your code, locate the line where you're calling http.createServer and providing the handle_request function as a parameter.

2. On that line, hover your mouse cursor over the createServer function to highlight it.

3. With the createServer function highlighted, use the keyboard shortcut Ctrl + Click (or Cmd + Click on macOS) to jump to the function's declaration.

Doing this should take you to the definition of createServer. This is where things might look a bit complex due to the TypeScript type declarations. This is the type declaration for the method http.createServer:

function createServer<
  Request extends typeof IncomingMessage = typeof IncomingMessage,
  Response extends typeof ServerResponse = typeof ServerResponse
→ (requestListener?: RequestListener<Request, Response>): Server<Request, Response>;

We only care about the following lines:

Request extends typeof IncomingMessage = typeof IncomingMessage
Response extends typeof ServerResponse = typeof ServerResponse

• Request extends typeof IncomingMessage means that Request is constrained to be a subtype of the IncomingMessage type. In simpler words, Request can only be a type that inherits from or is compatible with IncomingMessage.

So in short, the IncomingMessage can be used for the type of request parameter. Same for the response, we're using the type ServerResponse.

Let's talk about status codes now.

Status Codes

Status Codes are three-digit numeric codes generated by a web server in reply to a client's request made through an HTTP request. They provide crucial information about the status of the requested resource or the outcome of the client's request.

Status codes are grouped into five classes, each indicating a specific category of response. Let's look at a simple analogy on why do we need status codes.

When you ask a librarian for a specific book, they may respond in one of three ways: "Here's the book," "Sorry, we don't have that book," or "I'm not sure, let me go check." These responses provide information about the status of your request. Similarly, when your web browser sends a request to a web server, it's like asking for something in the vast digital library of the internet.

HTTP status codes work in a similar way. They're responses from the web server to your browser's request. When you click on a link, fill out a form, or make any kind of request on the internet, the web server replies with a status code to let your browser know what happened with your request.

These codes are grouped into classes to help you understand the general situation. Imagine you're playing a game, and you have four types of cards: success cards, redirection cards, client error cards, and server error cards. Each card type represents a different situation. The status code classes work like these card types, categorizing responses based on their nature.

• 1xx Informational Responses: These are like hints that the web server is still working on your request. It's like the librarian saying, "I'm checking the back shelves for your book."

• 2xx Successful Responses: These are success cards. They mean your request was understood and fulfilled. It's like the librarian handing you the book you asked for.

• 3xx Redirection Responses: These are like directions the librarian gives you to find the book in a different section. Similarly, your browser might be directed to a different URL.

• 4xx Client Error Responses: These are cards that say something is wrong with your request. Maybe you're asking for something that doesn't exist or you're not allowed to access.

• 5xx Server Error Responses: These cards mean the library (web server) is having trouble. It's like the librarian apologizing for not being able to find the book due to a problem.

As server/API programmers we should make sure that we're sending valid and reasonable response codes back to the client. We'll learn about many status codes when we build our backend framework. But in-case you're curious you can check all the status codes here: