CacheLane Logo

Chapter 06

Ex. Query Parameters (Advanced)

Query parameters, also known as URL parameters or GET parameters, are an important concept in HTTP based communication. They provide a way to pass data to a web server through the URL of a web page or any client application that's able to make GET requests. Query parameters are an essential part of the HTTP protocol and play a crucial role in creating dynamic and interactive web/server applications.

In this chapter, we'll dive deep into query parameters, exploring their structure, usage, and best practices. We'll also discuss their importance in modern web development and how they differ from other methods of data transmission.

Anatomy of a URL with Query Parameters

Before we delve into the specifics of query parameters, let's break down the structure of a URL that includes them:

https://www.cachelane.com/path/to/page?param1=value1&param2=value2

This URL can be divided into several components:

  • Protocol: https://
  • Domain: www.cachelane.com
  • Path: /path/to/page
  • Query string: ?param1=value1&param2=value2

The query string begins with a question mark (?) and contains the query parameters. Each parameter is a key-value pair, where the key and value are separated by an equals sign (=). Multiple parameters are separated by ampersands (&).

Key Components of Query Parameters

Question Mark (?):

  • Signals the start of the query string.
  • Separates the path from the query parameters

Parameter Name:

  • The key in the key-value pair
  • Typically descriptive of the data it represents

Equals Sign (=):

  • Separates the parameter name from its value

Parameter Value:

  • The actual data being passed
  • Can be a string, number, or even a more complex data structure (when properly encoded)

Ampersand (&):

  • Used to separate multiple parameters in the query string. For example, ?param1=value1&param2=value2 contains two parameters - param1 and param2, with the corresponding values value1 and value2 respectively.

Challenge 1: Implementing Basic Query Parameter Parsing

In this challenge, we'll enhance our Router class to parse and handle basic query parameters. We'll focus on extracting query parameters from the URL and making them available to route handlers.

Modify the Router class to parse query parameters from incoming URLs and include them in the route matching result.

Requirements

  1. Update the findRoute method to separate the path from query parameters.
  2. Implement a new method parseQueryParams to extract query parameters from the URL.
  3. Include the parsed query parameters in the object returned by findRoute.
  4. Ensure that routes without query parameters still work as before.
  5. Important: Handle URL encoding and decoding properly. Learn more about URL encoding.

Hints

  1. You may use the URL or URLSearchParams class in Node.js to parse query parameters.
  2. Don't worry about the performance yet, just make sure that your logic works correctly.
  3. Use the current Router class implementation as your starting point. You'll need to modify the findRoute method and add a new parseQueryParams method.

Expected Outcome

After implementing this feature, the findRoute method should return an object that includes:

  • params: Dynamic route parameters (as before)
  • query: An object containing parsed query parameters
  • handler: The matched route handler (as before)

A quick heads up

You may need to change the implementation of the findRoute method, and update how it's extracting the path params. For example, you might notice that if the path parameter is at the end of the URL, the current implementation will not work as expected. You may need to update the implementation to handle such cases.

Example

const router = new Router();
router.get("/users/:id", (req, res) => {
  console.log(req.params.id);
  console.log(req.query);
});

const result = router.findRoute("/users/123?name=Velocy&age=1", "GET");
console.log(result);
// Expected output:
// {
//   params: { id: '123' },
//   query: { name: 'Velocy', age: '1' },
//   handler: [Function]
// }

// Actual output:
// {
//   params: { id: '123?name=Velocy&age=1' },
//   query: { name: 'velocy', age: '1' },
//   handler: [Function]
// }

Solution

Here's the basic solution I came up with for this challenge:

// file: lib/router.js
class Router {
  ...

  /**
   * Parse query parameters from the URL
   * @param {string} queryString - URL segment containing query parameters
   * @returns {object} - Parsed query parameters as key-value pairs
   */
  #parseQueryParams(queryString) {
    if (!queryString) return {};
    // Extract the query string from the URL

    const queryParams = {};
    // Split the query string into key-value pairs
    const pairs = queryString.split("&");

    for (const pair of pairs) {
      // Split each pair into key and value
      const [key, value] = pair.split("=");
      if (key && value) {
        // Decode the value and store the key-value pair
        queryParams[key] = decodeURIComponent(value);
      }
    }

    return queryParams;
  }
 ...
}

The updated findRoute method:

findRoute(path, method) {
+    const indexOfDelimiter = path.indexOf("?");
+    let _path, querySegment;
+
+    if (indexOfDelimiter !== -1) {
+      _path = path.substring(0, indexOfDelimiter);
+      querySegment = path.substring(indexOfDelimiter + 1);
+    } else {
+      _path = path;
+    }
+
+    let segments = _path.split("/").filter(Boolean);
-    let segments = path.split("/").filter(Boolean);

     ...

+    let query = querySegment ? this.#parseQueryParams(querySegment) : {};
    return {
      params,
+      query,
      handler: currentNode.handler.get(method),
    };
  }

The updated run method inside lib/index.js

// file: lib/index.js

function run(router, port) {
  ...

  createServer(function _create(req, res) {
    const route = router.findRoute(req.url, req.method);

    if (route?.handler) {
      req.params = route.params || {};
+    req.query = route.query || {};
      route.handler(req, res);
    } else {
      ...
    }
  }).listen(port);
}

Explanation

// file lib/index.js
function run(router, port) {
  createServer(function _create(req, res) {
    ... unchanged

    if (route?.handler) {
      req.params = route.params || {};
      req.query = route.query || {}; // added this line
      route.handler(req, res);
    } else {
      ...
    }

    ... unchanged
  })
}
req.query = route.query || {};

The very first thing we need to make sure is that the run method in lib/index.js is updated to include the query object in the req object passed to the route handler.

That way, the users of our library can access the query parameters in their route handlers using req.query.

findRoute(path, method) {
  // Separate path from query parameters
  const indexOfDelimiter = path.indexOf("?");
  let _path, querySegment;

  // If query parameters exist, split the URL into path and query segments
  if (indexOfDelimiter !== -1) {
    _path = path.substring(0, indexOfDelimiter);
    querySegment = path.substring(indexOfDelimiter + 1);
  } else {
    _path = path;
  }
  ...
}

First, we need to separate the path from the query parameters. We do this by finding the position of the ? character in the URL using path.indexOf("?"). If the ? character is found, ie. path.indexOf("?") !== -1, we split the URL into _path (the part before ?) and querySegment (the part after ?). If the ? character is not found, _path is set to the entire URL, and querySegment remains undefined.

let query = querySegment ? this.#parseQueryParams(querySegment) : {};

At the end of the findRoute method, we need to parse the query parameters if they exist. We check if querySegment is defined. If it is, we call this.#parseQueryParams(querySegment) to parse the query parameters. If it is not, we set query to an empty object {}.

return {
  params,
  query,
  handler: currentNode.handler.get(method),
};

Finally, we include the parsed query parameters in the object returned by the findRoute method. We add the query object to the return value, alongside params and handler.

Now, let's break down the #parseQueryParams method step by step.

if (!queryString) return {};

Next, we check if the queryString is empty. If it is, we return immediately since there are no query parameters to parse.

const queryParams = {};
const pairs = queryString.split("&");

We then initialize an empty object queryParams to store the parsed key-value pairs. We split the queryString by the & character to get individual key-value pairs. Each pair represents a single query parameter.

for (const pair of pairs) {
  const [key, value] = pair.split("=");

  if (key && value) {
    queryParams[key] = decodeURIComponent(value);
  }
}

return queryParams;

We iterate over each pair in the pairs array. For each pair, we split it by the = character to separate the key and value and check if both key and value are present. If they are, we decode the value using decodeURIComponent to handle URL encoding and store the key-value pair in the queryParams object. For example, name=Velocy%20Framework would be parsed as { name: 'Velocy Framework' }.

Finally, we return the queryParams object containing all the parsed query parameters.

Let's test our implementation

❯ node test.js
{
  params: [Object: null prototype] { id: '123' },
  query: { name: 'John', age: '30' },
  handler: [Function (anonymous)]
}

Everything looks good.

We need to tackle more edge cases

The current implementation of the #parseQueryParams method works well for basic query parameters. However, it doesn't handle more complex scenarios, such as:

  • Special characters in parameter names and values
  • URL-encoded characters in both the key and value parts
  • Efficiency and performance optimizations by avoiding unnecessary string allocations

Challenge 2: Parsing Query Parameters Manually

In this challenge, we'll take our query parameter parsing to the next level by implementing it manually without relying on built-in Node.js classes like URL or URLSearchParams, or using the decodeURIComponent function. This will give you a deeper understanding of how query parameter parsing works under the hood.

Requirements

  • Improve our parseQueryParams method that takes a query string and returns an object with parsed parameters.
  • Handle multiple parameters, including those with the same name. For now, you can overwrite the value if a parameter appears multiple times.
  • Properly decode URL-encoded characters in both parameter names and values.
  • Handle edge cases such as empty values, missing values, and special characters.
  • Integrate this custom parsing function into your Router class.
  • Remember to handle URL decoding for both keys and values.

Expected Outcome

Your custom parseQueryParams function should be able to handle a variety of query strings and return an object with correctly parsed parameters. The Router class should use this function to populate the query property in the object returned by findRoute.

Example

function parseQueryParams(queryString) {
  // Your implementation here
}

// Test cases
console.log(parseQueryParams("name=John&age=30"));
// Expected: { name: 'John', age: '30' }

console.log(parseQueryParams("color=red&color=blue&color=green"));
// Expected: { color: 'green' } // any one value, based on what you want - the first or the last one.

console.log(parseQueryParams("my%20message=Hello%20World%21&empty=&noval"));
// Expected: { 'my message': 'Hello World!', empty: '', noval: '' }

// Integration with Router class
const router = new Router();
router.get("/search", (req, res) => {
  console.log(req.query);
});

const result = router.findRoute("/search?q=nodejs&limit=10", "GET");
console.log(result.query);
// Expected: { q: 'nodejs', limit: '10' }

Solution

The very first thing that I did, was to create a new file lib/utils.js and created an object that maps all the special characters to their respective values.

// file lib/utils.js

// prettier-ignore
const encodedMap = { "3A": ":", "2F": "/", "3F": "?", 23: "#", "5B": "[", "5D": "]", 40: "@", 21: "!", 24: "$", 26: "&", 27: "'", 28: "(", 29: ")", "2A": "*", "2B": "+", "2C": ",", "3B": ";", "3D": "=", 25: "%", 20: " ", 22: '"', "2D": "-", "2E": ".", 30: "0", 31: "1", 32: "2", 33: "3", 34: "4", 35: "5", 36: "6", 37: "7", 38: "8", 39: "9", 41: "A", 42: "B", 43: "C", 44: "D", 45: "E", 46: "F", 47: "G", 48: "H", 49: "I", "4A": "J", "4B": "K", "4C": "L", "4D": "M", "4E": "N", "4F": "O", 50: "P", 51: "Q", 52: "R", 53: "S", 54: "T", 55: "U", 56: "V", 57: "W", 58: "X", 59: "Y", "5A": "Z", 61: "a", 62: "b", 63: "c", 64: "d", 65: "e", 66: "f", 67: "g", 68: "h", 69: "i", "6A": "j", "6B": "k", "6C": "l", "6D": "m", "6E": "n", "6F": "o", 70: "p", 71: "q", 72: "r", 73: "s", 74: "t", 75: "u", 76: "v", 77: "w", 78: "x", 79: "y", "7A": "z", "5E": "^", "5F": "_", 60: "`", "7B": "{", "7C": "|", "7D": "}", "7E": "~", }; // prettier-ignore

Here's a new decoder function that I created to parse query parameters manually, using the encodedMap object:

// file lib/utils.js

let encodedMap = {...}

function fastDecode(string) {
  let result = "";
  let lastIndex = 0;
  let index = string.indexOf("%");

  while (index !== -1) {
    result += string.substring(lastIndex, index);
    const hexVal = string.substring(index + 1, index + 3);
    result += encodedMap[hexVal] || "%" + hexVal;
    lastIndex = index + 3;
    index = string.indexOf("%", lastIndex);
  }

  return result + string.substring(lastIndex);
 }

Here are the changes that I made to the parseQueryParams method:

// file lib/router.js

// Added a default value for queryString to handle empty strings
// and more performance improvements
##parseQueryParams(queryString = "") {
  if (!queryString) return {};

  const queryParams = {};
  const pairs = queryString.split("&");

  for (const pair of pairs) {
    const splitPair = pair.split("=");
    let key = splitPair[0];
    let value = splitPair[1] || "";

    if (key.indexOf("%") !== -1) {
      key = fastDecode(key);
    }

    if (value.indexOf("%") !== -1) {
      value = fastDecode(value);
    }

    if (key) {
      queryParams[key] = value;
    }
  }

  return queryParams;
}

Explanation

The fastDecode function is an optimized method for decoding URL-encoded strings. Let's break down this function and examine why and how it works

// Define the function that takes a string as input
function fastDecodeAll(string) {
  // Initialize an empty result string
  let result = "";

  // Set the initial lastIndex to 0
  let lastIndex = 0;

  // Find the first occurrence of '%' in the string
  let index = string.indexOf("%");

  // More code follows...
}

This function initializes key variables: result to store the decoded string, lastIndex to keep track of the last processed index, and index to find encoded characters.

Main Decoding Loop

// Continue loop while '%' is found in the string
while (index !== -1) {
  // Add the substring from lastIndex to current index to the result
  result += string.substring(lastIndex, index);

  // Extract the two characters following '%'
  const hexVal = string.substring(index + 1, index + 3);

  // Look up the decoded value in encodedMap or keep original if not found
  result += encodedMap[hexVal] || "%" + hexVal;

  // Update lastIndex to skip the processed encoded character
  lastIndex = index + 3;

  // Find the next '%' in the string
  index = string.indexOf("%", lastIndex);
}

This is the core of the decoding process. It efficiently handles encoded characters by:

Appending unencoded substrings directly to the result. Using a pre-computed encodedMap for quick lookups of decoded values. Updating indices to skip processed parts of the string.

Efficiency: This approach is more efficient than our previous solution for several reasons:

  • It avoids repeated string allocations by using a single result string.
  • It uses a pre-computed map for constant-time lookups of decoded values.
  • It processes the string in a single pass, reducing time complexity.

Handling the Remainder

// Add any remaining unprocessed part of the string to the result
return result + string.substring(lastIndex);

This final step ensures that any remaining unencoded portion of the string is included in the result.

Explanation of #parseQueryParams

Function Declaration and Initial Check

// Define the function with a default empty string parameter
##parseQueryParams(queryString = "") {
  // Return an empty object if queryString is falsy
  if (!queryString) return {};

  // Initialize an empty object to store parsed query parameters
  const queryParams = {};

This part of the code remains largely unchanged. The function now has a default parameter of an empty string, which is a minor improvement in handling edge cases.

Splitting the Query String

// Split the query string into key-value pairs
const pairs = queryString.split("&");

This line splits the query string into an array of key-value pairs. It's identical in both versions and has a time complexity of O(n), where n is the length of the query string.

Parsing Individual Pairs

  for (const pair of pairs) {

    // Split each pair into key and value
    const splitPair = pair.split("=");

    // Assign the key (always exists)
    let key = splitPair[0];

    // Assign the value (may be undefined, default to empty string)
    let value = splitPair[1] || "";

This section shows is almost simpler, but the main difference is that we're not destructuring the splitPair array directly into key and value. Instead, we're assigning them individually.

Decoding Key and Value

// Check if key needs decoding
if (key.indexOf("%") !== -1) {
  key = fastDecode(key);
}

// Check if value needs decoding
if (value.indexOf("%") !== -1) {
  value = fastDecode(value);
}

This is the most crucial change in the new implementation. Instead of always using decodeURIComponent, it first checks if decoding is necessary by looking for the '%' character. If found, it uses a custom fastDecode (not shown in the snippet) to decode the string.

This approach is more efficient for several reasons:

  1. It avoids unnecessary decoding operations when the string doesn't contain encoded characters.
  2. The custom fastDecode can be optimized for the specific use case, potentially outperforming the generic decodeURIComponent.

Assigning to Query Params Object

// Add the key-value pair to the queryParams object if key exists
if (key) {
  queryParams[key] = value;
}

This part is similar to the original version, but it doesn't need to check if the value exists because we've already defaulted it to an empty string if it was undefined.

Returning the Result

  // Return the object containing all parsed query parameters
  return queryParams;
}

Here's a quick overview of the changes made to the #parseQueryParams method:

  for (const pair of pairs) {
-   const [key, value] = pair.split("=");
-   if (key && value) {
-     queryParams[key] = decodeURIComponent(value);
-   }
+   const splitPair = pair.split("=");
+   let key = splitPair[0];
+   let value = splitPair[1] || "";
+
+   if (key.indexOf("%") !== -1) {
+     key = fastDecode(key);
+   }
+
+   if (value.indexOf("%") !== -1) {
+     value = fastDecode(value);
+   }
+
+   if (key) {
+     queryParams[key] = value;
+   }
  }

Note

Note: The fastDecode function here is still not the complete implementation. You may need to add more characters to the encodedMap object to handle all possible encoded characters.

Conclusion

Query parameters are a powerful and flexible tool, which allow for the transmission of data through URLs, enabling dynamic and interactive client-side applications. By understanding their structure, usage, and best practices, we can create more efficient, user-friendly, and secure web applications.

Remember to always consider security implications when working with query parameters, and choose the most appropriate method of data transmission based on your specific use case and requirements.

Previous
Building our first web-server