Skip to Content
🎉 Welcome to my notes 🎉
Node.js3. Buffers🧱 ArrayBuffer() in JavaScript

🧱 ArrayBuffer() in JavaScript

An ArrayBuffer is a low-level object used to represent a generic, fixed-length chunk of raw binary data. Think of it as a block of memory. You cannot directly read from or write to an ArrayBuffer; instead, you must use a “view” object to interpret and manipulate its contents.

🔑 Key Concepts

  • Raw Data Container: It holds a sequence of bytes but doesn’t have any specific format or mechanism to access those bytes.
  • Fixed Length: The size (in bytes) of an ArrayBuffer is set when it’s created and cannot be changed.
  • Requires a View: To interact with the data inside the buffer, you need a view. The two main types of views are:
    • TypedArray: Treats the data as an array of a specific numeric type (e.g., Uint8Array for 8-bit unsigned integers, Float32Array for 32-bit floating-point numbers).
    • DataView: Allows you to read and write different data types at any byte offset within the buffer, giving you more fine-grained control.

💻 Code Examples

  1. Creating an ArrayBuffer and a View Here, we create a 16-byte buffer and then create a Uint8Array view to interact with it as an array of 8-bit integers.

    buffer-creation.js
    // Create a 16-byte buffer, initialized with zeros.
const buffer = new ArrayBuffer(16);
 
// Create a view to interpret the buffer as an array of 8-bit unsigned integers.
const view = new Uint8Array(buffer);

  2. Writing to and Reading from the Buffer via the View Once you have a view, you can use standard array syntax to manipulate the data.

    buffer-write-read.js
    // Write data to the buffer using the view
view[0] = 255; // Set the first byte
view[1] = 100; // Set the second byte
 
// Read data back
console.log(view[0]); // Output: 255
console.log(buffer); // Output: ArrayBuffer { [Uint8Contents]: <ff 64 00 00 ...>, byteLength: 16 }

🎯 Signed and Unsigned Values

When working with binary data, a sequence of bits (like 10001001) can represent different numbers depending on how we interpret it. The most fundamental interpretation is whether the number is signed (can be positive or negative) or unsigned (can only be positive or zero). This choice directly impacts the range of values that can be stored.

➕ Unsigned Integers

An unsigned integer uses all its bits to represent the magnitude (the numerical value). This means it cannot represent negative numbers.

  • Purpose: To represent values that are always non-negative, like counts, lengths, or colors (e.g., RGB values from 0-255).
  • Range (for n bits): 0 to 2^n - 1.
  • Example (8-bit): An 8-bit unsigned integer (Uint8) can store values from 0 to 255 (2^8 - 1).
    • Binary 00000000 is 0.
    • Binary 11111111 is 255.

➕➖ Signed Integers

A signed integer uses one of its bits (the most significant bit or MSB) to indicate the sign. The standard method for this is called Two’s Complement.

  • Purpose: To represent values that can be positive or negative, like temperatures, financial transactions, or coordinates.
  • Range (for n bits): -2^(n-1) to 2^(n-1) - 1.
  • Example (8-bit): An 8-bit signed integer (Int8) can store values from -128 to 127.
    • Binary 01111111 is 127.
    • Binary 10000000 is -128.
    • Binary 11111111 is -1.

⚖️ A Direct Comparison

The key takeaway is that the same binary pattern has a different value depending on the interpretation. Consider the 8-bit pattern 11111111:

  • As a Uint8 (unsigned), it is 255.
  • As an Int8 (signed), it is -1.

This is why you must know the data type you are working with. When you use a TypedArray in JavaScript or a Buffer in Node.js, you must choose the correct view (Uint8Array vs. Int8Array) or method (buf.readUInt8() vs. buf.readInt8()) to avoid misinterpreting the data.

📖 Reading and Writing to ArrayBuffers

To read and write data to an ArrayBuffer, you typically use a TypedArray or a DataView. These views allow you to interpret the raw binary data in various formats.

  1. 🔢 TypedArray: Use a TypedArray when your buffer contains a simple list of same-sized numbers. You interact with it just like a regular JavaScript array.

  • How it works: You create a view for a specific type (like Uint16Array for 2-byte unsigned integers). The view handles interpreting the bytes for you.

  • Best for: Image data (pixels), audio samples, WebGL data, or any large, uniform numeric dataset.

buffer-typedarray.js
// Here, we store two 16-bit numbers in a 4-byte buffer.
 
const buffer = new ArrayBuffer(4); // Create a 4-byte buffer.
 
// Create a view to treat the buffer as an array of 16-bit unsigned integers.
// 4 bytes / 2 bytes per integer = 2 integers.
const view = new Uint16Array(buffer);
 
// Write to the buffer using array syntax.
view[0] = 1000;
view[1] = 2000;
 
// Read from the buffer.
console.log(view[0]); // Output: 1000
console.log(view[1]); // Output: 2000
 
// At the byte level, the buffer now contains the binary representation of these numbers.
  1. 🔬 DataView: Use a DataView when your buffer contains a structured mix of different data types. It gives you complete control over where and how you read/write data.

  • How it works: You must specify the exact byte offset and data type for every operation (e.g., setUint16, getFloat32).

  • Endianness: DataView also lets you control endianness—the order of bytes in a multi-byte number. You can pass true as the last argument for little-endian or false (the default) for big-endian. This is critical for network protocols (which often use big-endian) and file formats.

  • Best for: Parsing binary file formats (.png, .class) or handling complex network protocol packets.

buffer-dataview.js
// Here, we write a Uint8 and a Float32 into the same 5-byte buffer.
 
const buffer = new ArrayBuffer(5); // Create a 5-byte buffer.
 
// Create a DataView of the buffer.
const dataView = new DataView(buffer);
 
// Write an 8-bit unsigned integer at byte offset 0.
dataView.setUint8(0, 255);
 
// Write a 32-bit (4-byte) float starting at byte offset 1.
dataView.setFloat32(1, 3.14);
 
// Read the values back.
console.log(dataView.getUint8(0)); // Output: 255
console.log(dataView.getFloat32(1)); // Output: 3.140000104904175

🚃 Transferring ArrayBuffer Data to Disk and Network

Once you have data in an ArrayBuffer, the next step is often to do something with it, like saving it to a file or sending it to a server. Node.js is excellent at this, but its core APIs (for file system, networking, etc.) are designed to work with its own special class: Buffer. Therefore, the key to transferring ArrayBuffer data is to first convert it into a Node.js Buffer.

🌉 The Bridge: From ArrayBuffer to Buffer

Think of Buffer as Node.js’s super-powered version of Uint8Array. In fact, the Buffer class is a subclass of Uint8Array. This makes it easy to work with ArrayBuffers.

To make an ArrayBuffer usable by Node.js APIs, you simply “wrap” it in a Buffer object.

The critical conversion step is: Buffer.from(myArrayBuffer)

buffer-conversion.js
// You have an ArrayBuffer from somewhere (e.g., a file upload, Web API).
const arrayBuffer = new ArrayBuffer(4);
const view = new Uint32Array(arrayBuffer);
view[0] = 1234567890; // Put some data in it.
 
// Convert it to a Node.js Buffer.
const nodeBuffer = Buffer.from(arrayBuffer);
 
console.log(nodeBuffer); // Output: <Buffer d2 02 96 49>

Now nodeBuffer can be used directly with Node’s file system and networking modules.

💾 To Disk: Writing a Buffer to a File

The built-in fs (File System) module lets you read and write files. Its writeFile and writeFileSync methods accept Buffer objects directly.

buffer-write-file.js
import fs from 'fs';
 
// 1. Create our binary data in an ArrayBuffer.
const buffer = new ArrayBuffer(8);
const view = new DataView(buffer);
view.setUint32(0, 2024);    // Write a 4-byte integer.
view.setFloat32(4, 3.14);  // Write a 4-byte float.
 
// 2. Convert to a Node.js Buffer.
const nodeBuffer = Buffer.from(buffer);
 
// 3. Write the Buffer directly to a file.
try {
  fs.writeFileSync('data.bin', nodeBuffer);
  console.log('File saved successfully!');
} catch (err) {
  console.error('Error writing file:', err);
}
 
// To read it back:
// const dataFromFile = fs.readFileSync('data.bin');
// console.log(dataFromFile);

🌐 To the Network: Sending a Buffer over HTTP

Node.js’s http module allows you to create servers and clients that can send and receive binary data. When sending binary data, you can use the Buffer directly in the response.

buffer-server.js
import http from 'http';
 
// Create a simple HTTP server.
const server = http.createServer((req, res) => {
  // 1. Create our binary data in an ArrayBuffer.
  const buffer = new ArrayBuffer(8);
  const view = new DataView(buffer);
  view.setUint32(0, 2024); // Write a 4-byte integer.
  view.setFloat32(4, 3.14); // Write a 4-byte float.
 
  // 2. Convert to a Node.js Buffer.
  const nodeBuffer = Buffer.from(buffer);
 
  //   3. Set the response headers to allow CORS and indicate binary content.
  res.setHeader("Access-Control-Allow-Origin", "*");
 
  // 4. Send the Buffer as the response.
  res.writeHead(200, { "Content-Type": "application/octet-stream" });
  res.end(nodeBuffer);
});
 
// Start the server.
server.listen(5000, () => {
  console.log("Server listening on http://localhost:5000");
});

Now, when you access http://localhost:5000, the server will respond with the binary data stored in the ArrayBuffer, wrapped in a Node.js Buffer.

buffer-client.js
// using fetch in a browser or Node.js client
async function fetchData() {
  try {
    const response = await fetch('http://localhost:5000');
 
    if (!response.ok) {
      throw new Error(`HTTP error! Status: ${response.status}`);
    }
 
    // Get the response body as an ArrayBuffer
    const arrayBuffer = await response.arrayBuffer();
 
    // Create a DataView to read from the buffer
    const view = new DataView(arrayBuffer);
 
    // Read the 4-byte unsigned integer from the start (offset 0)
    const intValue = view.getUint32(0);
 
    // Read the 4-byte float from offset 4
    const floatValue = view.getFloat32(4);
 
    console.log(`Successfully fetched and parsed data:`);
    console.log(`Integer Value: ${intValue}`);   // Should be 2024
    console.log(`Float Value: ${floatValue}`); // Should be ~3.14
 
  } catch (error) {
    console.error('Failed to fetch data:', error);
  }
}
 
fetchData();
Last updated on
📦 Mastering Buffers - Handling Binary Data in Node.js4. Event Driven Architecture