Appearance
Buffers and Core Modules
42. What are the DNS, OS, path, crypto, and zlib modules used for?
The short answer
They are built-in Node.js modules, imported with the node: prefix. dns turns host names into IP addresses. os reports facts about the machine, such as CPU count, memory and host name. path builds and takes apart file paths correctly on every operating system. crypto provides hashing, random values, encryption and signatures. zlib compresses and decompresses data with gzip, deflate and Brotli. You use them without installing anything.
ts
import dns from 'node:dns/promises'
import os from 'node:os'
import path from 'node:path'
import { randomUUID, createHash } from 'node:crypto'
import { gzipSync, gunzipSync } from 'node:zlib'
await dns.lookup('api.payments.example') // (1) { address: '203.0.113.7', family: 4 }
os.availableParallelism() // (2) e.g. 8
path.join('uploads', 'invoices', '2026', 'A12.pdf') // (3) 'uploads/invoices/2026/A12.pdf'
randomUUID() // (4) '3b241101-e2bb-4255-8caf-4136c566a962'
gzipSync(Buffer.from('a'.repeat(10_000))).length // (5) a few dozen bytesdns.lookupuses the operating system's resolver, the same as most clients, and runs on the thread pool (question 19).osfor sizing worker pools and reporting machine details.path.joinuses the right separator,/or\, and cleans up..and duplicate slashes.cryptofor IDs, hashes and more (question 43).zlibcompresses repetitive data very well; for real traffic use its streams.
| Module | You use it for | Watch out for |
|---|---|---|
dns | Resolving names, checking records | lookup uses the thread pool; resolve* query DNS servers directly |
os | CPU count, memory, temp directory, hostname | In containers, os.cpus() may show the host's cores; prefer availableParallelism() |
path | Joining, resolving, splitting paths | Never build paths from user input without checking the result stays inside your folder |
crypto | Hashes, HMAC, random values, encryption, signatures | Use the safe defaults described in question 43 |
zlib | gzip, deflate, Brotli | Synchronous versions block the event loop on big data |
Path traversal, the classic path bug:
ts
const file = path.join('/srv/uploads', req.params.name) // name = '../../etc/passwd'
// becomes /etc/passwdCheck that path.resolve(base, name) still starts with your base folder, or never use user input in paths at all; store files by generated IDs.
43. How does the Node.js crypto module support secure applications?
The short answer
node:crypto provides the building blocks: cryptographically secure random values for tokens and IDs, hash functions such as SHA-256, HMAC for proving a message came from someone who knows a shared secret, slow password hashing with scrypt, authenticated encryption with AES-256-GCM, digital signatures with key pairs, and a constant-time comparison to avoid leaking secrets through timing. Use these high-level, well-known choices; never invent your own scheme.
| Need | Use | Not |
|---|---|---|
| Random token or ID | randomBytes(32), randomUUID() | Math.random() |
| Checksum of a file | createHash('sha256') | MD5, SHA-1 |
| Verify a webhook came from a partner | createHmac('sha256', secret) + timingSafeEqual | === on the strings |
| Store passwords | scrypt with a random salt, or Argon2 or bcrypt libraries | Plain SHA-256 |
| Encrypt data at rest | AES-256-GCM with a random IV | AES-ECB, reusing an IV |
| Sign data | sign and verify with Ed25519 or RSA keys | A shared secret when parties should not share keys |
Verifying a payment webhook:
ts
import { createHmac, timingSafeEqual } from 'node:crypto'
function isValidWebhook(rawBody: Buffer, signatureHex: string, secret: string) {
const expected = createHmac('sha256', secret).update(rawBody).digest() // (1)
const received = Buffer.from(signatureHex, 'hex')
return received.length === expected.length && timingSafeEqual(received, expected) // (2)
}- Recompute the HMAC of the exact raw bytes received, with the shared secret.
- Compare in constant time. A normal comparison stops at the first different byte, so an attacker measuring response times could guess the signature byte by byte.
timingSafeEqualalways takes the same time.
Encrypting a value, such as a stored API key, with AES-256-GCM:
ts
import { randomBytes, createCipheriv, createDecipheriv } from 'node:crypto'
function encrypt(plain: string, key: Buffer) { // key: 32 random bytes from a KMS
const iv = randomBytes(12) // (1)
const cipher = createCipheriv('aes-256-gcm', key, iv)
const data = Buffer.concat([cipher.update(plain, 'utf8'), cipher.final()])
const tag = cipher.getAuthTag() // (2)
return Buffer.concat([iv, tag, data]).toString('base64')
}
function decrypt(encoded: string, key: Buffer) {
const buf = Buffer.from(encoded, 'base64')
const iv = buf.subarray(0, 12), tag = buf.subarray(12, 28), data = buf.subarray(28)
const decipher = createDecipheriv('aes-256-gcm', key, iv)
decipher.setAuthTag(tag) // (3)
return Buffer.concat([decipher.update(data), decipher.final()]).toString('utf8')
}- A fresh random IV, initialisation vector, every time. Reusing an IV with the same key in GCM breaks its security.
- The authentication tag proves the data was not changed.
- Decryption checks the tag, and
final()throws if anyone tampered with the data.
Where the key comes from matters most. Encryption is only as safe as the key's storage: load it from a secret manager or KMS, never from source code. Keys, rotation and password hashing are covered in depth in the security module.
44. What is the difference between binary data and text data in Node.js?
The short answer
Text data is characters, such as JSON, HTML or CSV, stored as bytes using a character encoding, usually UTF-8; to use it as a JavaScript string, the bytes must be decoded. Binary data is bytes that are not text, such as images, PDFs, ZIP files or encrypted data, and must be kept as bytes, in a Buffer. Treating binary data as text, for example by reading an image with 'utf8', corrupts it, because many byte sequences are not valid characters and get replaced.
ts
import { readFile } from 'node:fs/promises'
const csv = await readFile('orders.csv', 'utf8') // (1) a string
const logo = await readFile('logo.png') // (2) a Buffer of raw bytes
const broken = Buffer.from(await readFile('logo.png', 'utf8'), 'utf8') // (3)
broken.equals(logo) // false: corrupted- With an encoding, Node.js decodes bytes into a string.
- Without one, you get the bytes as a
Buffer. - Decoding an image as text and back replaces invalid sequences with the replacement character, so the image is damaged.
| Text | Binary | |
|---|---|---|
| Examples | JSON, CSV, HTML, logs | Images, PDFs, archives, encrypted data |
| In JavaScript | string | Buffer or Uint8Array |
| Needs an encoding to read | Yes, usually UTF-8 | No |
| Safe to send in JSON | Yes | Only after base64 encoding |
A subtle trap with streams: a multi-byte character, such as £, can be split across two chunks. Converting each chunk to a string separately breaks that character. Call stream.setEncoding('utf8'), or use a StringDecoder, which keeps incomplete characters until the next chunk (question 46).
45. What is a Buffer, and why is it needed?
The short answer
A Buffer is a fixed-size piece of raw memory holding bytes, Node.js's way to work with binary data. JavaScript strings can only hold text, but files, network sockets, compression and crypto all deal in bytes, so Node.js needs a type for them. Buffer is a subclass of Uint8Array, its memory lives outside the V8 heap, and you create one with Buffer.from, Buffer.alloc or by reading data. It is what you get from fs.readFile without an encoding and from network data events.
ts
const a = Buffer.from('£19', 'utf8') // (1) <Buffer c2 a3 31 39>
a.length // 4 bytes, but only 3 characters
const b = Buffer.alloc(16) // (2) 16 bytes, all zero
const c = Buffer.allocUnsafe(16) // (3) 16 bytes, NOT cleared
b.writeUInt32BE(1999, 0) // (4) write a number as 4 bytes
b.readUInt32BE(0) // 1999
const header = a.subarray(0, 2) // (5) a view, not a copy- From a string:
£takes two bytes in UTF-8,c2 a3, so the buffer is longer than the string. allocgives zero-filled memory, safe to expose.allocUnsafeis faster but may contain old data from other parts of the process, such as another user's request. Only use it when you immediately overwrite every byte.- Reading and writing numbers in a chosen byte order, used for binary file formats and protocols.
subarrayshares memory with the original: changing one changes the other.
Why outside the heap matters. Buffer memory is not counted in heapUsed but in external and arrayBuffers. A service holding many large uploads as Buffers can run out of memory while its heap looks small (question 31). That is a reason to stream large data instead of buffering it (question 48).
46. How does Buffer encoding and decoding work?
The short answer
Encoding turns text into bytes using a character encoding, and decoding turns bytes back into text. Buffer.from(text, 'utf8') encodes; buf.toString('utf8') decodes. Other encodings represent bytes as text, which is needed to put binary data into JSON, URLs or headers: 'base64' and 'base64url' use letters, digits and a few symbols, and 'hex' writes each byte as two hexadecimal digits. Encoding and decoding must use the same scheme, or the result is garbage.
ts
const bytes = Buffer.from('£19', 'utf8') // <Buffer c2 a3 31 39>
bytes.toString('hex') // 'c2a33139'
bytes.toString('base64') // 'wqMxOQ=='
Buffer.from('wqMxOQ==', 'base64').toString('utf8') // '£19'
Buffer.byteLength('£19') // 4: bytes
'£19'.length // 3: UTF-16 code units, roughly characters| Encoding | Use |
|---|---|
utf8 | Text: the default for almost everything |
base64 | Binary data inside JSON or email, such as an image in a JSON body |
base64url | Binary data in URLs and JWTs: no +, / or = |
hex | Hashes and IDs shown to humans |
latin1 | Old systems that use one byte per character |
Base64 makes data about a third larger, because every 3 bytes become 4 characters. Sending large files as base64 inside JSON wastes bandwidth and memory; use a file upload or a direct upload to object storage instead.
Byte length versus string length matters for limits: a database column limited to 255 bytes can overflow with fewer than 255 characters if they include £, accented letters or emoji. Check limits with Buffer.byteLength(text).