Appearance
Files and Uploads
55. How do you read, write, and append files asynchronously?
The short answer
Use the promise API in node:fs/promises: await readFile(path, 'utf8') to read, await writeFile(path, data) to create or replace, and await appendFile(path, data) to add to the end. For large files or many small writes, use streams or an open file handle instead. To avoid leaving a half-written file if the process crashes, write to a temporary file in the same folder and then rename it over the original, which is atomic on the same file system.
ts
import { readFile, writeFile, appendFile, rename, open } from 'node:fs/promises'
const settings = JSON.parse(await readFile('settings.json', 'utf8')) // (1)
await appendFile('audit.log', `${new Date().toISOString()} refund A12\n`) // (2)
async function saveSettingsSafely(data: object) {
const tmp = 'settings.json.tmp'
await writeFile(tmp, JSON.stringify(data, null, 2)) // (3)
await rename(tmp, 'settings.json') // (4)
}
const file = await open('report.csv', 'w') // (5)
try {
for (const row of rows) await file.write(`${row.id},${row.total}\n`)
} finally {
await file.close() // (6)
}- Read the whole file as a string. Fine for small files.
- Append creates the file if needed and adds to its end.
- Write to a temporary file first.
- Rename over the real file. Readers see either the old file or the new one, never a half-written one.
- An open file handle for many writes, instead of reopening the file each time.
- Always close the handle, in
finally, or file descriptors leak (question 56).
Errors you should handle:
| Code | Meaning |
|---|---|
ENOENT | The file or folder does not exist |
EACCES, EPERM | No permission |
EEXIST | The file already exists, with the wx flag |
EMFILE | Too many open files (question 56) |
ENOSPC | The disk is full |
In containers, local disk is temporary. Files written inside a container are lost when it is replaced, and other replicas cannot see them. Keep uploads and generated files in object storage, such as S3 or Azure Blob Storage, not on the container's disk.
56. What are file descriptors?
The short answer
A file descriptor is a small number the operating system gives a process for each open file, socket or pipe, which the process uses to refer to it in later read, write and close calls. Every open file and every network connection uses one. A process has a limit, often 1,024 by default on Linux, and if you open things without closing them, or simply handle many connections at once, you hit it and get EMFILE: too many open files.
ts
import { open } from 'node:fs/promises'
const handle = await open('invoice.pdf', 'r')
console.log(handle.fd) // e.g. 23: the descriptor number
await handle.close() // gives the number back to the OSWhere descriptors leak:
| Leak | Fix |
|---|---|
open without close on error paths | try/finally, or await using in newer Node.js |
| Streams that error and are never destroyed | pipeline, which destroys every stream on error (question 52) |
| Outgoing HTTP connections never closed | A shared agent with keep-alive and limits |
| Reading thousands of files in parallel | A concurrency limit (question 99) |
Checking the limit and usage on Linux:
bash
ulimit -n # the current per-process limit
ls /proc/<pid>/fd | wc -l # how many this process has openA busy server with many sockets may need a higher limit, set in the service manager or container runtime, but a steadily rising count under steady traffic is a leak.
57. How can you watch for file-system changes?
The short answer
fs.watch(path, listener) asks the operating system to notify you when a file or folder changes, using inotify on Linux, FSEvents on macOS and ReadDirectoryChangesW on Windows, so it is efficient but its events differ between platforms and can arrive twice. fs.watchFile polls instead, which is slower but works on network drives. Most tools use a library such as chokidar, which smooths over the differences. For restarting your own app during development, node --watch is built in.
ts
import { watch } from 'node:fs/promises'
const ac = new AbortController()
for await (const event of watch('./imports', { signal: ac.signal })) { // (1)
if (event.eventType === 'rename' && event.filename?.endsWith('.csv')) {
enqueueImport(`./imports/${event.filename}`) // (2)
}
}- An async iterator of change events, stopped with an abort signal.
- A new CSV appeared: queue it for processing.
Things that go wrong with file watching:
- Events fire more than once, or before the file is completely written. Wait until the size stops changing, or have the producer write to a temporary name and rename when done.
- Watching does not work reliably on network file systems and some container volumes.
- In production, a folder of incoming files is usually better replaced by object storage events, such as "a file was uploaded to this bucket", delivered through a queue.
bash
node --watch src/server.js # restart when files change, during development58. How do you handle file uploads in Node.js?
The short answer
Browsers send files as multipart/form-data, a request body made of parts separated by boundaries. Node.js does not parse that format itself, so you use a parser: Multer with Express, Busboy, which Multer is built on, or a framework's built-in support. The parser streams each file to memory, disk or directly to object storage, and gives you the other form fields. Always set limits on file size and count, check the real file type, and store files under generated names, never the name the user sent.
ts
import multer from 'multer'
import { randomUUID } from 'node:crypto'
import { fileTypeFromBuffer } from 'file-type'
const upload = multer({
storage: multer.memoryStorage(), // (1)
limits: { fileSize: 5 * 1024 * 1024, files: 1 }, // (2)
})
app.post('/products/:id/image', requireAdmin, upload.single('image'), async (req, res) => {
if (!req.file) return res.status(400).json({ error: 'image is required' })
const type = await fileTypeFromBuffer(req.file.buffer) // (3)
if (!type || !['image/jpeg', 'image/png', 'image/webp'].includes(type.mime)) {
return res.status(415).json({ error: 'only JPEG, PNG or WebP images' })
}
const key = `products/${req.params.id}/${randomUUID()}.${type.ext}` // (4)
await storage.put(key, req.file.buffer, { contentType: type.mime }) // (5)
res.status(201).json({ key })
})- Keep the file in memory, fine only because the limit below is small (question 60).
- Limits: Multer stops reading and returns an error once the file passes 5 MB, so nobody can fill memory with a huge upload.
- Check the file's real type from its first bytes, its "magic number". The
Content-Typeand the file extension are chosen by the client and can lie. - A generated name. The user's file name can contain
../or strange characters. - Store it in object storage, not on the server's disk (question 55).
The security checklist for uploads:
| Risk | Defence |
|---|---|
| Huge files filling memory or disk | Size limits in Multer and at the proxy |
| A script uploaded as an "image" | Check magic bytes; serve uploads from a separate domain with Content-Disposition |
| Path traversal through file names | Generated names, never user names |
| Malware shared with other users | Virus scanning before making files available |
| Image parser exploits, huge dimensions | Re-encode images with a library such as sharp, with pixel limits |
Uploads are covered from the security side in the web security module.
59. What is Multer, and when would you use it?
The short answer
Multer is Express middleware for multipart/form-data, the format used for file uploads. It parses the request body, puts the text fields in req.body, and handles files according to a storage engine you choose: in memory as a Buffer, on local disk, or through a custom engine that streams to object storage. It gives you upload.single, upload.array and upload.fields for different form layouts, limits on size and count, and a fileFilter to reject files early. Use it for uploads that pass through your Express server.
ts
const upload = multer({
storage: multer.diskStorage({
destination: '/tmp/uploads',
filename: (_req, _file, cb) => cb(null, randomUUID()), // (1)
}),
limits: { fileSize: 20 * 1024 * 1024, files: 5 },
fileFilter: (_req, file, cb) => cb(null, file.mimetype === 'application/pdf'), // (2)
})
app.post('/claims', requireUser, upload.array('documents', 5), async (req, res) => { // (3)
const files = req.files as Express.Multer.File[]
await claims.create({ userId: req.user.id, reason: req.body.reason, files: files.map((f) => f.path) })
res.status(201).end()
})
app.use((err, _req, res, next) => { // (4)
if (err instanceof multer.MulterError) return res.status(413).json({ error: err.code })
next(err)
})- Generated file names on disk.
- An early filter on the declared type, to reject obvious mistakes. It is not a security check, because the client controls this value (question 58).
- Up to five files from the
documentsfield. - Turn Multer's limit errors into a clear
413 Payload Too Large.
Keep it updated. Multer 2 was released in 2025 to fix denial-of-service vulnerabilities in how malformed uploads were handled; older 1.x versions should not be used.
When not to use it: very large files or high upload volumes, where sending the bytes through your server at all is wasteful. Use direct uploads to object storage instead (question 61).
60. What are the differences between memory storage and disk storage for uploads?
The short answer
Memory storage keeps each uploaded file as a Buffer in the process's memory: it is simple and fast, and convenient when you immediately process the file or send it to object storage, but every concurrent upload uses its full size in RAM. Disk storage streams each file to a local file as it arrives, so memory stays small even for large files, but you must clean up the temporary files, have enough disk, and remember that container disks are temporary and not shared between replicas.
| Memory storage | Disk storage | |
|---|---|---|
| Where the file goes | A Buffer in RAM | A temporary file |
| Memory use | Full file size per upload | Small |
| Good for | Small files processed at once: avatars, CSVs under a few MB | Larger files processed afterwards |
| Clean-up | Automatic when the Buffer is released | You must delete temp files |
| Risk | Many concurrent uploads exhaust memory | Disk fills; files vanish with the container |
The arithmetic that decides it: 50 concurrent uploads of 20 MB each with memory storage is 1 GB of RAM in one process. With a 5 MB limit, the same load is 250 MB, which may be fine.
Often the best answer is neither: stream the upload straight to object storage, or let the browser upload directly to it (question 61).
61. How would you safely support very large file uploads?
The short answer
Do not pass large files through your API servers at all. Have the API create a short-lived, pre-signed upload URL for object storage, scoped to one object, a size and a content type, and let the browser upload directly to the bucket, in parts with multipart uploads so a failed part can be retried and uploads can resume. The API then records the upload, and a background job checks the file, scans it and processes it. Set size limits everywhere, and expire unfinished uploads.
The API side, creating a pre-signed URL for AWS S3:
ts
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3'
import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
const s3 = new S3Client({})
app.post('/uploads', requireUser, async (req, res) => {
const { size, contentType } = UploadRequest.parse(req.body) // (1)
if (size > 5 * 1024 ** 3) return res.status(413).json({ error: 'max 5 GB' })
const key = `uploads/${req.user.id}/${randomUUID()}` // (2)
const url = await getSignedUrl(
s3,
new PutObjectCommand({ Bucket: 'shop-uploads', Key: key, ContentType: contentType, ContentLength: size }),
{ expiresIn: 900 }, // (3)
)
await uploads.create({ key, userId: req.user.id, status: 'pending' }) // (4)
res.json({ url, key })
})- Validate the request: declared size and type.
- A key per user and upload, generated by the server.
- The URL works for 15 minutes, and only for this one object with this length and type.
- Record it as pending until it is confirmed and checked.
For files over a few hundred megabytes, use multipart uploads, where each part has its own signed URL, so a failed 10 MB part is retried instead of the whole file.
Safety around it:
| Concern | Defence |
|---|---|
| Malware | Scan after upload, before anyone else can download it |
| Wrong type | Check magic bytes in the background job |
| Abandoned uploads | Lifecycle rules that delete incomplete uploads after a day |
| Unlimited spending | Per-user quotas, size limits in the signed request |
| Public exposure | Private bucket; download through short-lived signed URLs |
If uploads must go through your server, stream them, req piped to the storage client with pipeline, with size limits at the reverse proxy, such as Nginx's client_max_body_size, and in the parser, and generous but finite timeouts.