JavaScript · 10 topics

Modules, tooling and the ecosystem

The group people skip and then get caught by. What actually happens when you run npm install, and why your package broke after an upgrade.

Published
Reading time
19 min read
Topics covered
10 topics

This is the group people skip, and then get caught by. Nobody fails an interview on map. Plenty of people stall on "what actually happens when you run npm install?" or "why does this package break when you import it?".

It is also the most practical group here. Everything below is something you will debug in a real job.

CommonJS and ES modules

In simple words: two different systems for splitting code into files. CommonJS came from Node and loads files while the program runs. ES modules are the standard and are analysed before anything runs.

That one difference explains everything else.

// CommonJS -- Node's original system
const express = require("express");
module.exports = { createServer };
 
// ES modules -- the standard
import express from "express";
export { createServer };
COMMONJS — RESOLVED WHILE RUNNINGrun line 1hit requireload + run that filecarry onthe graph is only known once execution reaches each requireES MODULES — RESOLVED BEFORE RUNNINGparse all importsbuild whole graphlinkthen executethe full graph is known before any code runs — this is what enables tree shaking
CommonJS resolves imports while executing, so a require can be conditional. ES modules are resolved before execution, which is what makes tree shaking and top-level await possible.
CommonJSES modules
Resolvedwhile runningbefore running
Can be conditionalyes, require inside an ifno, import must be top level
What you geta copy of the value at that momenta live binding to the original
this at top levelmodule.exportsundefined
Filename helpers__dirname, __filenameimport.meta.url
Top-level awaitnoyes
Tree shakeablenoyes

Live bindings is the difference people cannot explain. With ESM, the importer sees updates to an exported variable. With CommonJS it got a copy and never sees the change.

// counter.mjs
export let count = 0;
export function increment() { count++; }
 
// main.mjs
import { count, increment } from "./counter.mjs";
console.log(count);   // 0
increment();
console.log(count);   // 1  -- live binding, updated
 
// the same thing in CommonJS would still print 0

Circular dependencies

Both handle cycles, badly, in different ways.

  • CommonJS gives you a partially filled module.exports — whatever had been assigned before the cycle closed. You get undefined for anything not yet set, with no error.
  • ESM hoists function declarations, so functions usually work, but a const accessed too early throws a ReferenceError from the temporal dead zone.

The ESM failure is louder, which is better. Either way the real fix is to break the cycle — usually by extracting the shared piece into a third module.

Dynamic import and top-level await

In simple words: import() as a function loads a module on demand and returns a promise. This is how code splitting works.

// only downloads the chart library when the user actually opens the chart
button.addEventListener("click", async () => {
  const { renderChart } = await import("./heavy-chart.js");
  renderChart(data);
});
 
// conditional loading, which a static import cannot do
const locale = navigator.language.startsWith("hi") ? "hi" : "en";
const messages = await import(`./locales/${locale}.js`);

React.lazy is a thin wrapper over exactly this.

Top-level await lets a module await before it finishes loading. Anything importing it waits.

// config.mjs
const response = await fetch("/config.json");
export const config = await response.json();

Useful, but it blocks every importer. Do not put a slow call at the top level of a widely imported module.

npm, lockfiles, and the ecosystem

In simple words: package.json says what you want. package-lock.json records exactly what you got. Without the lock, two machines can install different code from the same file.

FieldMeaning
dependenciesneeded at runtime, shipped to production
devDependenciesonly needed to build and test
peerDependencies"you must provide this yourself" — how plugins avoid duplicate React
optionalDependenciesinstall failure is not fatal — platform-specific binaries

install versus ci

npm install     # may update the lockfile, resolves ranges again
npm ci          # installs exactly the lockfile, deletes node_modules first, fails if
                # package.json and the lock disagree

Use npm ci in CI and Docker builds, always. It is faster and it guarantees the same tree every time. npm install in a pipeline can silently pull a new patch version and change your build.

The three package managers

  • npm — bundled with Node, nested-then-flattened node_modules
  • yarn — arrived first with lockfiles and workspaces; Yarn 2+ can skip node_modules entirely with Plug'n'Play
  • pnpm — one global content-addressed store, hard-linked into each project. Much less disk, much faster, and strict: a package you did not declare is not importable

That strictness is pnpm's real selling point. With npm's flat node_modules, you can accidentally import a transitive dependency and it works — until it is removed upstream and your build breaks for no visible reason.

Semver

In simple words: MAJOR.MINOR.PATCH. Major means breaking, minor means new features that do not break, patch means fixes.

RangeMeansExample: from 1.2.3
^1.2.3patch and minor updatesup to but not including 2.0.0
~1.2.3patch updates onlyup to but not including 1.3.0
1.2.3exactly thisonly 1.2.3
>=1.2.3 <2explicit range
*anythingnever do this

^ is npm's default and is the right choice most of the time. The catch: it trusts every maintainer to follow semver correctly, and plenty do not. That is exactly what the lockfile protects you from.

Bundlers

In simple words: a bundler follows your imports, produces a dependency graph, and outputs files a browser can load efficiently.

What it isWhere it fits
Webpackthe mature, configurable onelarge legacy apps, unusual requirements
Vitedev server on native ESM, Rollup for productionthe default for new projects
Rolluplibrary-focused, cleanest outputpublishing packages
esbuildwritten in Go, extremely fastpowers other tools' transform step
Turbopack / RspackRust rewrites of the Webpack idealarge apps wanting Webpack semantics, faster

Why Vite feels instant in development: it does not bundle. Your browser requests modules over native ESM, and Vite transforms each file on demand. Adding files barely changes startup time. For production it does bundle with Rollup, because hundreds of separate requests would be slow over a network.

Tree shaking

In simple words: removing code you imported but never used. It only works because ESM imports are statically analysable.

import { formatDate } from "./utils.js";   // shakeable -- only this is kept
const { formatDate } = require("./utils"); // not shakeable -- dynamic

Three things break it:

  1. CommonJS. The bundler cannot know what is used.
  2. Side effects. If importing a file does something — registers a polyfill, mutates a global — the bundler must keep it.
  3. Barrel files. A big index.js re-exporting everything can pull in far more than you asked for.

Declare that your package is side-effect free so bundlers can be aggressive:

{
  "sideEffects": false,
  "sideEffects": ["*.css", "./src/polyfills.js"]
}

The exports field

Modern packages declare their entry points explicitly:

{
  "main": "./dist/index.cjs",
  "module": "./dist/index.mjs",
  "types": "./dist/index.d.ts",
  "exports": {
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/index.mjs",
      "require": "./dist/index.cjs"
    },
    "./styles.css": "./dist/styles.css"
  }
}

exports also seals the package: anything not listed cannot be imported. That is why deep imports like some-lib/dist/internal/thing suddenly stop working after an upgrade.

Babel, transpiling and polyfilling

In simple words: two different problems that people conflate.

  • Transpiling rewrites new syntax into older syntax. Arrow functions become function. Optional chaining becomes an if.
  • Polyfilling adds missing functions at runtime. Array.prototype.flat or Promise are added if absent.

Syntax cannot be polyfilled — old parsers throw on it before any code runs. Functions cannot be transpiled — there is nothing to rewrite, the method simply does not exist.

// .browserslistrc drives both. Tools read it and only include what is needed.
{
  "browserslist": ["> 0.5%", "last 2 versions", "not dead"]
}

With @babel/preset-env and useBuiltIns: "usage", Babel inspects your code and injects only the polyfills your targets actually lack. Widening your browser support list makes the bundle bigger — that is the trade.

Source maps

In simple words: a file mapping your minified output back to the original source, so a stack trace points at real line numbers.

//# sourceMappingURL=app.js.map
SettingTrade-off
eval-cheap-module-source-mapfast rebuilds, good enough for development
source-mapfull accuracy, slow — production
hidden-source-mapgenerated, but not referenced in the bundle
nonefastest, unreadable stack traces

For production, the usual approach is to generate full source maps and upload them to your error tracker (Sentry, Bugsnag) rather than serve them publicly — you get readable traces without publishing your source.

Monorepos

In simple words: several packages in one repository, sharing dependencies and tooling.

{ "workspaces": ["packages/*", "apps/*"] }

Workspaces symlink local packages into each other's node_modules, so apps/web can import { Button } from "@acme/ui" and get your local source with no publishing step.

Turborepo and Nx add the part that matters at scale: a task graph with caching. Change one package and only its dependents rebuild; unchanged results come from cache, locally or shared with the team.

You do not need deep knowledge here. Know what a workspace is, why symlinking beats copying, and that the caching layer is what makes a large monorepo bearable.

Publishing a package

If asked how you would ship a library:

  1. Build both formats — ESM for bundlers, CommonJS for older Node
  2. Emit type declarations with tsc --emitDeclarationOnly
  3. Wire up exports, main, module and types
  4. Set "sideEffects": false if true
  5. Restrict what ships with the files field
  6. npm publish --access public

The single most common mistake is shipping only ESM. Any consumer still on CommonJS then cannot require your package at all.

Ten questions

  1. 01
    What is the fundamental difference between CommonJS and ES modules?
  2. 02
    What is a live binding, and how does it differ from what CommonJS gives you?
  3. 03
    What is the difference between npm install and npm ci, and which belongs in CI?
  4. 04
    What does ^1.2.3 allow, and how does it change below version 1.0.0?
  5. 05
    Why are peerDependencies used, and what problem do they solve?
  6. 06
    Why is Vite so much faster than Webpack in development?
  7. 07
    What three things stop tree shaking from working?
  8. 08
    What is the difference between transpiling and polyfilling?
  9. 09
    Should production source maps be served publicly?
  10. 10
    You upgrade a package and a deep import like lib/dist/internal/x breaks. Why?

Six exercises

  1. 01

    Prove live bindings exist

    Write two ESM files where one exports a mutable counter and a function that increments it. Import both and show the value changes for the importer. Then write the CommonJS equivalent and show it does not.

  2. 02

    Break a circular dependency

    Create a genuine cycle in CommonJS and observe the partially filled exports. Recreate it in ESM and observe the temporal dead zone error instead. Then fix it by extracting the shared piece into a third module.

  3. 03

    Read your own lockfile

    Pick a dependency in a real project and trace it through package-lock.json: which version resolved, why that one, and which packages depend on it. Then run npm ls for the same package and compare.

  4. 04

    Measure code splitting

    Take any page that imports a heavy library. Build and note the bundle size. Convert the import to a dynamic import() behind a user action, rebuild, and record the difference in the initial bundle.

  5. 05

    Publish a dual-format package

    Build a tiny utility to both ESM and CommonJS with type declarations, wire up the exports field, and verify with npm pack that only the files you intended are included. Test both import and require.

  6. 06

    See browserslist change the output

    In a project using preset-env, build once targeting modern browsers only, then again including an old target. Diff the output size and find one specific transform or polyfill that the wider target added.

Common questions

What is the difference between CommonJS and ES modules?
CommonJS resolves imports while the program runs, so a require can be conditional. ES modules are parsed and linked before any code executes. Everything else follows from that: ESM enables tree shaking and top-level await, and gives live bindings instead of copies, while requiring imports to sit at the top level.
What is the difference between npm install and npm ci?
npm install re-resolves version ranges and may update the lockfile. npm ci deletes node_modules, installs exactly what the lockfile specifies, and fails if the lockfile and package.json disagree. Always use npm ci in CI and Docker builds, because it is faster and guarantees an identical dependency tree.
What does the caret in ^1.2.3 allow?
It allows any patch and minor update below the next major version, so anything from 1.2.3 up to but excluding 2.0.0. Below version 1.0.0 the rule tightens: ^0.2.3 only allows patch updates, because in 0.x the minor number is treated as potentially breaking.
Why does tree shaking sometimes not remove unused code?
Three common reasons: the module uses CommonJS, so the bundler cannot statically determine what is used; the module has side effects at import time, so it must be kept; or a barrel file re-exports everything and pulls in more than you imported. The sideEffects field in package.json tells bundlers when it is safe to be aggressive.
What is the difference between transpiling and polyfilling?
Transpiling rewrites new syntax into older syntax, such as turning optional chaining into a conditional. Polyfilling adds missing runtime functions such as Array.prototype.flat. They are not interchangeable, because syntax cannot be polyfilled - old parsers fail before execution - and a missing method cannot be transpiled.