~/blog

encodeURIComponent throws on half an emoji; URLSearchParams does not

published

#javascript#unicode#urls#node

TL;DR

encodeURIComponent and encodeURI throw URIError: URI malformed when the string contains a lone surrogate, which is what you get when you slice a string through the middle of an emoji. URL, URLSearchParams, TextEncoder and Node’s Buffer.from accept the same string and silently replace the broken half with U+FFFD. Fix the truncation that produced the lone surrogate, then decide explicitly: reject with isWellFormed() or replace with toWellFormed().

The problem

A share link built from a truncated preview:

const title = 'Launch day 👍 thanks everyone';
const preview = title.slice(0, 12);
console.log(JSON.stringify(preview)); // "Launch day \ud83d"

const href = `https://example.com/share?text=${encodeURIComponent(preview)}`;
// URIError: URI malformed

👍 is two UTF-16 code units, and index 12 falls between them. slice does not care, so preview ends with \ud83d, the first half of the pair. The same input through the URL API does not throw at all:

function shareUrl(text) {
  const url = new URL('https://example.com/share');
  url.searchParams.set('text', text);
  return url.href;
}

console.log(shareUrl(preview)); // https://example.com/share?text=Launch+day+%EF%BF%BD

%EF%BF%BD is the UTF-8 encoding of U+FFFD, the replacement character. One API crashed the request; the other shipped a link with a black diamond in it. Neither is what you meant.

Why it happens

The two families follow two different specs.

encodeURI and encodeURIComponent are ECMAScript. Their shared Encode operation reads the string one code point at a time and says: “If codePoint.[[IsUnpairedSurrogate]] is true, throw a URIError exception.” A lone surrogate has no UTF-8 encoding, so the function refuses.

URL, URLSearchParams, TextEncoder and Request are web APIs whose string arguments are declared as USVString in Web IDL. Converting a JavaScript string to a USVString means “converting string to a sequence of scalar values”, and that conversion swaps every lone surrogate for U+FFFD before the API ever sees it. They cannot throw on this input because the input has already been changed.

Results on Node.js v26.2.0, with lone = '\uD83D':

CallResult
encodeURIComponent(lone)URIError: URI malformed
encodeURI(lone)URIError: URI malformed
new URLSearchParams({ q: lone }).toString()q=%EF%BF%BD
new URL('https://e.test/?q=' + lone).search?q=%EF%BF%BD
new Request('https://e.test/?q=' + lone).urlhttps://e.test/?q=%EF%BF%BD
new TextEncoder().encode(lone)bytes ef bf bd
Buffer.from(lone).toString('hex')efbfbd
JSON.stringify(lone)"\ud83d"
escape(lone)%uD83D

The trailing half throws too: encodeURIComponent('\uDC4D') is the same URIError.

Where lone surrogates come from

Truncation is the common one, but not the only one:

const fromJson = JSON.parse('{"name":"\\ud83d"}').name;
console.log(fromJson.isWellFormed()); // false

const fromCode = String.fromCharCode(0xd83d);
console.log(fromCode.isWellFormed()); // false

JSON allows \ud83d as an escape, and JSON.stringify writes a lone surrogate back out in that same escaped form, so a broken string survives a round trip through a database or an API intact. It can be stored on Monday and crash an encodeURIComponent call on Thursday, in code that never truncated anything.

What to do

1. Stop producing them. If the lone surrogate came from your own truncation, fix the truncation. Slicing by code point is enough to keep pairs together:

const safe = Array.from(title).slice(0, 12).join('');
console.log(safe); // Launch day 👍
console.log(encodeURIComponent(safe)); // Launch%20day%20%F0%9F%91%8D

For user-visible limits, slice on grapheme clusters with Intl.Segmenter instead, so a family emoji or a flag is not split into its parts. The string length post covers why .length and Array.from both miscount those.

2. Reject at the boundary when the data matters. isWellFormed() is the check:

function requireWellFormed(s) {
  if (!s.isWellFormed()) throw new RangeError('text contains an unpaired surrogate');
  return s;
}

Use it where a silently replaced character would be a bug: identifiers, names, anything you will compare or store.

3. Replace on purpose when it does not. toWellFormed() performs the same U+FFFD substitution the web APIs do, but where you can see it:

const href = `https://example.com/share?text=${encodeURIComponent(preview.toWellFormed())}`;
console.log(href); // https://example.com/share?text=Launch%20day%20%EF%BF%BD

MDN’s encodeURIComponent page recommends exactly this pair: isWellFormed() to check, toWellFormed() to repair.

4. Or build query strings with URLSearchParams. It never throws on this input, and it is the better tool for queries anyway. Note it encodes spaces as + rather than %20; the plus sign post explains why both are valid. Just remember that “never throws” means “replaces without telling you”, which is option 3 without the choice.

Caveats

References