URL encoding, HTML entities and MIME types: three things people mix up

A user is called Ben & Co. Putting that name in a URL, on a page, and in a downloaded file all require doing something to the ampersand, and the three somethings are different. Mixing them up produces bugs that look the same from the outside and have unrelated causes.

Percent-encoding is for URL parsers

A URL has a grammar. ? starts the query, & separates parameters, # starts the fragment, / separates path segments. If your data contains one of those characters, the parser will read it as structure.

Percent-encoding replaces the character with % and its byte value in hex. A space becomes %20, an ampersand %26, a slash %2F. Non-ASCII characters are encoded as UTF-8 first, so é becomes %C3%A9, which is why it takes six characters.

The part that trips people is that JavaScript has two functions and they are not interchangeable:

encodeURI('https://x.com/a b?q=1&r=2')
// https://x.com/a%20b?q=1&r=2      structure preserved

encodeURIComponent('Ben & Co')
// Ben%20%26%20Co                   everything escaped

encodeURI is for an entire URL and deliberately leaves ? & / : # alone, because it assumes they are structure. encodeURIComponent is for one value going into one slot and escapes them, because it assumes they are data.

The + character deserves its own warning. In a query string, historically, + means a space. In a path it means a literal plus. So a+b in a query reads as a b, which is why a phone number like +44... must be encoded as %2B44 or it arrives with a leading space.

When a URL is behaving strangely, a URL parser that shows the decoded components separately is usually faster than staring at the string, and an encoder tells you what a value should look like.

Entity encoding is for HTML parsers

An HTML parser has a different grammar with different dangerous characters: < starts a tag, & starts an entity, and inside an attribute the quote character ends the attribute.

Entity encoding replaces these with named or numeric references: & < > " '.

This is the same problem as percent-encoding and a completely separate solution. Percent-encoding text before putting it on a page does not make it safe, and a browser will display %3Cscript%3E as literal text rather than treating it as escaped markup.

The two do stack. A URL inside an HTML attribute needs both, applied in order:

<!-- value: Ben & Co -->
<a href="/search?q=Ben%20%26%20Co">Ben &amp; Co</a>

Percent-encoding for the URL grammar, entity encoding for the HTML grammar, each doing its own job.

Double encoding

If you have ever seen %2520 or & in production, that is double encoding: something encoded a string that was already encoded.

%2520 decodes once to %20 and twice to a space. & renders as & and then as &. The fix is never to decode twice at the end, it is to find the layer doing the extra encode. Usually a framework is already escaping output and application code is escaping it again on the way in.

An HTML entity encoder is useful here precisely because it lets you decode a suspicious string step by step and see how many layers deep it goes.

MIME types are about content, not names

The third confusion is different in kind. A MIME type such as image/png or application/pdf tells a program what a stream of bytes is.

The critical point is that the file extension is not the MIME type. An extension is a naming convention that anyone can change. The real type is determined by the magic bytes at the start of the file: a PNG begins with 89 50 4E 47, a PDF with %PDF-, a ZIP with PK.

This has two practical consequences.

For debugging, a file that will not open is often a file that is not what its name claims. A MIME checker reading the magic bytes will tell you that the .xlsx your system produced is actually an HTML error page, which is a surprisingly common outcome of a failed export.

For security, never trust an uploaded file's extension or its client-supplied Content-Type header. Both are set by whoever is uploading. Check the magic bytes server-side.

The short version

When something arrives garbled, identify which layer is wrong before changing anything: percent signs point to URL encoding, ampersand sequences to HTML entities, and a file that opens in the wrong program to its MIME type.