URL Encode / Decode

The plus-sign problem

A space is %20 in a path and + in an application/x-www-form-urlencoded query string. Both conventions are in active use, and they disagree. encodeURIComponent always produces %20, which is safe in both positions because a literal + in a query value must itself be escaped as %2B.

Where this bites is decoding. decodeURIComponent("a+b") returns a+b, not a b — it does not know it is looking at form encoding. If you are parsing a query string by hand, replace + with a space *before* percent-decoding. URLSearchParams in the browser does this for you correctly, which is one more reason not to parse query strings with split('&').

Non-ASCII, IDNs, and punycode

Percent-encoding operates on bytes, and modern URLs use UTF-8. é is two UTF-8 bytes, so it encodes to %C3%A9; an emoji is four bytes and produces four escapes. If you see %E9 (a single byte) for é, something upstream used Latin-1, and decoding it as UTF-8 will fail.

The hostname is the exception. Percent-encoding is not allowed there — internationalised domain names use punycode instead, so bücher.de becomes xn--bcher-kva.de on the wire. This is a DNS-level encoding, not a URL one, and it is why a non-ASCII domain resolves under a name that looks nothing like what you typed. If a lookup for an accented domain fails, convert it to its xn-- form first and try the DNS lookup with that.

Punycode is also a phishing vector: several Cyrillic letters render identically to Latin ones, so a homograph domain can be visually indistinguishable from a real one while being an entirely different name in DNS. Browsers now display the punycode form when a label mixes scripts, and a WHOIS lookup on the xn-- name will show you who actually registered it.

Double encoding

Encoding an already-encoded string escapes the % itself: %20 becomes %2520. The result decodes back to %20 rather than to a space, so the value arrives one layer short. This is the single most common percent-encoding bug, and it usually comes from a framework that already encoded a parameter before your code encoded it again.

The symptom is a literal %20 or %2F showing up in rendered output, a filename, or a log line. The tell is %25 in the encoded form — that is an escaped %, which only appears legitimately when your data genuinely contained a percent sign. This tool flags it: if a decode still leaves % sequences in the output, it says so, and decoding a second time gives you the original.

Double encoding is also a security concern. A filter that rejects ../ will happily pass %252e%252e%252f, which becomes %2e%2e%2f after one decode and ../ after two. If any layer of your stack decodes more than once, path traversal defences that look at the raw string are worthless. Decode exactly once, then validate.

Command-line equivalents

# Encode a component with curl, then print the URL it built
curl -s -o /dev/null -w '%{url_effective}\n' --get --data-urlencode 'q=a b&c' 'http://x/'
# -> http://x/?q=a+b%26c
# Note the +: --data-urlencode uses form encoding, so a space becomes '+',
# while the & is correctly escaped to %26. Both decode to "a b&c".

# Python one-liners, the most portable option
python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1], safe=''))" 'a b&c'
python3 -c "import urllib.parse,sys; print(urllib.parse.unquote(sys.argv[1]))" 'a%20b%26c'

# jq, if it is already installed
jq -rn --arg v 'a b&c' '$v|@uri'

# Punycode conversion for an internationalised domain
idn2 'bücher.de'          # -> xn--bcher-kva.de
python3 -c "print('bücher.de'.encode('idna').decode())"

# Then look the real name up
dig +short xn--bcher-kva.de A

Note the safe='' argument in the Python example. By default urllib.parse.quote leaves / unescaped — it behaves like full-URL mode. Passing an empty safe set makes it behave like encodeURIComponent. Getting that argument wrong is the Python equivalent of choosing the wrong mode on this page.

Practical rules

• Encode each value separately, then assemble the URL. Never encode the assembled URL and hope for the best.
• Build query strings with URLSearchParams (JS), urlencode (Python), or your framework's equivalent rather than string concatenation. They handle both the escaping and the +/%20 question.
• Never put credentials, tokens, or personal data in a query string at all. URLs land in server access logs, browser history, and Referer headers sent to third parties. Use a header or a request body.
• Encode exactly once, at the boundary where the value enters the URL. Most double-encoding bugs are two layers each doing their job.
• When debugging a redirect chain, decode each Location value before reading it — nested redirect_uri parameters are frequently double-encoded. The redirect checker will follow the chain for you.