Skip to content

Do not emit multiple UTF-8 BOM markers - #2136

Open
arronKler wants to merge 1 commit into
postcss:mainfrom
arronKler:fix/2133-duplicate-bom
Open

Do not emit multiple UTF-8 BOM markers#2136
arronKler wants to merge 1 commit into
postcss:mainfrom
arronKler:fix/2133-duplicate-bom

Conversation

@arronKler

Copy link
Copy Markdown

Why

v8.5.24 started restoring a leading BOM on stringify (#2119). That is correct for a single file, but parse/stringify could still produce more than one U+FEFF:

  • input with several leading BOMs kept extras after the first
  • concatenated CSS (a{}\uFEFFb{}) re-emitted the mid-file marker as part of the next selector
  • a Document of several BOM-tagged roots wrote a BOM before every root

Those extra markers show up in the middle of generated CSS and break the first rule after each one.

Solution

  • Strip every leading U+FEFF / U+FFFE in Input, not only the first
  • Skip stray BOM markers in the tokenizer so they are not part of the AST
  • Emit at most one BOM from Stringifier#root

A single leading BOM is still preserved. parse('\uFEFFa{}').toString() remains \uFEFFa{}.

Fixes #2133

8.5.24 restored a leading BOM on stringify. Concatenated CSS and
documents with several BOM-tagged roots then contained U+FEFF in the
middle of the output.

Strip every leading BOM on parse, skip stray markers while tokenizing,
and emit at most one BOM when stringifying.

Fixes postcss#2133
Comment thread lib/tokenize.js

function nextToken(opts) {
if (returned.length) return returned.pop()
skipBom()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need it if we already have this.css.replace(/^[\uFEFF\uFFFE]+/, '')?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multiple BOM symbols inside generated css in v8.5.24

3 participants