From 752c8ecbd35117f328b69a667777103d75dd74cb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Afonso=20Janu=C3=A1rio?= Date: Sun, 6 Sep 2026 01:25:04 +0100 Subject: [PATCH] Fix scanner counting the leading BOM as a column when encoding is explicit When the input encoding is set explicitly via yaml_parser_set_encoding(), yaml_parser_determine_encoding() (which strips a BOM during auto-detection without touching mark.column) is skipped, so the BOM reaches the scanner instead. yaml_parser_scan_to_next_token() then calls SKIP(), which bumps mark.column along with the buffer position, so the rest of the first line ends up one column further right than it should be. A root-level block mapping terminates its first entry at the wrong spot once the line wraps, and parsing fails with "did not find expected ". Per YAML 1.2 section 5.2 the BOM isn't part of the document content, so skipping it shouldn't move mark.column at all. This mirrors what the auto-detection path already does. Reported in #334, originally surfaced through Ruby's Psych binding, which always sets the encoding explicitly and so hit this on every BOM-prefixed document. --- src/scanner.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/src/scanner.c b/src/scanner.c index e7ffd83f..8923f3c2 100644 --- a/src/scanner.c +++ b/src/scanner.c @@ -1945,8 +1945,18 @@ yaml_parser_scan_to_next_token(yaml_parser_t *parser) if (!CACHE(parser, 1)) return 0; - if (parser->mark.column == 0 && IS_BOM(parser->buffer)) - SKIP(parser); + /* + * The BOM is not a character in the document (YAML 1.2 spec, + * section 5.2), so it must not advance mark.column the way a + * regular SKIP() would - otherwise the rest of the first line + * is off by one column. + */ + + if (parser->mark.column == 0 && IS_BOM(parser->buffer)) { + parser->mark.index ++; + parser->unread --; + parser->buffer.pointer += WIDTH(parser->buffer); + } /* * Eat whitespaces.