GH-51470: [C++] Delimit JSON documents without parsing them

[taepper]

Rationale for this change

Resolves #51470:

simdjson has a multi-stage parser that first finds structural elements in the entire buffer (including document boundaries). Only the second stage, which is lazy, actually need to parse the documents.

Similarily parse_many already computes all document boundaries and we can iterate its output to consume the documents without parsing any documents:
https://github.com/simdjson/simdjson/blob/6913ee1a17a57d23a2e9f458d8e04e96cd43ecab/doc/parse_many.md?plain=1#L85-L88

What changes are included in this PR?

This delimits JSON documents without fully parsing them

Are these changes tested?

Yes.

before:

ChunkJSONPrettyPrinted                                             528656 ns       528396 ns         1340 
bytes_per_second=394.823Mi/s json_size=218.757k
ChunkJSONPrettyPrintedMultipleBlocks                               664758 ns       625186 ns         1143 block_size=27.344k 
bytes_per_second=333.697Mi/s json_size=218.757k

after:

ChunkJSONPrettyPrinted                                              91815 ns        91811 ns         7358 
bytes_per_second=2.21906Gi/s json_size=218.757k
ChunkJSONPrettyPrintedMultipleBlocks                               197358 ns       167654 ns         3719 block_size=27.344k 
bytes_per_second=1.2152Gi/s json_size=218.757k

Are there any user-facing changes?

No.

Was AI used for this PR?

In accordance to the AI generation guidelines, please disclose below whether and how AI was used in this PR.

PR code and description written by:

• Human
• (AI assisted in code drafting for the initial version)

Reviewed before submission by:

• Human
• AI
• Not reviewed

• GitHub Issue: [C++] Delimit JSON documents without parsing them #51470

[github-actions]

⚠️ GitHub issue #51470 has been automatically assigned in GitHub to PR creator.

[taepper]

If you would have time for review, I think this is a good improvement to the chunker @pitrou (or please @ others that might be a good fit to review this) :)

[pitrou]

This is neat, thank you!