From c8e65a18fc07ee1c660a187feff68502b6f611a5 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 2 Sep 2026 06:19:22 +0000 Subject: [PATCH 1/3] spec(MODEL-MM-QWEN4-EXP): a value tap cannot see a routing flip, so tap the SELECTION Wave PREFILLDIV left a second, independent source of CPU/CUDA prefill divergence at decoder layer 0: with the Gated DeltaNet source removed the MoE block still turns a 2.1e-05 input difference into a 7.269e-05 output difference. Nothing measured so far can say whether that is an expert-selection flip or re-association inside the expert GEMM, because `VT_Q4EXP_LAYER_FP` taps values and a discrete selection has bimodal error rather than a tolerance. This is the spec for the instrument that can tell them apart: a selected-expert-id tap that prints the sorted top-k set per token, an FNV-1a hash of it so set equality between two arms is one string comparison per layer, and the boundary margin in bf16 ULPS rather than in probability space, because the softmax denominator is itself a device-order reduction while the bf16 logits are what the selection is a function of. Refs #2552, #2547. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code] --- .agents/specs/qwen4-exp-flash-next.md | 102 ++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/.agents/specs/qwen4-exp-flash-next.md b/.agents/specs/qwen4-exp-flash-next.md index de64f8721..932a71648 100644 --- a/.agents/specs/qwen4-exp-flash-next.md +++ b/.agents/specs/qwen4-exp-flash-next.md @@ -8304,6 +8304,108 @@ No speed number: every arm is n=1 on a shared box. UD-IQ1_S only, one sequence, attributes to the `step == nullptr` branch of `BuildCompletionLogProbs`; nothing here reads them. +## Wave MOEDIV — is the MoE prefill divergence a SELECTION FLIP? (#2552) + +### The question, and why the existing instrument cannot answer it + +Wave PREFILLDIV ([#2547](https://github.com/mudler/vllm.cpp/issues/2547), +[#2554](https://github.com/mudler/vllm.cpp/pull/2554)) isolated TWO sources of +CPU/CUDA prefill divergence at decoder layer 0, the only layer whose input is +bit-identical on both CUDA arms. The first is the chunked Gated DeltaNet +prefill, and it is not a defect: it mirrors vLLM's vendored Flash-Linear-Attention +precision map at all five anchors. The second is the MoE block, which turns a +`2.1e-05` input difference into a `7.269e-05` output difference and which +`VT_GDN_CHUNKED=0` moves by only 1.7x, against 332x on the Gated DeltaNet tap. +[#2552](https://github.com/mudler/vllm.cpp/issues/2552) owns that second source. + +`VT_Q4EXP_LAYER_FP` taps VALUES. **A value tap cannot separate an +expert-selection flip from re-association inside the expert GEMM**, because a +discrete selection has bimodal error and not a tolerance: one token's top-k set +either changes, and that token's MoE output changes by an O(1) amount, or it +does not change at all and the residue is rounding. Averaging both into one +`rel(sum|x|)` destroys exactly the bit that decides which of the two happened. + +### What this wave lands + +`VT_MOE_SEL_FP=` — a **selected-expert-id tap** on the reference arm of +the shared sparse-MoE block (`MoeBlock`, `qwen3_5.cpp`), which is the arm +`RunQwen4ExpMoeBlock` reaches on a stacked keep-quant checkpoint. For the first +`` block invocations of the process it prints, per token: + +* the selected expert ids **SORTED**, because the assertion between two arms is + SET equality and the selection ORDER is not part of it (`vt::MoeCombine` sums + over the k slots and is order-invariant given the weights); +* `lo`, the smallest selected router logit, and `hi`, the largest rejected one, + each with its raw bf16 bit pattern; +* `margin = lo - hi` in **LOGIT** space rather than probability space, because + the softmax denominator is itself a device-order f32 reduction and differs + between the arms, while the bf16 logits are the values the selection is + actually a function of; +* `ulps`, the number of representable **bf16** steps between `hi` and `lo` + under the sign-magnitude total order. `ulps=0` means the two logits are the + SAME bf16 value and the boundary was decided by the lowest-index tie-break — + a knife edge that one ulp of re-association anywhere upstream will flip. + `ulps` is the honest margin for a bimodal error: a probability difference + reads as "small" for a gap of one representable step and for a gap of fifty. + +and, per call, one digest line carrying `sel` (an FNV-1a hash over every token's +sorted id list, so **selection-set equality between two arms is one string +comparison per layer**), the minimum `ulps` over the call's tokens, and four +`sum|x|` axes that decompose the block's own output: `x` (the block input), +`logit` (the router logits), `exp` (the assembled per-slot expert outputs before +the combine) and `shr` (the shared expert). With the selection sets equal, those +four say WHICH of the block's GEMMs carries the residue; with them unequal, the +per-token lines say which token flipped and how near the tie was. + +### The counted property + +`lines=` on every digest line is the running total of value lines the tap has +printed. **An instrument that never ran and two arms whose taps agreed look the +same in a diff**, and this row has already paid for that twice (a job that +searched `$BLD/vllm-server` while ninja links `examples/vllm-server`, and a +doctest case whose name contained a comma so `-tc` split it). The measuring job +asserts `lines == calls * T` and refuses to report a comparison otherwise. + +### Scope, and what this wave does NOT do + +It is a diagnostic tap and one measurement. It changes no math on any arm: the +tap reads host buffers the reference path had already downloaded (`h`, `logits`, +`ids`, `expert_out`) plus one guarded download of the shared-expert output, and +it is inert with the environment variable unset. It lands on the reference arm +only. The three fused CUDA arms (`MoeBlockFusedCuda`, `MoeBlockFusedMarlinCuda`, +`MoeBlockBf16Cuda`) keep their ids on device and are NOT tapped, which is +recorded rather than hidden: no `qwen4_exp` checkpoint reaches them, and adding +a device readback to a capturable path to instrument a model that cannot enter +it would be dead code by construction. + +### Gates + +* `ctest -R test_qwen4_exp_moe` at rc 0, with the tap's ids and `ulps` gated + against `MoeReference`, the suite's independent double-precision oracle + reimplementation, on both the bf16 and the keep-quant arm. +* The reachability mutation deletes the tap's production call site inside + `MoeBlock` and the tap case must RED. +* `scripts/agent-preflight.sh --fail-on-skip` at rc 0. + +### Evidence required + +One `rc` lease on `thor:gpu0`, one binary, the released +`unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S artifact (shard 1 sha256 +`88a1420825a9304063e882ada29d438263617f51ac8923d438d927496693bafd`), the same +prompt and server flags PREFILLDIV used, and three arms: `--device cpu` as the +control, `--device cuda`, and `--device cuda` with `VT_GDN_CHUNKED=0` so the +MoE residue is read with the first source removed. The result is the per-layer +`sel` hash comparison plus the `ulps` distribution. + +### Stop conditions + +Stop and report `PENDING` if the artifact cannot be staged; do not substitute +another checkpoint. Stop if the answer is that the CUDA arm mirrors vLLM and our +CPU arm is the outlier, exactly as PREFILLDIV concluded for the Gated DeltaNet: +say so with the upstream anchors and close #2552 as answered rather than fixed. +**Do not force a fix.** Changing a CUDA arm to agree with our CPU arm, when the +CUDA arm is the one that mirrors the oracle, moves this model AWAY from vLLM. + ## Now `ACTIVE`. **THE COUNT IS THE TABLE, AND THIS SENTENCE NO LONGER RESTATES IT.** From 8e2ba6931dd5aae33955ffdd8e7258ff5d90eef0 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 2 Sep 2026 06:44:19 +0000 Subject: [PATCH 2/3] diag(MODEL-MM-QWEN4-EXP): tap the SELECTED EXPERT IDS, because a value tap cannot see a routing flip Wave PREFILLDIV left a second source of CPU/CUDA prefill divergence at decoder layer 0 and could not classify it: the MoE block turns a 2.1e-05 input difference into a 7.269e-05 output difference with the Gated DeltaNet source removed, and a `rel(sum|x|)` cannot say whether an expert-selection flipped or the expert GEMM re-associated. A discrete selection has bimodal error rather than a tolerance, so averaging the two into one number destroys exactly the bit that decides. `VT_MOE_SEL_FP=` prints the selection instead, on the reference arm of the shared sparse-MoE block, which is the arm a stacked keep-quant checkpoint takes. Per token: the selected ids SORTED, since the assertion between two arms is set equality and `vt::MoeCombine` is order-invariant; the smallest selected and the largest rejected router logit with their raw bf16 bit patterns; and the boundary margin counted in bf16 ULPS under the sign-magnitude total order. Probability space would have been the wrong space, because the softmax denominator is itself a device-order f32 reduction while the bf16 logits are what the selection is a function of, and a probability gap reads as "small" for one representable step and for fifty alike. Per call: an FNV-1a hash of every token's sorted list, so comparing two arms at one layer is one string comparison, plus four `sum|x|` axes that decompose the block's own output. The tap reads buffers the reference path had already downloaded, so it moves no extra bytes except one guarded shared-expert readback, and it is inert with the switch unset. The three fused CUDA arms keep their ids on device and are not tapped: no `qwen4_exp` checkpoint reaches them, and a readback added to a capturable path to instrument a model that cannot enter it would be dead code. Gated in `test_qwen4_exp_moe` against `MoeReference`, that suite's independent double-precision reimplementation of the lane-pinned oracle, with the case asserting its own capture through a sentinel so a broken redirect cannot read as "the tap printed nothing". A second ctest entry re-runs the same binary with the switch set, because the budget is a read-once static and the default run can only ever observe it unset; that entry is deliberately unfiltered, since a `-tc` pattern matching nothing exits 0 having run nothing. Refs #2552, #2547. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code] --- scripts/env-doc-allowlist.txt | 1 + src/vllm/model_executor/models/qwen3_5.cpp | 170 ++++++++++++++++ tests/CMakeLists.txt | 15 ++ tests/vllm/models/test_qwen4_exp_moe.cpp | 219 +++++++++++++++++++++ 4 files changed, 405 insertions(+) diff --git a/scripts/env-doc-allowlist.txt b/scripts/env-doc-allowlist.txt index 8803917ed..cce0f1680 100644 --- a/scripts/env-doc-allowlist.txt +++ b/scripts/env-doc-allowlist.txt @@ -154,6 +154,7 @@ VT_MOE_FUSED_W13 VT_MOE_GLUE_FUSE VT_MOE_HOST_FREE VT_MOE_PAD_STATS +VT_MOE_SEL_FP VT_MOE_SHARED_AUX_STREAM VT_MOE_SHARED_AUX_THRESHOLD VT_MOE_SPLIT_K diff --git a/src/vllm/model_executor/models/qwen3_5.cpp b/src/vllm/model_executor/models/qwen3_5.cpp index c495898c5..d5a429bb0 100644 --- a/src/vllm/model_executor/models/qwen3_5.cpp +++ b/src/vllm/model_executor/models/qwen3_5.cpp @@ -6834,6 +6834,163 @@ DBuf MoeBlockBf16Cuda(Dev d, const MoeBlockWeights& w, const HfConfig& cfg, return dout; } +// ─── VT_MOE_SEL_FP — the SELECTED-EXPERT-ID tap (MOEDIV, #2552) ────────────── +// +// `VT_Q4EXP_LAYER_FP` (#2547) taps VALUES, and a value tap cannot separate an +// expert-selection FLIP from re-association inside the expert GEMM. A discrete +// selection has BIMODAL error and not a tolerance: a token's top-k set either +// changes — and that token's MoE output moves by an O(1) amount — or it does not +// change at all and the residue is rounding. One `rel(sum|x|)` averages the two +// together and destroys exactly the bit that says which happened. +// +// So this prints the SELECTION. Per token: the selected ids SORTED, because the +// assertion between two arms is SET equality and the selection ORDER is not part +// of it (`vt::MoeCombine` sums over the k slots and is order-invariant given the +// weights). Per call: `sel`, an FNV-1a hash over every token's sorted list, so +// comparing two arms' selections at one layer is ONE string comparison. +// +// THE MARGIN IS IN LOGIT SPACE AND ITS UNIT IS bf16 ULPS. Probability space is +// the wrong space: the softmax denominator is a device-order f32 reduction and +// differs between the arms, while the bf16 logits are what the selection is +// actually a function of. And a probability difference reads as "small" for a +// gap of one representable step and for a gap of fifty. `ulps` is the number of +// representable bf16 values between the largest REJECTED logit and the smallest +// SELECTED one under the sign-magnitude total order; `ulps == 0` means the two +// are the SAME bf16 value and the boundary was decided by the lowest-index +// tie-break, which one ulp of re-association anywhere upstream will flip. +// +// `lines=` IS THE COUNTED PROPERTY. An instrument that never ran and two arms +// whose taps agreed look identical in a diff. The measuring job asserts +// `lines == calls * T` and refuses to report a comparison otherwise. +// +// IT IS ON THE REFERENCE ARM ONLY, which is the arm a stacked keep-quant +// checkpoint takes and the only arm any `qwen4_exp` checkpoint reaches. The +// three fused CUDA arms keep their ids on device; adding a readback to a +// capturable path, to instrument a model that cannot enter it, would be dead +// code by construction. See the spec's "Wave MOEDIV" section. +int64_t MoeSelFpCalls() { + static const int64_t n = [] { + const char* e = std::getenv("VT_MOE_SEL_FP"); + if (e == nullptr || e[0] == '\0') return static_cast(0); + const long long parsed = std::atoll(e); + return parsed > 0 ? static_cast(parsed) : static_cast(0); + }(); + return n; +} +int64_t& MoeSelFpCall() { + static int64_t call = 0; + return call; +} +int64_t& MoeSelFpLines() { + static int64_t lines = 0; + return lines; +} + +// bf16 under the sign-magnitude TOTAL ORDER, as a signed key. Monotone in the +// value it encodes, so the difference of two keys counts representable steps. +// -0 and +0 both map to 0, which is what makes an exact tie read `ulps=0`. +int32_t Bf16Ordered(uint16_t b) { + return (b & 0x8000u) != 0 ? -static_cast(b & 0x7fffu) + : static_cast(b & 0x7fffu); +} + +double SumAbsBf16(const std::vector& v) { + double s = 0.0; + for (uint16_t b : v) { + const float f = vt::BF16ToF32(b); + if (std::isfinite(f)) s += std::fabs(static_cast(f)); + } + return s; +} + +// One tapped MoE block invocation. Every buffer here is one the reference path +// had ALREADY downloaded, so the tap moves no extra bytes off the device except +// the one guarded shared-expert read its caller passes in. +void MoeSelFp(int dev_type, int64_t T, int64_t E, int64_t top_k, + const std::vector& h, const std::vector& logits, + const std::vector& ids, + const std::vector& expert_out, + const std::vector* shared) { + const int64_t call = MoeSelFpCall(); + if (call >= MoeSelFpCalls()) return; + uint64_t sel = 1469598103934665603ull; // FNV-1a offset basis + int64_t min_ulps = -1; + int64_t min_tok = -1; + std::vector sorted(static_cast(top_k)); + std::vector picked(static_cast(E)); + for (int64_t t = 0; t < T; ++t) { + for (int64_t j = 0; j < top_k; ++j) + sorted[static_cast(j)] = ids[static_cast(t * top_k + j)]; + std::sort(sorted.begin(), sorted.end()); + std::string idstr; + for (int64_t j = 0; j < top_k; ++j) { + if (j != 0) idstr += ","; + idstr += std::to_string(sorted[static_cast(j)]); + // The hash is over the SORTED list, so it is a set identity and not a + // selection-order identity. + const uint32_t v = static_cast(sorted[static_cast(j)]); + for (int b = 0; b < 4; ++b) { + sel ^= static_cast((v >> (8 * b)) & 0xffu); + sel *= 1099511628211ull; + } + } + // The boundary: the smallest SELECTED logit against the largest REJECTED + // one, read from the bf16 the router GEMM actually stored. + for (int64_t e = 0; e < E; ++e) picked[static_cast(e)] = 0; + for (int64_t j = 0; j < top_k; ++j) + picked[static_cast(sorted[static_cast(j)])] = 1; + int64_t lo_e = -1, hi_e = -1; + for (int64_t e = 0; e < E; ++e) { + const uint16_t b = logits[static_cast(t * E + e)]; + if (picked[static_cast(e)] != 0) { + if (lo_e < 0 || Bf16Ordered(b) < Bf16Ordered(logits[static_cast(t * E + lo_e)])) + lo_e = e; + } else { + if (hi_e < 0 || Bf16Ordered(b) > Bf16Ordered(logits[static_cast(t * E + hi_e)])) + hi_e = e; + } + } + const uint16_t lo_b = lo_e >= 0 ? logits[static_cast(t * E + lo_e)] : 0; + const uint16_t hi_b = hi_e >= 0 ? logits[static_cast(t * E + hi_e)] : 0; + // top_k == E leaves nothing rejected; the boundary is then undefined and is + // reported as such rather than as a zero margin. + const int64_t ulps = hi_e >= 0 && lo_e >= 0 + ? static_cast(Bf16Ordered(lo_b)) - + static_cast(Bf16Ordered(hi_b)) + : -1; + if (ulps >= 0 && (min_ulps < 0 || ulps < min_ulps)) { + min_ulps = ulps; + min_tok = t; + } + ++MoeSelFpLines(); + std::fprintf(stderr, + "moesel call=%lld dev=%d tok=%lld ids=%s lo_e=%lld lo=%.9g " + "lo_raw=0x%04x hi_e=%lld hi=%.9g hi_raw=0x%04x margin=%.9g " + "ulps=%lld\n", + static_cast(call), dev_type, static_cast(t), + idstr.c_str(), static_cast(lo_e), + static_cast(vt::BF16ToF32(lo_b)), static_cast(lo_b), + static_cast(hi_e), static_cast(vt::BF16ToF32(hi_b)), + static_cast(hi_b), + static_cast(vt::BF16ToF32(lo_b)) - + static_cast(vt::BF16ToF32(hi_b)), + static_cast(ulps)); + } + // The digest. `x`/`logit`/`exp`/`shr` decompose the block's own output, so + // with the selection sets equal these four say WHICH GEMM carries the residue. + std::fprintf(stderr, + "moesel call=%lld dev=%d T=%lld E=%lld k=%lld sel=%016llx " + "minulps=%lld mintok=%lld x=%.9g logit=%.9g exp=%.9g shr=%.9g " + "lines=%lld END\n", + static_cast(call), dev_type, static_cast(T), + static_cast(E), static_cast(top_k), + static_cast(sel), static_cast(min_ulps), + static_cast(min_tok), SumAbsBf16(h), SumAbsBf16(logits), + SumAbsBf16(expert_out), shared != nullptr ? SumAbsBf16(*shared) : 0.0, + static_cast(MoeSelFpLines())); + ++MoeSelFpCall(); +} + DBuf MoeBlock(Dev d, const MoeBlockWeights& w, const HfConfig& cfg, const Tensor& dh, int64_t T) { const int64_t H = cfg.hidden_size; @@ -6959,6 +7116,19 @@ DBuf MoeBlock(Dev d, const MoeBlockWeights& w, const HfConfig& cfg, std::optional shared; if (has_shared) shared.emplace(SharedExpert(d, w, cfg, dh, T, fp4)); + // MOEDIV (#2552): the selected-expert-id tap. Inert unless VT_MOE_SEL_FP is + // set; the outer guard is what keeps the shared-expert readback off the + // default path. Everything else it reads was already downloaded above. + if (MoeSelFpCalls() > 0) { + std::vector shr; + if (has_shared) { + shr.resize(static_cast(T) * H); + shared->Download(d, shr.data()); + } + MoeSelFp(static_cast(dh.device.type), T, E, top_k, h, logits, ids, + expert_out, has_shared ? &shr : nullptr); + } + // Combine (moe-semantics.md §6): out = shared + sum_j w_j * expert_out_j. DBuf deo(d, DType::kBF16, {T, top_k, H}, expert_out.data()); DBuf dwt(d, DType::kF32, {T, top_k}, weights.data()); diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 40bb2a914..7db1d5c34 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -3601,6 +3601,21 @@ target_include_directories(test_qwen4_exp_forward PRIVATE ${CMAKE_SOURCE_DIR}/sr # the src/ grant. vllm_cpp_add_test(test_qwen4_exp_moe vllm/models/test_qwen4_exp_moe.cpp) target_include_directories(test_qwen4_exp_moe PRIVATE ${CMAKE_SOURCE_DIR}/src) +# MOEDIV (#2552): the selected-expert-id tap. `MoeSelFpCalls()` is another +# read-once static, so the default run above can only ever observe the switch +# UNSET -- where the tap case asserts the tap is INERT. Re-run the SAME binary +# with it set: that is the arm which gates the printed ids, the bf16-ulp margin +# and the set-equality hash against `MoeReference`. +# +# DELIBERATELY UNFILTERED. A `-tc=` pattern that matches nothing exits 0 having +# run nothing, and doctest reports that as `assertions: 0 ... SUCCESS!` -- a +# skip wearing a pass, which this row has already paid for. Running the whole +# binary cannot fail that way, and it buys a second property for free: every +# value case in the suite re-runs with the tap ON, so a tap that perturbed the +# block's own output would red them. +add_test(NAME test_qwen4_exp_moe_sel_fp COMMAND test_qwen4_exp_moe) +set_tests_properties(test_qwen4_exp_moe_sel_fp PROPERTIES + ENVIRONMENT "VT_MOE_SEL_FP=4096") # The five suites that reach src/vllm/support/platform_compat.h, granted per # target rather than globally: the file already carries 123 explicit diff --git a/tests/vllm/models/test_qwen4_exp_moe.cpp b/tests/vllm/models/test_qwen4_exp_moe.cpp index 13ecfc6ff..0625b6600 100644 --- a/tests/vllm/models/test_qwen4_exp_moe.cpp +++ b/tests/vllm/models/test_qwen4_exp_moe.cpp @@ -107,10 +107,19 @@ #include #include #include +#include #include #include #include +// MOEDIV (#2552): the selected-expert-id tap prints on stderr, so this suite +// captures the descriptor around ONE `RunBlock` call. POSIX only; the tap case +// refuses to report on a platform without it rather than passing vacuously. +#if defined(__unix__) || defined(__APPLE__) +#include +#define VLLM_MOE_SEL_FP_CAPTURE 1 +#endif + #include "vllm/model_executor/models/qwen4_exp.h" #include "vllm/model_executor/models/qwen4_exp_weights.h" #include "vt/backend.h" @@ -318,6 +327,10 @@ struct RefOut { std::vector y; // [T, H] std::vector> selected; // [T][top_k], oracle order std::vector margin; // [T] p_k - p_{k+1} + // MOEDIV (#2552): the bf16 BITS of the router logits. The tap's margin is in + // logit space and its unit is bf16 ulps, so gating it needs the bit patterns + // and not the doubles they were rounded from. + std::vector logit_bits; // [T * E] }; RefOut MoeReference(const MoeSource& s, const std::vector& xc) { @@ -325,6 +338,7 @@ RefOut MoeReference(const MoeSource& s, const std::vector& xc) { r.y.assign(static_cast(kT) * kH, 0.0); r.selected.resize(static_cast(kT)); r.margin.assign(static_cast(kT), 0.0); + r.logit_bits.assign(static_cast(kT) * kE, 0); for (int64_t t = 0; t < kT; ++t) { std::vector x(static_cast(kH)); for (int64_t h = 0; h < kH; ++h) @@ -339,6 +353,7 @@ RefOut MoeReference(const MoeSource& s, const std::vector& xc) { a += x[static_cast(h)] * (static_cast(s.router[static_cast(e * kH + h)]) * kWU); logit[static_cast(e)] = Bf16(a); + r.logit_bits[static_cast(t * kE + e)] = vt::F32ToBF16(static_cast(a)); } const double mx = *std::max_element(logit.begin(), logit.end()); std::vector prob(static_cast(kE)); @@ -710,3 +725,207 @@ TEST_CASE("Qwen4Exp MoE: the router and shared gate reach the seam as bf16") { if (vt::BF16ToF32(rg[i]) == static_cast(s.router[i]) * kWU) ++exact; CHECK(exact == s.router.size()); } + +// ─── MOEDIV (#2552): the SELECTED-EXPERT-ID tap ────────────────────────────── +// +// `VT_Q4EXP_LAYER_FP` taps values, and a value tap cannot separate an +// expert-selection FLIP from re-association inside the expert GEMM: a discrete +// selection has BIMODAL error and not a tolerance. `VT_MOE_SEL_FP` prints the +// selection instead. This case is what makes the printed line trustworthy — +// it gates the ids against `MoeReference`, this suite's independent +// double-precision reimplementation of the lane-pinned oracle, and it gates +// `ulps` against the reference's own bf16 logit BITS. +// +// THE CASE NAME CONTAINS NO COMMA. `-tc` splits its filter on commas, so a +// comma in a case name silently selects nothing and the run reports +// `assertions: 0 ... SUCCESS!` at rc 0 — a skip wearing a pass. The ctest entry +// `test_qwen4_exp_moe_sel_fp` re-runs this binary with the switch set and +// filters on this exact string. +// +// IT ASSERTS ITS OWN CAPTURE. A sentinel is written into the redirected +// descriptor and read back, so a capture that silently caught nothing fails +// here rather than reading as "the tap printed nothing", which is the polarity +// the OFF arm asserts and the one a broken instrument would fake. +namespace { + +#ifdef VLLM_MOE_SEL_FP_CAPTURE +// Everything written to stderr while `body` runs, as a string. +template +std::string CaptureStderr(F&& body) { + std::fflush(stderr); + int saved = ::dup(2); + char path[] = "/tmp/vllm_moesel_XXXXXX"; + int fd = ::mkstemp(path); + REQUIRE(saved >= 0); + REQUIRE(fd >= 0); + ::dup2(fd, 2); + body(); + std::fflush(stderr); + ::dup2(saved, 2); + ::close(saved); + ::lseek(fd, 0, SEEK_SET); + std::string out; + char buf[4096]; + ssize_t n = 0; + while ((n = ::read(fd, buf, sizeof(buf))) > 0) out.append(buf, static_cast(n)); + ::close(fd); + ::unlink(path); + return out; +} + +std::vector LinesWithPrefix(const std::string& s, const std::string& pfx) { + std::vector out; + size_t i = 0; + while (i < s.size()) { + const size_t e = s.find('\n', i); + const std::string line = s.substr(i, e == std::string::npos ? std::string::npos : e - i); + if (line.rfind(pfx, 0) == 0) out.push_back(line); + if (e == std::string::npos) break; + i = e + 1; + } + return out; +} + +// `key=` of a `moesel` line, as a string. +std::string Field(const std::string& line, const std::string& key) { + const std::string k = " " + key + "="; + const size_t i = line.find(k); + if (i == std::string::npos) return std::string(); + const size_t b = i + k.size(); + const size_t e = line.find(' ', b); + return line.substr(b, e == std::string::npos ? std::string::npos : e - b); +} + +// bf16 under the sign-magnitude total order — the tap's own `Bf16Ordered`, +// re-derived here rather than shared, so a sign-handling defect in the tap has +// nothing on this side agreeing with it. +int32_t RefBf16Ordered(uint16_t b) { + return (b & 0x8000U) != 0 ? -static_cast(b & 0x7fffU) + : static_cast(b & 0x7fffU); +} +#endif // VLLM_MOE_SEL_FP_CAPTURE + +} // namespace + +TEST_CASE("Qwen4Exp MoE: the selected-expert-id tap names the oracle's own set") { +#ifndef VLLM_MOE_SEL_FP_CAPTURE + MESSAGE("no POSIX stderr capture on this platform; the tap is NOT gated here"); + return; +#else + MoeSource s = BuildSource(/*keep_quant=*/false); + const Qwen4ExpParams p = Params(); + MoeBlockWeights mw = vllm::Qwen4ExpMoeBlockWeights(s.w, p); + const std::vector xc = HiddenCodes(); + const RefOut ref = MoeReference(s, xc); + AssertRoutingIsNonTrivial(ref); + + std::vector got; + const std::string cap = CaptureStderr([&] { + std::fprintf(stderr, "moesel-capture-sentinel\n"); + got = RunBlock(mw, p, xc); + }); + // The capture's OWN precondition. Without this line the OFF arm below is an + // assertion of absence from a redirect that may never have happened. + REQUIRE(cap.find("moesel-capture-sentinel") != std::string::npos); + REQUIRE(got.size() == static_cast(kT) * kH); + + const std::vector lines = LinesWithPrefix(cap, "moesel call="); + const char* budget = std::getenv("VT_MOE_SEL_FP"); + if (budget == nullptr || budget[0] == '\0' || std::string(budget) == "0") { + // A DIAGNOSTIC MUST BE INERT ON THE DEFAULT PATH. The sentinel above proves + // the capture ran, so this is a measurement and not a failed grep. + MESSAGE("VT_MOE_SEL_FP unset: the tap must print nothing"); + CHECK(lines.empty()); + return; + } + + // ON. One value line per token plus one digest line. + REQUIRE(lines.size() == static_cast(kT) + 1); + const std::string& digest = lines.back(); + CHECK(Field(digest, "T") == std::to_string(kT)); + CHECK(Field(digest, "E") == std::to_string(kE)); + CHECK(Field(digest, "k") == std::to_string(kTopK)); + // `lines` is the PROCESS-WIDE running total, and this entry runs the whole + // binary, so other cases have already spent some of it. The exact per-call + // count is asserted above as `lines.size() == kT + 1` over the captured + // window; this asserts the running total is at least what this call added, + // which is the property the measuring job reads back. + CHECK(std::stoll(Field(digest, "lines")) >= static_cast(kT)); + + uint64_t want_sel = 1469598103934665603ULL; + int64_t want_min_ulps = -1; + int64_t want_min_tok = -1; + for (int64_t t = 0; t < kT; ++t) { + const std::string& ln = lines[static_cast(t)]; + CAPTURE(ln); + CHECK(Field(ln, "tok") == std::to_string(t)); + + // (1) THE SET. Sorted, because two arms selecting the same experts in a + // different order have made the same routing decision. + std::vector want = ref.selected[static_cast(t)]; + std::sort(want.begin(), want.end()); + std::string want_ids; + for (size_t j = 0; j < want.size(); ++j) { + if (j != 0) want_ids += ","; + want_ids += std::to_string(want[j]); + const uint32_t v = static_cast(want[j]); + for (int b = 0; b < 4; ++b) { + want_sel ^= static_cast((v >> (8 * b)) & 0xffU); + want_sel *= 1099511628211ULL; + } + } + CHECK(Field(ln, "ids") == want_ids); + + // (2) THE BOUNDARY, from the reference's OWN bf16 logit bits. + int lo_e = -1, hi_e = -1; + for (int64_t e = 0; e < kE; ++e) { + const bool picked = std::find(want.begin(), want.end(), static_cast(e)) != want.end(); + const uint16_t b = ref.logit_bits[static_cast(t * kE + e)]; + const int32_t o = RefBf16Ordered(b); + if (picked) { + if (lo_e < 0 || o < RefBf16Ordered(ref.logit_bits[static_cast(t * kE + lo_e)])) + lo_e = static_cast(e); + } else { + if (hi_e < 0 || o > RefBf16Ordered(ref.logit_bits[static_cast(t * kE + hi_e)])) + hi_e = static_cast(e); + } + } + REQUIRE(lo_e >= 0); + REQUIRE(hi_e >= 0); + const int64_t want_ulps = + static_cast(RefBf16Ordered(ref.logit_bits[static_cast(t * kE + lo_e)])) - + static_cast(RefBf16Ordered(ref.logit_bits[static_cast(t * kE + hi_e)])); + CHECK(Field(ln, "lo_e") == std::to_string(lo_e)); + CHECK(Field(ln, "hi_e") == std::to_string(hi_e)); + CHECK(Field(ln, "ulps") == std::to_string(want_ulps)); + // The margin has to be POSITIVE and the ulp count has to agree with it: a + // selected logit below a rejected one is a routing defect, not a rounding. + CHECK(want_ulps > 0); + if (want_min_ulps < 0 || want_ulps < want_min_ulps) { + want_min_ulps = want_ulps; + want_min_tok = t; + } + } + // (3) The digest hash IS the set-equality assertion two arms use. Recomputed + // here from the oracle's sets, so a tap that hashed the SELECTION ORDER (and + // would therefore report a false flip between two arms that agree) reds. + char want_hex[32]; + std::snprintf(want_hex, sizeof(want_hex), "%016llx", + static_cast(want_sel)); + CHECK(Field(digest, "sel") == std::string(want_hex)); + CHECK(Field(digest, "minulps") == std::to_string(want_min_ulps)); + CHECK(Field(digest, "mintok") == std::to_string(want_min_tok)); + + // (4) The four decomposition axes are POPULATED and finite. `x` is the block + // input and is checkable in closed form; the other three are only asserted + // non-zero, because their value is the measurement and not the contract. + double want_x = 0.0; + for (size_t i = 0; i < xc.size(); ++i) + want_x += std::fabs(static_cast( + vt::BF16ToF32(vt::F32ToBF16(static_cast(xc[i]) * kAU)))); + CHECK(std::stod(Field(digest, "x")) == doctest::Approx(want_x).epsilon(1e-9)); + CHECK(std::stod(Field(digest, "logit")) > 0.0); + CHECK(std::stod(Field(digest, "exp")) > 0.0); + CHECK(std::stod(Field(digest, "shr")) > 0.0); +#endif +} From eea91d0b7c0d2260e895d23b60d1eab10ac84d83 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 2 Sep 2026 07:19:46 +0000 Subject: [PATCH 3/3] docs(MODEL-MM-QWEN4-EXP): the selections DIFFER, and a third of the boundaries are exact ties MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tap ran on `thor:gpu0` over the released UD-IQ1_S artifact, three arms on one binary. At `E = 512` and `top_k = 10` the router's top-k boundary is an EXACT bf16 tie at 79 of 240 prefill token-slots on the CPU control, 32.9%, and inside one representable bf16 step at 55.8%. At a tie the selection carries no information: the lowest-index tie-break decides it, and any perturbation upstream reshuffles it. The flip rate is the tie rate, 75 of 240 slots on the `VT_GDN_CHUNKED=0` arm against 79 tied. #2552's own headline number is NOT a flip, which is the half the issue got wrong. At layer 0 with the Gated DeltaNet source removed, the arm its table was taken on, both arms select the same experts for all five tokens. The `7.269e-05` residue decomposes onto the expert GEMM instead: `x` 2.139e-05 in, `logit` 2.378e-05 (the router GEMM does not amplify), `exp` 1.421e-04 at 6.6x, `shr` 4.310e-05. It is not a defect. vLLM routes this model on bf16 logits at 512 experts too: `Qwen4ExpSparseMoeBlock` inherits Qwen3Next's plain `ReplicatedLinear` gate with no `params_dtype`, the runner applies it as a plain `F.linear`, and the f32 widening happens inside `topk_softmax` — our polarity, our widening point, our lowest-index tie-break. The expert GEMM's arm-to-arm difference is the re-association `cuda_quant_dot.cu` already documents, and llama.cpp splits its own CPU and CUDA arms the same way. Read at a forward reference 1566 commits past the pin, which carries no `qwen4_exp` at all, and recorded as a source read rather than a measurement: no vLLM process was run on this checkpoint. Two independent instruments agree bit for bit on the MoE block's input. This tap's `x` axis reads the tensor `VT_Q4EXP_LAYER_FP` calls `L00 mhc.mix`, from a different tree, and all three arms reproduce PREFILLDIV's values exactly. The decode half is VOID and is published only to say so. On `origin/main` `a99b9c69a` the CUDA arm answers `11751 271 271 271 271 271 0 0`, not the fluent sequence PREFILLDIV recorded, because that wave measured a tree carrying #2550's decode fix and #2550 has not landed. From step 1 the arms run different token sequences, so the 336-of-336 decode flip count is a different-input artifact. `docs/USAGE.md` is deliberately untouched: its checkpoint row still claims no `qwen4_exp` step reaches a CUDA queue, which is false, and #2554 already carries that correction. Refs #2547. Refs #2550. Closes #2552. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code] --- .agents/specs/qwen4-exp-flash-next.md | 61 +++ .../qwen4exp-moe-selection-20260902.md | 238 +++++++++++ .../cmp-CUDA-GDNSEQ.txt | 80 ++++ .../cmp-CUDA-PROD.txt | 80 ++++ .../digests-CPU-CTRL.txt | 384 ++++++++++++++++++ .../digests-CUDA-GDNSEQ.txt | 384 ++++++++++++++++++ .../digests-CUDA-PROD.txt | 384 ++++++++++++++++++ .../results.txt | 75 ++++ 8 files changed, 1686 insertions(+) create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902.md create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt create mode 100644 docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt diff --git a/.agents/specs/qwen4-exp-flash-next.md b/.agents/specs/qwen4-exp-flash-next.md index 932a71648..5a484f1f6 100644 --- a/.agents/specs/qwen4-exp-flash-next.md +++ b/.agents/specs/qwen4-exp-flash-next.md @@ -8406,6 +8406,67 @@ say so with the upstream anchors and close #2552 as answered rather than fixed. **Do not force a fix.** Changing a CUDA arm to agree with our CPU arm, when the CUDA arm is the one that mirrors the oracle, moves this model AWAY from vLLM. +### Outcome + +**MEASURED. The selections DIFFER, and it is NOT a defect.** Full result in +[the evidence file](../../docs/bench-evidence/qwen4exp-moe-selection-20260902.md); +`thor:gpu0` `sm_110`, job `a5bf074b-0f1a-490d-ab51-5d561857ef9e`, one binary +`b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf`, three arms, +the released UD-IQ1_S artifact verified inside the lease. + +**The number that answers the row.** At `E = 512` and `top_k = 10` the router's +top-k boundary is an **EXACT bf16 tie at 79 of 240 prefill token-slots (32.9%)** +on the CPU control, and inside one representable bf16 step at 55.8%. At a tie +the selection carries no information: the lowest-index tie-break decides it. The +flip rate IS the tie rate — 75 of 240 slots flip on the `VT_GDN_CHUNKED=0` arm +(31.3%), 78 on the production arm. + +**#2552's own headline number is NOT a flip, and this is the half the issue got +wrong.** At layer 0 with the Gated DeltaNet source removed — the arm its table +was taken on — both arms select the same experts for all five tokens. The +`7.269e-05` residue decomposes onto the expert GEMM: `x` 2.139e-05, `logit` +2.378e-05 (the router GEMM does not amplify), `exp` **1.421e-04** (6.6x), `shr` +4.310e-05. The keep-quant grouped GEMM is the amplifier, and +`cuda_quant_dot.cu:2158-2170` already says what it is — the integer core is +bit-identical and only the per-block float scale sum is reassociated, which +llama.cpp's own CPU/CUDA split does too. + +**Why it is not a defect.** vLLM routes this model on bf16 logits at 512 experts +as well: `Qwen4ExpSparseMoeBlock` inherits Qwen3Next's plain `ReplicatedLinear` +gate with no `params_dtype`, `moe_runner.py:897-902` runs it as a plain +`F.linear`, and the f32 widening happens inside `topk_softmax` — exactly our +polarity, our widening point and our lowest-index tie-break. Upstream sits on +the same knife edge. Read at `cdefd9d499`, a FORWARD REFERENCE 1566 commits past +the pin, which carries no `qwen4_exp` at all. **This is a source read, not a +measurement: no vLLM process was run on this checkpoint.** + +**One thing to WATCH rather than fix.** vLLM owns a fp32-capable router gate, +`GateLinear` (`gate_linear.py:18-33`), a `PluggableLayer` with three of five +tiers emitting fp32 logits. Qwen3Next does not opt in, so it is unreachable for +this architecture today. If upstream ever flips `qwen4_exp` onto it we mirror +that; widening ours first would move this model AWAY from the oracle, which is +what the stop condition above forbids. + +**The gate question, settled with a mechanism.** PREFILLDIV observed that token +agreement is not monotone in numerical distance and proposed no CPU-vs-CUDA +token gate. This says WHY: a third of the routing decisions carry zero margin, +so the emitted sequence is a function of tie-break order rather than of +accuracy. **No CPU-vs-CUDA token-exactness gate is well posed for `qwen4_exp`.** +The selection-set agreement rate and the tie-rate histogram ARE well posed and +are now measurable; a gate against vLLM stays the right target and stays OWED, +because vLLM's GGUF support is an out-of-tree plugin and every safetensors arm +of this model exceeds the largest fleet box. + +**What this run could NOT prove, and one thing it found by accident.** The +decode half is VOID: on `origin/main` `a99b9c69a` the CUDA arm answers +`11751 271 271 271 271 271 0 0`, not the fluent sequence PREFILLDIV recorded, +because that wave measured a tree carrying +[#2550](https://github.com/mudler/vllm.cpp/pull/2550)'s decode fix and #2550 has +not landed. From step 1 the two arms run different token sequences, so the +comparator's 336-of-336 decode flip count is a different-input artifact and is +published only to say so. Step 0 is the only comparable phase until #2550 +merges. + ## Now `ACTIVE`. **THE COUNT IS THE TABLE, AND THIS SENTENCE NO LONGER RESTATES IT.** diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902.md b/docs/bench-evidence/qwen4exp-moe-selection-20260902.md new file mode 100644 index 000000000..aa6de5ab8 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902.md @@ -0,0 +1,238 @@ +# The `qwen4_exp` MoE routers DO pick different experts, and the reason is that a third of the boundaries are EXACT TIES, 2 September 2026 + +Wave MOEDIV of [`MODEL-MM-QWEN4-EXP`](../../.agents/specs/qwen4-exp-flash-next.md), +[#2552](https://github.com/mudler/vllm.cpp/issues/2552). + +**The one-line result.** The two arms' expert selections **DIFFER** — 75 of 240 +prefill token-slots on the arm #2552 measured, 78 of 240 on the production CUDA +arm. **They are not a defect.** At `num_experts = 512` and +`num_experts_per_tok = 10` the router's top-k boundary is an EXACT bf16 tie at +**79 of 240 (32.9%)** of prefill token-slots on the CPU control, and within ONE +representable bf16 step at 55.8%. At a tie the selection carries no information +at all: it is decided by the lowest-index tie-break, and any perturbation +anywhere upstream reshuffles it. **The flip rate is the tie rate** — 31.3% of +slots flip against 32.9% tied. + +**And #2552's own headline number is NOT a flip.** At decoder layer 0 with the +Gated DeltaNet source removed, which is the arm the issue's table was taken on, +the two arms select the **same experts** for all five tokens. The `7.269e-05` +residue there is arithmetic, and this run says which GEMM produces it. + +## 1. What was run, and on what + +**Tree.** `8e2ba6931dd5aae33955ffdd8e7258ff5d90eef0` — `origin/main` `a99b9c69a` +plus this wave's `VT_MOE_SEL_FP` tap and nothing else. Source tarball sha256 +`be5f31ec1d0b3c7ce4ed724647cd9295caa48db62dbff1689e501654d775fd44`. + +**Host.** `thor:gpu0` under an `rc` lease, job +`a5bf074b-0f1a-490d-ab51-5d561857ef9e`, pod `rc-worker-n8smh`, aarch64, NVIDIA +Thor, compute capability 11.0, driver 595.78, nvcc 13.0.88, built `sm_110`. +`CMAKE rc=0`; `BUILD rc=0 wall=921s objects=585`; server +`b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf`, ONE binary +for all three arms. + +**Artifact.** The released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S, shard 1 +sha256 `88a1420825a9304063e882ada29d438263617f51ac8923d438d927496693bafd`, +72,546,461,650 bytes, verified inside the lease before any arm ran. + +**Request.** `examples/vllm-server`, greedy, `max_tokens=8`, prompt +`The capital of France is`, `--block-size 16 --num-blocks 128 --max-model-len +256`, `VT_CPU_QUANT_REPACK=0` on every arm — the same request PREFILLDIV used. +Geometry read off the tap itself: `T=5 E=512 k=10`, 48 MoE calls per prefill +step, so **the MoE call index IS the decoder layer index at step 0**. + +**The tap gate ran on this box and this toolchain**, not only on the development +machine: `TAP GATE OFF rc=0 … assertions: 121 | 121 passed | 0 failed` and +`TAP GATE ON rc=0 … assertions: 172 | 172 passed | 0 failed`. + +**The instrument asserts that it ran.** Every arm printed 960 `moesel` lines, +384 digests and 48 prefill calls, and every arm's last digest reads +`lines=576`, which is the derivable total (48 prefill calls x 5 tokens + 336 +decode calls x 1). The comparator re-derives it and prints +`COUNTED-PROPERTY … agree=True` for both sides of both comparisons. + +**Two independent instruments agree on the MoE block's input, bit for bit.** +This tap's `x` axis reads the same tensor `VT_Q4EXP_LAYER_FP` calls `L00 +mhc.mix`, from a different tree, and the numbers are identical: + +| arm | this tap, `x` at call 0 | [PREFILLDIV](qwen4exp-cuda-prefill-divergence-20260902.md), `L00 mhc.mix` | +|---|---|---| +| CPU-CTRL | 3613.82031 | 3613.82031 | +| CUDA-PROD | 3615.62777 | 3615.62777 | +| CUDA-GDNSEQ | 3613.74301 | 3613.74301 | + +Both `rel` values reproduce too: `4.999e-04` and `2.139e-05`. + +## 2. The selections + +`sel` is an FNV-1a hash over each token's SORTED selected ids, so one string +comparison per layer is the set-equality assertion. Step 0 only. + +| comparison | prefill calls | `sel` mismatched | token-slots | FLIPPED | first flip | +|---|---|---|---|---|---| +| CPU-CTRL vs CUDA-PROD | 48 | 40 | 240 | **78** | call 0 (layer 0), tok 2 | +| CPU-CTRL vs CUDA-GDNSEQ | 48 | 38 | 240 | **75** | **call 1 (layer 1)**, tok 2 | + +**Layer 0 separates the two sources here exactly as it did for the values.** On +the `VT_GDN_CHUNKED=0` arm — the one whose MoE residue #2552 named — layer 0's +selections AGREE on all five tokens, and the first flip is at layer 1. On the +production arm layer 0 already flips, because the chunked Gated DeltaNet has +put 23x more difference into that layer's MoE input (`4.999e-04` against +`2.139e-05`). So the flip threshold at layer 0 sits between those two numbers. + +## 3. Why they flip: the boundary is a TIE + +`ulps` is the number of representable **bf16** values between the largest +REJECTED logit and the smallest SELECTED one, under the sign-magnitude total +order. `ulps = 0` means they are the SAME bf16 value. + +| arm | slots | `ulps`=0 | 1 | 2 | 3 | 4 | 5 | >=6 | tied | <=1 ulp | +|---|---|---|---|---|---|---|---|---|---|---| +| CPU-CTRL | 240 | **79** | 55 | 31 | 32 | 16 | 13 | 14 | **32.9%** | 55.8% | +| CUDA-PROD | 240 | 63 | 62 | 48 | 33 | 9 | 13 | 12 | 26.2% | 52.1% | +| CUDA-GDNSEQ | 240 | 72 | 68 | 41 | 29 | 10 | 9 | 11 | 30.0% | 58.3% | + +The median MoE call's smallest margin is `minulps = 0` on every arm. The first +flip on the GDNSEQ arm reads + +``` +base ids=16,19,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) +other ids=16,48,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) +``` + +— identical bit patterns on both sides of the boundary, on both arms. Expert 19 +and expert 48 hold the same bf16 logit as the tenth-place expert, and which one +is selected is decided by nothing but which index the greedy scan reaches +first. **A discrete selection has bimodal error, and here a third of the +boundaries have no margin at all.** + +That is the whole mechanism, and the arithmetic is the trigger rather than the +cause. `logit` (the router GEMM's own output) differs by only `2.378e-05` at +layer 0 on the GDNSEQ arm — no amplification over its `2.139e-05` input — yet +by layer 5 the block INPUT differs by `1.630e-02`, three orders of magnitude +higher. That jump is flipped experts entering the residual stream, not rounding +compounding. + +## 4. Where the layer-0 residue actually comes from + +With the selections equal, the digest's four `sum|x|` axes decompose the block. +Layer 0, `VT_GDN_CHUNKED=0`, `rel` against the CPU control: + +| axis | what it is | rel | vs the `2.139e-05` input | +|---|---|---|---| +| `x` | the block input | 2.139e-05 | 1.0x | +| `logit` | the bf16 router logits | 2.378e-05 | 1.1x | +| `exp` | the assembled per-slot expert outputs | **1.421e-04** | **6.6x** | +| `shr` | the shared expert | 4.310e-05 | 2.0x | + +and PREFILLDIV's `L00 moe` (the combined block output) reads `7.269e-05`, +between `exp` and `shr` as the weighted combine of the two requires. **The +router GEMM is not the amplifier. The keep-quant grouped expert GEMM is.** + +## 5. Is it a defect? No, and here is the reading + +**The router mirrors vLLM.** Read at vLLM +`cdefd9d4997f00da72dc6245cc60678b50761b7e`, a **FORWARD REFERENCE** 1566 commits +beyond this row's parity pin `5559679229bc961848b121ccdeaa8fa5d79bec98`, which +has no `vllm/models/qwen4_exp/` at all: + +| anchor | what it says | +|---|---| +| `vllm/models/qwen4_exp/nvidia/model.py:160` | `class Qwen4ExpSparseMoeBlock(Qwen3NextSparseMoeBlock)` — it adds a sequence-parallel refusal and `n_shared_experts`, and inherits the router untouched | +| `vllm/model_executor/models/qwen3_next.py:170-176` | `self.gate = ReplicatedLinear(hidden_size, num_experts, bias=False, quant_config=None)` — **no `params_dtype`**, so the gate weight is the resolved model dtype | +| `vllm/model_executor/layers/fused_moe/runner/moe_runner.py:897-902` | `router_logits, _ = self.gate(hidden_states)` — plain `F.linear`, bf16 in, **bf16 out** | +| `vllm/model_executor/layers/fused_moe/router/fused_topk_router.py:26-41` -> `ops.topk_softmax` | the widening to f32 happens INSIDE the top-k kernel, after the GEMM | +| `vllm/model_executor/layers/fused_moe/experts/cpu_moe.py:69` | the CPU arm of the same: `gating_output = gating_output.float()` | + +Ours: `MoeBlock`'s reference path computes `logits = MatmulBf16(...)` into a +**bf16** `[T,E]` buffer and hands it to `vt::MoeRouterTopK`, whose CPU +(`src/vt/cpu/cpu_ops.cpp:2951`) and CUDA (`src/vt/cuda/cuda_moe.cu:64`) kernels +both widen into an f32 softmax and take a greedy top-k with a strict `>` +ascending scan, so the lowest expert index wins an exact tie. Same width, same +widening point, same tie-break. **vLLM routes this model on bf16 logits at 512 +experts too, so upstream sits on the same knife edge.** + +vLLM does own a fp32-capable gate — `GateLinear` +(`vllm/model_executor/layers/fused_moe/router/gate_linear.py:18-33`), a +`PluggableLayer` whose five dispatch tiers include three that emit fp32 logits. +**Qwen3Next does not opt into it**, so none of those tiers is reachable for this +architecture and Tier 5 (`F.linear`) is what runs. That is recorded as a thing +to mirror IF upstream ever flips this model onto `GateLinear`, and it is not a +licence to widen ours first: doing so would move us AWAY from the oracle. + +**The expert GEMM's arm-to-arm difference is a documented re-association, and +the secondary oracle has the same one.** `src/vt/cuda/cuda_quant_dot.cu:2158-2170` +states it outright — "The INTEGER core (sum x.qs*y.qs per 32-block) is +bit-identical; only the per-block float scale sum is reassociated (warp tree vs +CPU sequential)" — and the activation quantizer is a bit-exact port either way +(`cpu_quant_act.cpp:51-81` against `cuda_quant_dot.cu:1174-1195`: ternary amax, +`d = amax/127`, `F32ToF16(d)`, `roundf(x*id)`). The released UD-IQ1_S expert +towers are IQ4_NL / Q5_0, the 32-element Q8_0-activation lane +(`cuda_quant_dot.cu:2528-2534`). llama.cpp @ `237ad9b961f009ae19ac29dbce4cd0c1251f94b3` +splits the same way: `ggml/src/ggml-cuda/mmvq.cu:566-641` accumulates per-lane +partials and finishes with `warp_reduce_sum`, while the CPU `vec_dot` +(`ggml/src/ggml-cpu/arch/x86/quants.c:841-852`) sums sequentially. + +**So neither arm is wrong.** vLLM's own CPU and CUDA backends would disagree on +this model for the same two reasons, and at a 32.9% exact-tie rate the +disagreement is not a small one. #2552 is answered rather than fixed. + +## 6. What this run RETIRES, and what it says about a gate + +**#2552's own hypothesis is half right and the halves are separable.** The issue +asked whether the layer-0 MoE residue is a selection flip. It is **not** — layer +0 on the GDNSEQ arm agrees on every token, and the residue decomposes onto the +expert GEMM. But from layer 1 the flips are real, they are the dominant source +of everything downstream, and they were invisible to every value tap taken on +this row. + +**It also confirms PREFILLDIV's gate conclusion with a mechanism.** That wave +observed that token agreement is not monotone in numerical distance and proposed +no CPU-vs-CUDA token gate. This says why: a third of the routing decisions in +this model carry zero margin, so the token sequence is a function of tie-break +order, not of accuracy. **No CPU-vs-CUDA token-exactness gate can be well posed +for `qwen4_exp` at bf16 routing.** What IS well posed and is now measurable is +the selection-set agreement rate and the tie-rate histogram of §3, which are +properties of one arm and comparable across trees. A gate against **vLLM** stays +the right target and stays OWED: it needs an artifact vLLM can load, and vLLM's +GGUF support is an out-of-tree plugin (`docs/features/quantization/gguf.md`) +while every safetensors arm of this model exceeds the largest fleet box. + +## 7. The raw outputs + +Committed beside this file in +[`qwen4exp-moe-selection-20260902/`](qwen4exp-moe-selection-20260902/), so every +number above is checkable rather than transcribed: `results.txt` (the job's own +`RESULT` lines, including both `TAP GATE` rcs and every arm's counted property), +`cmp-CUDA-PROD.txt` and `cmp-CUDA-GDNSEQ.txt` (the full comparisons, with the +first twenty flips of each phase spelled out), and `digests-.txt` (all 384 +digest lines per arm — the `sel` hash, `minulps` and the four `sum|x|` axes for +every MoE call of every step). The per-token value lines are NOT committed: at +576 per arm they add nothing the digests and the flip lists do not already +carry. + +## 8. What this is NOT + +n = 1. One prompt, one artifact, one box, one repetition, greedy only, UD-IQ1_S +only, `num_reqs = 1`. **NOT a token gate:** no oracle decoded this prompt, and +the CPU arm is a CONTROL, not an oracle — nothing here says which arm is closer +to correct. No speed number: the tap synchronises once per MoE call for the +shared-expert readback, so the wall times are instrumented and compare to +nothing. The "vLLM sits on the same knife edge" conclusion in §5 is a **SOURCE +READ, not a measurement**; no vLLM process was run on this checkpoint. + +**THE DECODE COLUMNS ARE VOID AND ARE PUBLISHED ONLY TO SAY SO.** The comparator +reports 336 of 336 decode token-slots flipped, and that number means nothing: +on this tree the CUDA arm answers +`11751 271 271 271 271 271 0 0` (production) and +`11751 271 271 11751 271 271 271 0` (`VT_GDN_CHUNKED=0`) against the CPU +control's `11751 13 15767 411 2029 11 1092 369`, so from step 1 the two arms are +running different token sequences and a 100% flip rate is a different-INPUT +artifact rather than a numerics result. + +**That degeneracy is itself a finding.** PREFILLDIV recorded a fluent CUDA arm +(`11751 13 15767 411 1928 11 628 567`) — but it measured a tree that had +[#2550](https://github.com/mudler/vllm.cpp/pull/2550)'s decode fix merged in. +**On `origin/main` at `a99b9c69a`, `--device cuda` does NOT reproduce that**; the +fluency depends on a pull request that has not landed. Only step 0 is comparable +until it does. diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt new file mode 100644 index 000000000..6e32efbaf --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt @@ -0,0 +1,80 @@ +=== CPU-CTRL vs CUDA-GDNSEQ === +COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +CALLS common=384 prefill(T=5)=48 decode(T=1)=336 +--- PREFILL --- +COUNT PREFILL calls=48 sel-hash MISMATCH=38 +COUNT PREFILL token-slots=240 FLIPPED=75 +FIRST PREFILL FLIP call=1 tok=2 + base ids=16,19,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) + other ids=16,48,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) + FLIP call=1 tok=2 base=16,19,56,58,61,230,250,320,426,493 other=16,48,56,58,61,230,250,320,426,493 base_ulps=0 other_ulps=0 + FLIP call=2 tok=2 base=75,83,84,226,354,381,403,408,415,473 other=83,84,226,354,370,381,403,408,415,473 base_ulps=0 other_ulps=1 + FLIP call=3 tok=1 base=26,122,186,253,302,352,377,405,418,476 other=26,122,186,253,302,352,377,405,421,476 base_ulps=0 other_ulps=0 + FLIP call=3 tok=2 base=40,118,153,252,253,302,418,421,432,476 other=118,122,153,252,253,302,418,421,432,476 base_ulps=1 other_ulps=0 + FLIP call=4 tok=1 base=112,133,205,252,273,309,360,419,469,488 other=133,159,205,252,273,309,360,419,469,488 base_ulps=1 other_ulps=3 + FLIP call=4 tok=4 base=41,48,54,63,133,205,252,309,393,469 other=41,48,54,133,205,252,309,360,393,469 base_ulps=0 other_ulps=1 + FLIP call=5 tok=4 base=34,162,220,236,254,421,423,450,460,493 other=34,162,220,236,254,336,366,421,460,493 base_ulps=2 other_ulps=0 + FLIP call=7 tok=2 base=66,97,269,292,303,337,346,439,458,470 other=66,97,269,301,303,337,346,439,458,470 base_ulps=1 other_ulps=2 + FLIP call=7 tok=3 base=66,104,148,225,247,303,346,435,481,510 other=66,104,148,225,303,346,419,435,481,510 base_ulps=0 other_ulps=1 + FLIP call=8 tok=3 base=94,167,224,231,280,289,333,355,382,455 other=94,167,224,231,280,333,355,382,455,462 base_ulps=0 other_ulps=2 + FLIP call=8 tok=4 base=94,167,224,231,275,308,365,414,448,462 other=94,167,224,231,275,308,414,422,448,462 base_ulps=0 other_ulps=3 + FLIP call=9 tok=3 base=17,75,318,356,406,424,453,462,486,509 other=17,52,318,356,406,424,453,462,486,509 base_ulps=0 other_ulps=0 + FLIP call=9 tok=4 base=52,63,75,139,230,238,406,424,462,509 other=17,52,63,75,139,238,406,424,462,509 base_ulps=1 other_ulps=1 + FLIP call=11 tok=1 base=90,108,163,201,238,261,268,350,397,477 other=31,90,108,163,201,261,268,350,397,477 base_ulps=1 other_ulps=1 + FLIP call=13 tok=4 base=148,162,250,273,282,290,296,385,387,488 other=148,162,250,273,282,290,385,387,418,488 base_ulps=5 other_ulps=1 + FLIP call=14 tok=4 base=106,138,160,173,299,400,409,415,460,500 other=106,138,160,173,299,365,400,409,415,460 base_ulps=1 other_ulps=1 + FLIP call=18 tok=4 base=3,92,93,116,139,194,201,226,314,428 other=3,39,92,93,116,139,194,201,226,428 base_ulps=2 other_ulps=4 + FLIP call=19 tok=0 base=31,173,293,375,406,413,434,462,474,500 other=8,31,173,293,375,406,413,434,474,500 base_ulps=1 other_ulps=0 + FLIP call=20 tok=4 base=63,65,73,155,240,248,258,265,323,496 other=63,73,78,155,240,248,258,265,323,496 base_ulps=1 other_ulps=1 + FLIP call=21 tok=4 base=67,98,167,189,195,225,329,412,473,488 other=98,142,167,189,195,225,329,412,473,488 base_ulps=0 other_ulps=1 +MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +AXIS PREFILL x first_nonzero_call=0 worst_rel=1.630e-02 at call=5 +AXIS PREFILL logit first_nonzero_call=0 worst_rel=2.938e-03 at call=44 +AXIS PREFILL exp first_nonzero_call=0 worst_rel=3.892e-02 at call=45 +AXIS PREFILL shr first_nonzero_call=0 worst_rel=5.065e-02 at call=39 +CALL0 PREFILL base x=3613.82031 logit=17079.5703 exp=2540.09374 shr=250.805253 +CALL0 PREFILL other x=3613.74301 logit=17079.1641 exp=2540.45471 shr=250.794444 +CALL0 PREFILL rel x = 2.139e-05 +CALL0 PREFILL rel logit = 2.378e-05 +CALL0 PREFILL rel exp = 1.421e-04 +CALL0 PREFILL rel shr = 4.310e-05 +--- DECODE --- +COUNT DECODE calls=336 sel-hash MISMATCH=336 +COUNT DECODE token-slots=336 FLIPPED=336 +FIRST DECODE FLIP call=48 tok=0 + base ids=91,106,308,366,367,386,434,466,503,505 ulps=2 lo=-4.9375(0xc09e) hi=-5(0xc0a0) + other ids=31,109,168,205,212,240,251,258,294,442 ulps=7 lo=-4.28125(0xc089) hi=-4.5(0xc090) + FLIP call=48 tok=0 base=91,106,308,366,367,386,434,466,503,505 other=31,109,168,205,212,240,251,258,294,442 base_ulps=2 other_ulps=7 + FLIP call=49 tok=0 base=83,113,114,222,266,298,320,357,500,504 other=19,57,95,113,125,320,370,383,389,475 base_ulps=1 other_ulps=2 + FLIP call=50 tok=0 base=29,75,100,163,218,327,446,473,484,487 other=11,32,71,75,137,150,222,280,403,502 base_ulps=2 other_ulps=0 + FLIP call=51 tok=0 base=26,83,143,156,224,227,295,302,366,476 other=82,190,202,205,237,243,253,335,394,476 base_ulps=1 other_ulps=2 + FLIP call=52 tok=0 base=48,78,133,160,205,239,280,309,313,415 other=41,68,163,252,294,346,382,388,410,496 base_ulps=1 other_ulps=10 + FLIP call=53 tok=0 base=6,34,104,216,220,254,338,421,460,476 other=20,68,110,137,163,220,236,326,420,435 base_ulps=2 other_ulps=1 + FLIP call=54 tok=0 base=15,18,92,169,177,228,270,344,403,424 other=66,92,168,220,237,285,326,397,403,409 base_ulps=4 other_ulps=2 + FLIP call=55 tok=0 base=31,69,104,151,225,303,389,435,448,510 other=22,52,194,208,242,325,346,351,458,499 base_ulps=0 other_ulps=1 + FLIP call=56 tok=0 base=81,94,167,183,231,280,355,362,455,462 other=210,224,227,231,244,308,316,422,426,462 base_ulps=7 other_ulps=5 + FLIP call=57 tok=0 base=17,52,159,230,274,281,321,406,463,509 other=71,75,111,139,230,238,318,396,424,452 base_ulps=1 other_ulps=2 + FLIP call=58 tok=0 base=139,194,207,267,296,307,393,399,405,461 other=203,207,225,296,320,322,357,393,433,461 base_ulps=3 other_ulps=0 + FLIP call=59 tok=0 base=30,86,90,201,238,246,261,310,371,500 other=30,69,84,98,108,261,273,397,477,495 base_ulps=0 other_ulps=2 + FLIP call=60 tok=0 base=10,36,139,180,216,281,344,366,381,436 other=67,82,139,180,192,219,392,418,421,505 base_ulps=3 other_ulps=1 + FLIP call=61 tok=0 base=36,92,162,250,259,282,290,292,453,488 other=148,162,250,273,282,381,387,400,463,480 base_ulps=0 other_ulps=0 + FLIP call=62 tok=0 base=2,106,135,181,297,299,300,400,415,501 other=75,77,82,86,94,106,160,365,407,454 base_ulps=4 other_ulps=2 + FLIP call=63 tok=0 base=120,251,317,319,378,432,443,470,490,496 other=12,100,199,283,327,351,432,470,490,501 base_ulps=2 other_ulps=1 + FLIP call=64 tok=0 base=40,141,206,228,270,274,314,350,489,497 other=92,191,228,272,274,277,320,392,480,489 base_ulps=1 other_ulps=1 + FLIP call=65 tok=0 base=3,107,131,152,291,299,351,363,412,498 other=68,84,105,160,172,299,363,407,469,498 base_ulps=2 other_ulps=2 + FLIP call=66 tok=0 base=3,194,201,278,314,350,401,402,418,442 other=11,28,89,101,194,201,264,275,398,475 base_ulps=2 other_ulps=2 + FLIP call=67 tok=0 base=25,31,45,81,84,140,202,328,343,434 other=30,64,84,225,246,263,364,403,434,454 base_ulps=0 other_ulps=2 +MIN DECODE base minulps over calls: min=0 p10=0 median=1 +AXIS DECODE x first_nonzero_call=48 worst_rel=3.448e-01 at call=149 +AXIS DECODE logit first_nonzero_call=48 worst_rel=8.444e-02 at call=55 +AXIS DECODE exp first_nonzero_call=48 worst_rel=4.142e-01 at call=375 +AXIS DECODE shr first_nonzero_call=48 worst_rel=6.083e-01 at call=243 +CALL0 DECODE base x=711.202027 logit=3398.40625 exp=412.329928 shr=43.5615003 +CALL0 DECODE other x=723.470177 logit=3429.89062 exp=563.539422 shr=31.5357613 +CALL0 DECODE rel x = 1.696e-02 +CALL0 DECODE rel logit = 9.179e-03 +CALL0 DECODE rel exp = 2.683e-01 +CALL0 DECODE rel shr = 2.761e-01 +VERDICT prefill expert-selection SET EQUALITY: DIFFERS (75 flipped token-slots) diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt new file mode 100644 index 000000000..3c73b84b4 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt @@ -0,0 +1,80 @@ +=== CPU-CTRL vs CUDA-PROD === +COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +CALLS common=384 prefill(T=5)=48 decode(T=1)=336 +--- PREFILL --- +COUNT PREFILL calls=48 sel-hash MISMATCH=40 +COUNT PREFILL token-slots=240 FLIPPED=78 +FIRST PREFILL FLIP call=0 tok=2 + base ids=21,64,76,101,265,293,301,314,338,361 ulps=0 lo=-5(0xc0a0) hi=-5(0xc0a0) + other ids=21,76,101,265,293,301,314,338,361,369 ulps=1 lo=-5(0xc0a0) hi=-5.03125(0xc0a1) + FLIP call=0 tok=2 base=21,64,76,101,265,293,301,314,338,361 other=21,76,101,265,293,301,314,338,361,369 base_ulps=0 other_ulps=1 + FLIP call=2 tok=3 base=29,75,83,100,163,329,370,446,473,487 other=29,75,83,100,163,218,329,446,473,487 base_ulps=0 other_ulps=0 + FLIP call=3 tok=1 base=26,122,186,253,302,352,377,405,418,476 other=26,122,186,253,302,352,377,405,421,476 base_ulps=0 other_ulps=1 + FLIP call=4 tok=1 base=112,133,205,252,273,309,360,419,469,488 other=133,159,205,252,273,309,360,419,469,488 base_ulps=1 other_ulps=2 + FLIP call=4 tok=4 base=41,48,54,63,133,205,252,309,393,469 other=41,48,54,133,205,252,309,360,393,469 base_ulps=0 other_ulps=1 + FLIP call=5 tok=4 base=34,162,220,236,254,421,423,450,460,493 other=34,162,220,236,254,278,336,421,460,493 base_ulps=2 other_ulps=0 + FLIP call=6 tok=4 base=44,86,92,135,145,168,177,305,377,501 other=44,86,92,124,135,145,168,305,377,501 base_ulps=1 other_ulps=0 + FLIP call=7 tok=2 base=66,97,269,292,303,337,346,439,458,470 other=66,97,269,283,292,303,337,346,439,470 base_ulps=1 other_ulps=0 + FLIP call=7 tok=4 base=20,66,72,93,221,303,337,374,435,458 other=20,66,72,93,97,303,337,374,435,458 base_ulps=3 other_ulps=0 + FLIP call=8 tok=4 base=94,167,224,231,275,308,365,414,448,462 other=94,167,224,231,275,308,414,422,448,462 base_ulps=0 other_ulps=2 + FLIP call=9 tok=3 base=17,75,318,356,406,424,453,462,486,509 other=17,318,326,356,406,424,453,462,486,509 base_ulps=0 other_ulps=1 + FLIP call=9 tok=4 base=52,63,75,139,230,238,406,424,462,509 other=17,52,63,75,139,238,406,424,462,509 base_ulps=1 other_ulps=1 + FLIP call=11 tok=0 base=4,104,108,130,132,180,361,393,430,464 other=4,104,108,128,130,132,393,400,430,464 base_ulps=0 other_ulps=0 + FLIP call=11 tok=1 base=90,108,163,201,238,261,268,350,397,477 other=90,108,163,201,261,268,293,350,397,477 base_ulps=1 other_ulps=1 + FLIP call=12 tok=3 base=10,90,139,180,257,281,302,433,436,489 other=10,90,139,180,257,281,302,344,433,436 base_ulps=1 other_ulps=0 + FLIP call=13 tok=4 base=148,162,250,273,282,290,296,385,387,488 other=148,250,273,282,290,296,385,387,418,488 base_ulps=5 other_ulps=2 + FLIP call=16 tok=1 base=92,141,150,206,228,307,392,429,487,497 other=92,141,206,228,270,307,392,429,487,497 base_ulps=0 other_ulps=1 + FLIP call=16 tok=3 base=206,228,274,314,350,392,487,489,497,504 other=40,206,228,274,314,350,487,489,497,504 base_ulps=2 other_ulps=0 + FLIP call=18 tok=4 base=3,92,93,116,139,194,201,226,314,428 other=3,39,92,93,116,139,194,201,226,428 base_ulps=2 other_ulps=6 + FLIP call=19 tok=3 base=30,58,64,111,146,147,202,343,394,434 other=30,64,111,146,147,202,343,394,426,434 base_ulps=0 other_ulps=2 +MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +AXIS PREFILL x first_nonzero_call=0 worst_rel=1.121e-02 at call=5 +AXIS PREFILL logit first_nonzero_call=0 worst_rel=2.882e-03 at call=32 +AXIS PREFILL exp first_nonzero_call=0 worst_rel=3.878e-02 at call=45 +AXIS PREFILL shr first_nonzero_call=0 worst_rel=3.095e-02 at call=39 +CALL0 PREFILL base x=3613.82031 logit=17079.5703 exp=2540.09374 shr=250.805253 +CALL0 PREFILL other x=3615.62777 logit=17083.2891 exp=2527.65707 shr=250.743384 +CALL0 PREFILL rel x = 4.999e-04 +CALL0 PREFILL rel logit = 2.177e-04 +CALL0 PREFILL rel exp = 4.896e-03 +CALL0 PREFILL rel shr = 2.467e-04 +--- DECODE --- +COUNT DECODE calls=336 sel-hash MISMATCH=336 +COUNT DECODE token-slots=336 FLIPPED=336 +FIRST DECODE FLIP call=48 tok=0 + base ids=91,106,308,366,367,386,434,466,503,505 ulps=2 lo=-4.9375(0xc09e) hi=-5(0xc0a0) + other ids=31,109,168,205,212,240,251,258,294,442 ulps=7 lo=-4.28125(0xc089) hi=-4.5(0xc090) + FLIP call=48 tok=0 base=91,106,308,366,367,386,434,466,503,505 other=31,109,168,205,212,240,251,258,294,442 base_ulps=2 other_ulps=7 + FLIP call=49 tok=0 base=83,113,114,222,266,298,320,357,500,504 other=19,57,95,113,125,320,370,383,389,475 base_ulps=1 other_ulps=2 + FLIP call=50 tok=0 base=29,75,100,163,218,327,446,473,484,487 other=11,32,71,75,137,203,222,280,403,502 base_ulps=2 other_ulps=1 + FLIP call=51 tok=0 base=26,83,143,156,224,227,295,302,366,476 other=82,190,202,205,237,243,253,335,394,476 base_ulps=1 other_ulps=0 + FLIP call=52 tok=0 base=48,78,133,160,205,239,280,309,313,415 other=41,68,163,252,294,346,382,388,410,496 base_ulps=1 other_ulps=4 + FLIP call=53 tok=0 base=6,34,104,216,220,254,338,421,460,476 other=20,68,110,137,163,220,236,326,420,435 base_ulps=2 other_ulps=0 + FLIP call=54 tok=0 base=15,18,92,169,177,228,270,344,403,424 other=66,168,220,237,285,326,397,403,409,498 base_ulps=4 other_ulps=0 + FLIP call=55 tok=0 base=31,69,104,151,225,303,389,435,448,510 other=52,129,194,208,302,325,346,351,458,499 base_ulps=0 other_ulps=0 + FLIP call=56 tok=0 base=81,94,167,183,231,280,355,362,455,462 other=210,224,227,231,244,308,316,422,426,462 base_ulps=7 other_ulps=10 + FLIP call=57 tok=0 base=17,52,159,230,274,281,321,406,463,509 other=71,75,111,139,238,318,355,396,424,452 base_ulps=1 other_ulps=1 + FLIP call=58 tok=0 base=139,194,207,267,296,307,393,399,405,461 other=203,225,296,320,322,357,393,417,433,461 base_ulps=3 other_ulps=1 + FLIP call=59 tok=0 base=30,86,90,201,238,246,261,310,371,500 other=30,35,69,84,98,108,261,273,397,477 base_ulps=0 other_ulps=1 + FLIP call=60 tok=0 base=10,36,139,180,216,281,344,366,381,436 other=67,82,139,164,180,192,392,418,421,505 base_ulps=3 other_ulps=1 + FLIP call=61 tok=0 base=36,92,162,250,259,282,290,292,453,488 other=148,162,273,282,330,381,387,400,463,480 base_ulps=0 other_ulps=1 + FLIP call=62 tok=0 base=2,106,135,181,297,299,300,400,415,501 other=75,77,82,86,94,106,160,365,407,454 base_ulps=4 other_ulps=1 + FLIP call=63 tok=0 base=120,251,317,319,378,432,443,470,490,496 other=6,12,199,283,327,351,432,470,490,501 base_ulps=2 other_ulps=3 + FLIP call=64 tok=0 base=40,141,206,228,270,274,314,350,489,497 other=92,191,228,272,274,277,350,392,480,489 base_ulps=1 other_ulps=3 + FLIP call=65 tok=0 base=3,107,131,152,291,299,351,363,412,498 other=68,84,105,160,172,221,299,363,454,469 base_ulps=2 other_ulps=1 + FLIP call=66 tok=0 base=3,194,201,278,314,350,401,402,418,442 other=11,28,89,101,194,201,264,275,287,475 base_ulps=2 other_ulps=0 + FLIP call=67 tok=0 base=25,31,45,81,84,140,202,328,343,434 other=30,64,84,225,246,249,263,364,434,454 base_ulps=0 other_ulps=0 +MIN DECODE base minulps over calls: min=0 p10=0 median=1 +AXIS DECODE x first_nonzero_call=48 worst_rel=3.418e-01 at call=149 +AXIS DECODE logit first_nonzero_call=48 worst_rel=8.848e-02 at call=55 +AXIS DECODE exp first_nonzero_call=48 worst_rel=4.084e-01 at call=372 +AXIS DECODE shr first_nonzero_call=48 worst_rel=6.339e-01 at call=183 +CALL0 DECODE base x=711.202027 logit=3398.40625 exp=412.329928 shr=43.5615003 +CALL0 DECODE other x=723.664909 logit=3430.46875 exp=563.497883 shr=31.5413151 +CALL0 DECODE rel x = 1.722e-02 +CALL0 DECODE rel logit = 9.346e-03 +CALL0 DECODE rel exp = 2.683e-01 +CALL0 DECODE rel shr = 2.759e-01 +VERDICT prefill expert-selection SET EQUALITY: DIFFERS (78 flipped token-slots) diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt new file mode 100644 index 000000000..dfc59069d --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt @@ -0,0 +1,384 @@ +moesel call=0 dev=0 T=5 E=512 k=10 sel=f06a56dd5ffbf75d minulps=0 mintok=2 x=3613.82031 logit=17079.5703 exp=2540.09374 shr=250.805253 lines=5 END +moesel call=1 dev=0 T=5 E=512 k=10 sel=993cb7ac2f9ebaf3 minulps=0 mintok=2 x=3778.01525 logit=15559.4219 exp=2033.00741 shr=34.0071162 lines=10 END +moesel call=2 dev=0 T=5 E=512 k=10 sel=06f1af8ff5cd8f17 minulps=0 mintok=2 x=3839.48339 logit=15776.5469 exp=1795.80639 shr=51.7584638 lines=15 END +moesel call=3 dev=0 T=5 E=512 k=10 sel=d6bea8c3978b8bf0 minulps=0 mintok=1 x=4867.46251 logit=15097 exp=2573.91393 shr=37.8174276 lines=20 END +moesel call=4 dev=0 T=5 E=512 k=10 sel=379aaf5199791a68 minulps=0 mintok=4 x=5361.50879 logit=15455.75 exp=3039.10715 shr=47.7793482 lines=25 END +moesel call=5 dev=0 T=5 E=512 k=10 sel=c3f74a3051738847 minulps=0 mintok=1 x=4507.43522 logit=15666.25 exp=2064.90387 shr=34.3856177 lines=30 END +moesel call=6 dev=0 T=5 E=512 k=10 sel=2ba8099d8f428dab minulps=0 mintok=0 x=5471.63166 logit=15897.3125 exp=3109.92718 shr=46.9859591 lines=35 END +moesel call=7 dev=0 T=5 E=512 k=10 sel=b5644706ca226ea6 minulps=0 mintok=3 x=5342.22944 logit=15933.3281 exp=2723.02001 shr=44.1138059 lines=40 END +moesel call=8 dev=0 T=5 E=512 k=10 sel=e961aa0c5c947daa minulps=0 mintok=3 x=5381.78599 logit=15765.5 exp=3050.63449 shr=28.1738703 lines=45 END +moesel call=9 dev=0 T=5 E=512 k=10 sel=3059c42228399c4d minulps=0 mintok=0 x=5517.82366 logit=15659.5625 exp=2961.69517 shr=35.0320792 lines=50 END +moesel call=10 dev=0 T=5 E=512 k=10 sel=b14a7db9158b4e90 minulps=0 mintok=0 x=4960.85244 logit=15764.1406 exp=2429.17253 shr=27.1544901 lines=55 END +moesel call=11 dev=0 T=5 E=512 k=10 sel=e19210b1d108db0d minulps=0 mintok=0 x=4866.6894 logit=15596.3594 exp=2163.10668 shr=18.8600467 lines=60 END +moesel call=12 dev=0 T=5 E=512 k=10 sel=98f4025615644ee6 minulps=0 mintok=0 x=4971.42549 logit=15651.6406 exp=2259.01844 shr=26.7149538 lines=65 END +moesel call=13 dev=0 T=5 E=512 k=10 sel=d570a4cab1b7404e minulps=0 mintok=0 x=4644.05666 logit=15960.7734 exp=2035.5708 shr=27.478671 lines=70 END +moesel call=14 dev=0 T=5 E=512 k=10 sel=cec11c8b5014de56 minulps=1 mintok=4 x=5252.21439 logit=15834.5469 exp=2900.69116 shr=23.0321477 lines=75 END +moesel call=15 dev=0 T=5 E=512 k=10 sel=88cb94c69381a618 minulps=0 mintok=0 x=6735.20952 logit=15805.5938 exp=3958.20042 shr=29.6381709 lines=80 END +moesel call=16 dev=0 T=5 E=512 k=10 sel=a8002b1c0688c414 minulps=0 mintok=0 x=6149.08192 logit=15762.4844 exp=3181.60388 shr=28.2585843 lines=85 END +moesel call=17 dev=0 T=5 E=512 k=10 sel=cb0c0f7107d1b9f1 minulps=0 mintok=3 x=5359.06631 logit=15997.1562 exp=2583.30954 shr=23.7984908 lines=90 END +moesel call=18 dev=0 T=5 E=512 k=10 sel=541e04fa44573199 minulps=0 mintok=3 x=5321.07004 logit=16327.375 exp=2843.64529 shr=25.2339526 lines=95 END +moesel call=19 dev=0 T=5 E=512 k=10 sel=44be7bbc47cce29b minulps=0 mintok=3 x=5859.39894 logit=16217.5781 exp=2830.69161 shr=28.501454 lines=100 END +moesel call=20 dev=0 T=5 E=512 k=10 sel=0bcc4a933f992ede minulps=0 mintok=1 x=7145.87483 logit=16134.8125 exp=4328.92883 shr=43.0588427 lines=105 END +moesel call=21 dev=0 T=5 E=512 k=10 sel=7e056c1fcbffca34 minulps=0 mintok=0 x=6743.71262 logit=16218.3438 exp=3792.85942 shr=28.5697935 lines=110 END +moesel call=22 dev=0 T=5 E=512 k=10 sel=b3c62ea616442afa minulps=0 mintok=0 x=6487.36654 logit=16644.1406 exp=3624.94315 shr=35.2794219 lines=115 END +moesel call=23 dev=0 T=5 E=512 k=10 sel=0099e3fd59b42d18 minulps=0 mintok=0 x=7552.71469 logit=16095.1641 exp=4454.02452 shr=62.8391198 lines=120 END +moesel call=24 dev=0 T=5 E=512 k=10 sel=56a20f2fdec4ad10 minulps=2 mintok=0 x=5604.4169 logit=16318.8047 exp=2811.55343 shr=27.7470869 lines=125 END +moesel call=25 dev=0 T=5 E=512 k=10 sel=a5741eb95769f3e9 minulps=0 mintok=0 x=5636.99146 logit=16029.4375 exp=2677.11546 shr=28.7961929 lines=130 END +moesel call=26 dev=0 T=5 E=512 k=10 sel=35fd18b3f1d9f02a minulps=0 mintok=1 x=5239.42767 logit=16432.3125 exp=2448.74909 shr=22.5297654 lines=135 END +moesel call=27 dev=0 T=5 E=512 k=10 sel=b21b36340b39c152 minulps=0 mintok=1 x=5248.75765 logit=15959.7188 exp=2343.03231 shr=19.0916266 lines=140 END +moesel call=28 dev=0 T=5 E=512 k=10 sel=c0a026f799fbbd91 minulps=0 mintok=1 x=5273.6478 logit=16085.125 exp=2355.23044 shr=23.3756012 lines=145 END +moesel call=29 dev=0 T=5 E=512 k=10 sel=86319e59463c6c8c minulps=0 mintok=2 x=5503.20621 logit=16543.1953 exp=2749.27634 shr=24.8984516 lines=150 END +moesel call=30 dev=0 T=5 E=512 k=10 sel=da88b0bcd291a413 minulps=0 mintok=2 x=6399.91875 logit=16652.0312 exp=3776.82338 shr=32.4318897 lines=155 END +moesel call=31 dev=0 T=5 E=512 k=10 sel=475532a2b7f1fa69 minulps=0 mintok=1 x=8029.09873 logit=16363.8906 exp=4881.21871 shr=30.7941756 lines=160 END +moesel call=32 dev=0 T=5 E=512 k=10 sel=896d19b81f7e140e minulps=0 mintok=2 x=7358.70547 logit=16027.4609 exp=4366.77026 shr=32.0027352 lines=165 END +moesel call=33 dev=0 T=5 E=512 k=10 sel=6123bb6dff041fa0 minulps=0 mintok=0 x=6689.05148 logit=16679.5156 exp=3787.63997 shr=24.8722188 lines=170 END +moesel call=34 dev=0 T=5 E=512 k=10 sel=3188ef46a7207960 minulps=0 mintok=0 x=6143.31237 logit=16650.6406 exp=3836.00321 shr=34.0574557 lines=175 END +moesel call=35 dev=0 T=5 E=512 k=10 sel=b7e7492a70e7a207 minulps=0 mintok=1 x=7101.2012 logit=16468.8672 exp=4593.54665 shr=33.2344885 lines=180 END +moesel call=36 dev=0 T=5 E=512 k=10 sel=a6674c18f57e7d51 minulps=1 mintok=1 x=8783.09175 logit=16366.5 exp=8155.40826 shr=86.6171582 lines=185 END +moesel call=37 dev=0 T=5 E=512 k=10 sel=abf946cfa9894165 minulps=0 mintok=0 x=8288.97113 logit=16359.5 exp=6937.48806 shr=64.1228779 lines=190 END +moesel call=38 dev=0 T=5 E=512 k=10 sel=bd67adafe51584aa minulps=0 mintok=0 x=8446.94615 logit=16809.3906 exp=9107.50015 shr=108.932895 lines=195 END +moesel call=39 dev=0 T=5 E=512 k=10 sel=b9dc93801db102e5 minulps=0 mintok=0 x=8977.31795 logit=15670.2188 exp=8280.09688 shr=91.7971096 lines=200 END +moesel call=40 dev=0 T=5 E=512 k=10 sel=6d998f5738adf056 minulps=0 mintok=0 x=9499.97248 logit=15696.6094 exp=7250.69534 shr=94.329976 lines=205 END +moesel call=41 dev=0 T=5 E=512 k=10 sel=69d1d0a19457831d minulps=0 mintok=1 x=9620.84339 logit=16114.9375 exp=6914.52953 shr=100.657755 lines=210 END +moesel call=42 dev=0 T=5 E=512 k=10 sel=d6026c35ae47baa9 minulps=0 mintok=0 x=9800.45345 logit=16359.5938 exp=9362.44777 shr=129.539982 lines=215 END +moesel call=43 dev=0 T=5 E=512 k=10 sel=2c74865957babfa5 minulps=0 mintok=3 x=8213.96481 logit=15785.7031 exp=5864.67043 shr=89.8102634 lines=220 END +moesel call=44 dev=0 T=5 E=512 k=10 sel=3a0a55d78229bd21 minulps=0 mintok=1 x=9224.02147 logit=15879.1875 exp=7461.2741 shr=105.141125 lines=225 END +moesel call=45 dev=0 T=5 E=512 k=10 sel=cb02ec54ac780386 minulps=0 mintok=0 x=10381.2064 logit=16223.2344 exp=7515.08854 shr=146.931254 lines=230 END +moesel call=46 dev=0 T=5 E=512 k=10 sel=a05f9c4c058d7890 minulps=0 mintok=2 x=10104.3063 logit=16246 exp=8526.87531 shr=158.675702 lines=235 END +moesel call=47 dev=0 T=5 E=512 k=10 sel=73ac3458f62c111f minulps=0 mintok=3 x=9583.96682 logit=16356.6875 exp=9063.56577 shr=825.793385 lines=240 END +moesel call=48 dev=0 T=1 E=512 k=10 sel=aabd9229c965eb93 minulps=2 mintok=0 x=711.202027 logit=3398.40625 exp=412.329928 shr=43.5615003 lines=241 END +moesel call=49 dev=0 T=1 E=512 k=10 sel=7ef1f3cf7d4f2804 minulps=1 mintok=0 x=792.780496 logit=3110 exp=443.90434 shr=8.94934824 lines=242 END +moesel call=50 dev=0 T=1 E=512 k=10 sel=6c6615cd7c923744 minulps=2 mintok=0 x=796.914679 logit=3283.23438 exp=325.523548 shr=17.2280005 lines=243 END +moesel call=51 dev=0 T=1 E=512 k=10 sel=d2d4fb3fb5f1410d minulps=1 mintok=0 x=1045.87296 logit=3109.07812 exp=593.517361 shr=8.9673944 lines=244 END +moesel call=52 dev=0 T=1 E=512 k=10 sel=86fa045227ed7bd5 minulps=1 mintok=0 x=1060.38456 logit=3132.9375 exp=599.172895 shr=11.508544 lines=245 END +moesel call=53 dev=0 T=1 E=512 k=10 sel=a0b3b88800b67d76 minulps=2 mintok=0 x=761.397766 logit=3059.70312 exp=302.961598 shr=5.06307073 lines=246 END +moesel call=54 dev=0 T=1 E=512 k=10 sel=334effe309200143 minulps=4 mintok=0 x=1079.98165 logit=3158.14062 exp=643.891552 shr=8.80490369 lines=247 END +moesel call=55 dev=0 T=1 E=512 k=10 sel=84252597ab352ed7 minulps=0 mintok=0 x=1068.59662 logit=3243.07812 exp=614.197879 shr=12.7280557 lines=248 END +moesel call=56 dev=0 T=1 E=512 k=10 sel=c1d7aa426974b328 minulps=7 mintok=0 x=1058.49208 logit=3085.25 exp=571.210994 shr=6.95671796 lines=249 END +moesel call=57 dev=0 T=1 E=512 k=10 sel=36b57d61ee92457d minulps=1 mintok=0 x=1048.09664 logit=3101.10938 exp=487.261373 shr=4.99845667 lines=250 END +moesel call=58 dev=0 T=1 E=512 k=10 sel=5a66fc9d4b643f28 minulps=3 mintok=0 x=992.641977 logit=3070.71875 exp=495.778773 shr=5.26890968 lines=251 END +moesel call=59 dev=0 T=1 E=512 k=10 sel=325cd7cf26f72b80 minulps=0 mintok=0 x=1016.83878 logit=3050.8125 exp=488.083284 shr=4.35687687 lines=252 END +moesel call=60 dev=0 T=1 E=512 k=10 sel=8b2e8b711474c593 minulps=3 mintok=0 x=1027.50875 logit=3144.85938 exp=463.934765 shr=5.94254702 lines=253 END +moesel call=61 dev=0 T=1 E=512 k=10 sel=afc0ffcf0383f64d minulps=0 mintok=0 x=942.910339 logit=3106.48438 exp=439.91314 shr=5.42763743 lines=254 END +moesel call=62 dev=0 T=1 E=512 k=10 sel=8efcec71a42408c1 minulps=4 mintok=0 x=1083.31629 logit=3125.53125 exp=560.578572 shr=4.22694099 lines=255 END +moesel call=63 dev=0 T=1 E=512 k=10 sel=4c88cd86fff12233 minulps=2 mintok=0 x=1561.24098 logit=3174.85938 exp=974.06361 shr=7.47097771 lines=256 END +moesel call=64 dev=0 T=1 E=512 k=10 sel=e337f23a62631c60 minulps=1 mintok=0 x=1327.90812 logit=3092.71875 exp=655.407484 shr=7.19295359 lines=257 END +moesel call=65 dev=0 T=1 E=512 k=10 sel=c2a57becde66269e minulps=2 mintok=0 x=1127.2688 logit=3065.51562 exp=518.199874 shr=6.03081573 lines=258 END +moesel call=66 dev=0 T=1 E=512 k=10 sel=34eab51d3a21e181 minulps=2 mintok=0 x=1084.93185 logit=3244.125 exp=571.740371 shr=6.15451421 lines=259 END +moesel call=67 dev=0 T=1 E=512 k=10 sel=8bae9070b56a03f1 minulps=0 mintok=0 x=1197.07237 logit=3153.26562 exp=557.419123 shr=5.07023441 lines=260 END +moesel call=68 dev=0 T=1 E=512 k=10 sel=d0797c4e5cc68d4d minulps=1 mintok=0 x=1449.0214 logit=3137.42188 exp=828.382667 shr=8.49615016 lines=261 END +moesel call=69 dev=0 T=1 E=512 k=10 sel=aae6e24f36be3df4 minulps=3 mintok=0 x=1294.88361 logit=3147.82812 exp=717.240796 shr=5.74877287 lines=262 END +moesel call=70 dev=0 T=1 E=512 k=10 sel=b4ae2ec987af98e8 minulps=1 mintok=0 x=1249.63897 logit=3281.98438 exp=638.724468 shr=7.53334141 lines=263 END +moesel call=71 dev=0 T=1 E=512 k=10 sel=6b54745e8564670b minulps=1 mintok=0 x=1630.56089 logit=3178.07812 exp=954.66822 shr=12.8149843 lines=264 END +moesel call=72 dev=0 T=1 E=512 k=10 sel=d578ce2c70dda8a2 minulps=1 mintok=0 x=1121.67649 logit=3185.1875 exp=533.201703 shr=7.53353781 lines=265 END +moesel call=73 dev=0 T=1 E=512 k=10 sel=b967a85589fd4045 minulps=4 mintok=0 x=1134.39417 logit=3123.57812 exp=520.904717 shr=7.10389788 lines=266 END +moesel call=74 dev=0 T=1 E=512 k=10 sel=29fcdce5f80bc6fb minulps=2 mintok=0 x=1009.4706 logit=3177.17188 exp=460.661269 shr=4.92624117 lines=267 END +moesel call=75 dev=0 T=1 E=512 k=10 sel=8d915ffd5d146bc2 minulps=1 mintok=0 x=1093.21958 logit=3118.75 exp=519.55021 shr=4.85974144 lines=268 END +moesel call=76 dev=0 T=1 E=512 k=10 sel=e3fb5328b5bcbf88 minulps=3 mintok=0 x=1069.93763 logit=3186.10938 exp=476.545881 shr=7.33836451 lines=269 END +moesel call=77 dev=0 T=1 E=512 k=10 sel=18f42552e2caa5c6 minulps=0 mintok=0 x=1124.13273 logit=3191.39062 exp=574.651633 shr=7.35481319 lines=270 END +moesel call=78 dev=0 T=1 E=512 k=10 sel=870bf2b9e0039a8f minulps=5 mintok=0 x=1300.92135 logit=3295.96875 exp=673.63485 shr=8.77113053 lines=271 END +moesel call=79 dev=0 T=1 E=512 k=10 sel=104c9bab786b8034 minulps=0 mintok=0 x=1615.60873 logit=3189.5625 exp=1040.70751 shr=9.76467319 lines=272 END +moesel call=80 dev=0 T=1 E=512 k=10 sel=f3c5730de2391c7a minulps=3 mintok=0 x=1540.40412 logit=3144.60938 exp=858.872363 shr=11.5954378 lines=273 END +moesel call=81 dev=0 T=1 E=512 k=10 sel=e1acf7c505479e63 minulps=5 mintok=0 x=1351.78171 logit=3187.21875 exp=796.809841 shr=8.571166 lines=274 END +moesel call=82 dev=0 T=1 E=512 k=10 sel=69e138de258ce2b4 minulps=0 mintok=0 x=1238.17522 logit=3358.51562 exp=864.889995 shr=7.35813867 lines=275 END +moesel call=83 dev=0 T=1 E=512 k=10 sel=dc8dae3dcd3b07fe minulps=0 mintok=0 x=1430.11096 logit=3220.23438 exp=965.800933 shr=10.1994688 lines=276 END +moesel call=84 dev=0 T=1 E=512 k=10 sel=7e4bdcc479f2fdc8 minulps=2 mintok=0 x=1781.60821 logit=3198 exp=1843.95605 shr=22.3955349 lines=277 END +moesel call=85 dev=0 T=1 E=512 k=10 sel=a0335a14c82fc482 minulps=5 mintok=0 x=1654.70413 logit=3197.04688 exp=1433.99453 shr=14.6948673 lines=278 END +moesel call=86 dev=0 T=1 E=512 k=10 sel=e49bb2f60e4e5bcd minulps=2 mintok=0 x=1721.5401 logit=3309.60938 exp=1631.11502 shr=19.8436993 lines=279 END +moesel call=87 dev=0 T=1 E=512 k=10 sel=1cb0178ca0c9d16b minulps=0 mintok=0 x=1897.42815 logit=3090.01562 exp=2055.54628 shr=26.1610985 lines=280 END +moesel call=88 dev=0 T=1 E=512 k=10 sel=7fe288e3bf1ddb91 minulps=0 mintok=0 x=1875.99385 logit=3063.79688 exp=1450.68229 shr=20.1768826 lines=281 END +moesel call=89 dev=0 T=1 E=512 k=10 sel=8815269d7fe8cfd1 minulps=5 mintok=0 x=2016.26 logit=3151.4375 exp=1488.89249 shr=18.9155179 lines=282 END +moesel call=90 dev=0 T=1 E=512 k=10 sel=c6e629caff9850fa minulps=0 mintok=0 x=1955.149 logit=3205.03125 exp=2227.8235 shr=25.1312252 lines=283 END +moesel call=91 dev=0 T=1 E=512 k=10 sel=c64704829f793e80 minulps=1 mintok=0 x=1922.13663 logit=3174.67188 exp=1526.55625 shr=19.1960543 lines=284 END +moesel call=92 dev=0 T=1 E=512 k=10 sel=387df9e5ab2ea36e minulps=3 mintok=0 x=2144.15098 logit=3208.09375 exp=1932.42218 shr=20.5665565 lines=285 END +moesel call=93 dev=0 T=1 E=512 k=10 sel=2a1c2563aa83e949 minulps=1 mintok=0 x=2240.82428 logit=3280.60938 exp=1734.29974 shr=24.7781918 lines=286 END +moesel call=94 dev=0 T=1 E=512 k=10 sel=6d131570aad3e56c minulps=4 mintok=0 x=2286.95382 logit=3284.53125 exp=1653.91675 shr=34.6715817 lines=287 END +moesel call=95 dev=0 T=1 E=512 k=10 sel=f9e6ace4226e4cf9 minulps=0 mintok=0 x=2058.42086 logit=3239.64062 exp=1604.6322 shr=169.649549 lines=288 END +moesel call=96 dev=0 T=1 E=512 k=10 sel=45d0e90aa559cadb minulps=4 mintok=0 x=721.959273 logit=3429.64062 exp=673.545117 shr=44.7621707 lines=289 END +moesel call=97 dev=0 T=1 E=512 k=10 sel=17cfb2d884234187 minulps=5 mintok=0 x=744.934237 logit=3130.04688 exp=481.194355 shr=6.59894305 lines=290 END +moesel call=98 dev=0 T=1 E=512 k=10 sel=0f243d9291985c24 minulps=1 mintok=0 x=814.22921 logit=3159.96875 exp=474.694537 shr=7.52003516 lines=291 END +moesel call=99 dev=0 T=1 E=512 k=10 sel=c9b415586c05f8b1 minulps=1 mintok=0 x=941.458203 logit=2922.34375 exp=492.467496 shr=4.55752426 lines=292 END +moesel call=100 dev=0 T=1 E=512 k=10 sel=19a4d45b3eb22e7f minulps=0 mintok=0 x=1023.1575 logit=2987.90625 exp=541.392979 shr=5.34364326 lines=293 END +moesel call=101 dev=0 T=1 E=512 k=10 sel=37da03305c750708 minulps=0 mintok=0 x=787.274973 logit=3032.64062 exp=387.346789 shr=4.08089032 lines=294 END +moesel call=102 dev=0 T=1 E=512 k=10 sel=7900205502e3194e minulps=3 mintok=0 x=998.247449 logit=3107.10938 exp=609.805782 shr=7.90607299 lines=295 END +moesel call=103 dev=0 T=1 E=512 k=10 sel=e70364c224214791 minulps=4 mintok=0 x=1068.05092 logit=3148.67188 exp=619.919969 shr=5.91998706 lines=296 END +moesel call=104 dev=0 T=1 E=512 k=10 sel=f733c95e60ef14fa minulps=0 mintok=0 x=1086.83845 logit=3106.29688 exp=620.237022 shr=5.41683059 lines=297 END +moesel call=105 dev=0 T=1 E=512 k=10 sel=de16fc878cf47c0a minulps=1 mintok=0 x=1088.87786 logit=3028.48438 exp=614.294931 shr=6.21226995 lines=298 END +moesel call=106 dev=0 T=1 E=512 k=10 sel=a882e4f32ccbc35b minulps=1 mintok=0 x=1031.23153 logit=3043.23438 exp=538.238265 shr=4.56079428 lines=299 END +moesel call=107 dev=0 T=1 E=512 k=10 sel=40c41ef8a695e12f minulps=4 mintok=0 x=1047.01792 logit=3047.71875 exp=506.939993 shr=3.99706445 lines=300 END +moesel call=108 dev=0 T=1 E=512 k=10 sel=ea91ccc52f46d0bd minulps=1 mintok=0 x=1087.70763 logit=3026.21875 exp=574.908465 shr=6.1161481 lines=301 END +moesel call=109 dev=0 T=1 E=512 k=10 sel=927b13896742804e minulps=1 mintok=0 x=1028.27619 logit=3039.89062 exp=485.082588 shr=5.54919302 lines=302 END +moesel call=110 dev=0 T=1 E=512 k=10 sel=1f8bd4e68bbe9712 minulps=1 mintok=0 x=1136.35674 logit=3027.96875 exp=557.59843 shr=5.56775392 lines=303 END +moesel call=111 dev=0 T=1 E=512 k=10 sel=acbf1db9c37e7c7e minulps=0 mintok=0 x=1530.08761 logit=3091.75 exp=872.510131 shr=6.87254228 lines=304 END +moesel call=112 dev=0 T=1 E=512 k=10 sel=9e461019ce68f288 minulps=2 mintok=0 x=1387.76517 logit=3103.46875 exp=698.423701 shr=4.94248048 lines=305 END +moesel call=113 dev=0 T=1 E=512 k=10 sel=c6cf25721cad5178 minulps=7 mintok=0 x=1176.06792 logit=3113.38281 exp=585.272764 shr=5.42286371 lines=306 END +moesel call=114 dev=0 T=1 E=512 k=10 sel=7af4174660dc2245 minulps=2 mintok=0 x=1132.73144 logit=3230.92188 exp=598.634267 shr=5.56820858 lines=307 END +moesel call=115 dev=0 T=1 E=512 k=10 sel=2d5aac9c0c37e5e6 minulps=0 mintok=0 x=1243.11194 logit=3106.14062 exp=625.335118 shr=6.19604698 lines=308 END +moesel call=116 dev=0 T=1 E=512 k=10 sel=b9578a923a9de6e4 minulps=1 mintok=0 x=1448.87344 logit=3119.29688 exp=946.334552 shr=9.86572686 lines=309 END +moesel call=117 dev=0 T=1 E=512 k=10 sel=0400ba7759444aa7 minulps=1 mintok=0 x=1422.42938 logit=3240.64062 exp=806.576878 shr=5.32338837 lines=310 END +moesel call=118 dev=0 T=1 E=512 k=10 sel=40f05b496f92669a minulps=2 mintok=0 x=1283.71828 logit=3383.8125 exp=682.072263 shr=5.8046688 lines=311 END +moesel call=119 dev=0 T=1 E=512 k=10 sel=c6ee49a3ef5f223a minulps=0 mintok=0 x=1585.95319 logit=3118.64062 exp=936.153902 shr=19.8668005 lines=312 END +moesel call=120 dev=0 T=1 E=512 k=10 sel=3f2beb660bafa2a8 minulps=1 mintok=0 x=1130.52855 logit=3220.75 exp=550.584942 shr=4.38061896 lines=313 END +moesel call=121 dev=0 T=1 E=512 k=10 sel=c5530d98ef4cad0d minulps=0 mintok=0 x=1148.46329 logit=3063.5 exp=540.042083 shr=5.89698556 lines=314 END +moesel call=122 dev=0 T=1 E=512 k=10 sel=369a394e422779dc minulps=10 mintok=0 x=1113.17404 logit=3146.79688 exp=587.988584 shr=3.85845712 lines=315 END +moesel call=123 dev=0 T=1 E=512 k=10 sel=43f36f076a312e3e minulps=3 mintok=0 x=1167.5815 logit=3111.65625 exp=593.37795 shr=5.10464762 lines=316 END +moesel call=124 dev=0 T=1 E=512 k=10 sel=7b57ad1904886660 minulps=4 mintok=0 x=1190.10042 logit=3186.73438 exp=589.303752 shr=5.49600425 lines=317 END +moesel call=125 dev=0 T=1 E=512 k=10 sel=0e748bdab1d7a80c minulps=1 mintok=0 x=1240.59509 logit=3252.04688 exp=672.454912 shr=5.33884706 lines=318 END +moesel call=126 dev=0 T=1 E=512 k=10 sel=fdce871467d97c16 minulps=2 mintok=0 x=1424.39816 logit=3211.09375 exp=885.052741 shr=6.50303227 lines=319 END +moesel call=127 dev=0 T=1 E=512 k=10 sel=b83c04319a08a51d minulps=1 mintok=0 x=1740.47599 logit=3239.39062 exp=1090.04158 shr=7.10091856 lines=320 END +moesel call=128 dev=0 T=1 E=512 k=10 sel=1bf8f022b2eec312 minulps=3 mintok=0 x=1647.05609 logit=3115.20312 exp=1011.3927 shr=6.60869288 lines=321 END +moesel call=129 dev=0 T=1 E=512 k=10 sel=ed35d736a07f76b2 minulps=2 mintok=0 x=1536.28169 logit=3285.55469 exp=1028.69928 shr=5.20611351 lines=322 END +moesel call=130 dev=0 T=1 E=512 k=10 sel=604f7f5982f7ec3a minulps=1 mintok=0 x=1492.01486 logit=3444.32812 exp=916.953086 shr=5.96958305 lines=323 END +moesel call=131 dev=0 T=1 E=512 k=10 sel=3f92392a733cf89b minulps=1 mintok=0 x=1694.0605 logit=3263.1875 exp=1261.00149 shr=7.00404574 lines=324 END +moesel call=132 dev=0 T=1 E=512 k=10 sel=e47c8261c4d73d43 minulps=1 mintok=0 x=2043.89637 logit=3133.75 exp=2080.5263 shr=23.460304 lines=325 END +moesel call=133 dev=0 T=1 E=512 k=10 sel=32b34a4857d3034a minulps=0 mintok=0 x=1972.11004 logit=3223.15625 exp=1378.16079 shr=10.0461127 lines=326 END +moesel call=134 dev=0 T=1 E=512 k=10 sel=ecfcaac740e8eb3e minulps=0 mintok=0 x=1891.92782 logit=3404.23438 exp=1795.02391 shr=14.3305215 lines=327 END +moesel call=135 dev=0 T=1 E=512 k=10 sel=3defa2c2ad1a19c9 minulps=0 mintok=0 x=2030.99916 logit=3141.89062 exp=1735.56049 shr=17.3080959 lines=328 END +moesel call=136 dev=0 T=1 E=512 k=10 sel=9d4c4d575530aaa1 minulps=2 mintok=0 x=2015.94075 logit=3064.125 exp=1498.87653 shr=14.0980233 lines=329 END +moesel call=137 dev=0 T=1 E=512 k=10 sel=a2ca04f2c32af24c minulps=5 mintok=0 x=2164.04088 logit=3173.60938 exp=1476.40351 shr=13.9556701 lines=330 END +moesel call=138 dev=0 T=1 E=512 k=10 sel=2c1052d79fe8a01c minulps=3 mintok=0 x=2116.30283 logit=3201.82812 exp=1908.28173 shr=19.9749909 lines=331 END +moesel call=139 dev=0 T=1 E=512 k=10 sel=63dfce62a1968997 minulps=8 mintok=0 x=1825.40135 logit=3097.98438 exp=1359.85714 shr=15.7230028 lines=332 END +moesel call=140 dev=0 T=1 E=512 k=10 sel=f3c6269367da1868 minulps=1 mintok=0 x=2168.14714 logit=3176.79688 exp=1596.21876 shr=23.0226604 lines=333 END +moesel call=141 dev=0 T=1 E=512 k=10 sel=acb23a0e28d2ff50 minulps=4 mintok=0 x=2369.13889 logit=3233.3125 exp=1501.22406 shr=26.2855051 lines=334 END +moesel call=142 dev=0 T=1 E=512 k=10 sel=7921d1ce05dc1780 minulps=2 mintok=0 x=2269.42081 logit=3240.53125 exp=1456.38467 shr=38.3288927 lines=335 END +moesel call=143 dev=0 T=1 E=512 k=10 sel=cb5ae746a6e98d0f minulps=2 mintok=0 x=1996.28304 logit=3240.89062 exp=1482.50299 shr=133.604757 lines=336 END +moesel call=144 dev=0 T=1 E=512 k=10 sel=64e4241874f1df00 minulps=2 mintok=0 x=704.012409 logit=3391.01562 exp=554.235773 shr=42.2263086 lines=337 END +moesel call=145 dev=0 T=1 E=512 k=10 sel=6e61ac8029b0eb57 minulps=4 mintok=0 x=757.554578 logit=3137.48438 exp=337.795793 shr=8.5666878 lines=338 END +moesel call=146 dev=0 T=1 E=512 k=10 sel=b08fb8a7f3061f39 minulps=1 mintok=0 x=790.976817 logit=3146.625 exp=386.331875 shr=8.11794358 lines=339 END +moesel call=147 dev=0 T=1 E=512 k=10 sel=cc2de51b46bf37b7 minulps=0 mintok=0 x=1030.9344 logit=3043.8125 exp=505.468475 shr=4.88330396 lines=340 END +moesel call=148 dev=0 T=1 E=512 k=10 sel=bb9732cb39f87a45 minulps=1 mintok=0 x=1026.28395 logit=3019.15625 exp=578.007647 shr=5.46301039 lines=341 END +moesel call=149 dev=0 T=1 E=512 k=10 sel=792156da8816e499 minulps=2 mintok=0 x=709.238798 logit=3039.48438 exp=305.731701 shr=4.0652933 lines=342 END +moesel call=150 dev=0 T=1 E=512 k=10 sel=adfa331c6b9ef442 minulps=2 mintok=0 x=1114.27743 logit=3156.125 exp=638.692997 shr=7.41232707 lines=343 END +moesel call=151 dev=0 T=1 E=512 k=10 sel=6ef63e64b3549c52 minulps=2 mintok=0 x=1072.8511 logit=3133.23438 exp=556.26751 shr=6.92626507 lines=344 END +moesel call=152 dev=0 T=1 E=512 k=10 sel=ea6f7ba943a4bce2 minulps=3 mintok=0 x=1066.75546 logit=3010.625 exp=561.820343 shr=5.50392725 lines=345 END +moesel call=153 dev=0 T=1 E=512 k=10 sel=cc820632974d0d21 minulps=1 mintok=0 x=1078.96722 logit=3054.46875 exp=567.575908 shr=5.9065972 lines=346 END +moesel call=154 dev=0 T=1 E=512 k=10 sel=c3ac69b7b9be9b4d minulps=4 mintok=0 x=1023.90607 logit=3072.53125 exp=473.886277 shr=4.85309371 lines=347 END +moesel call=155 dev=0 T=1 E=512 k=10 sel=2c4587f6d923075b minulps=1 mintok=0 x=993.635571 logit=3007.59375 exp=412.653947 shr=4.33550698 lines=348 END +moesel call=156 dev=0 T=1 E=512 k=10 sel=7368df348dda0d2a minulps=0 mintok=0 x=1067.73772 logit=3005.45312 exp=530.45088 shr=6.58520544 lines=349 END +moesel call=157 dev=0 T=1 E=512 k=10 sel=13f5f4bed552fb78 minulps=1 mintok=0 x=980.486447 logit=3051.84375 exp=474.45902 shr=4.74441737 lines=350 END +moesel call=158 dev=0 T=1 E=512 k=10 sel=415c63615f164733 minulps=0 mintok=0 x=1025.23302 logit=2934.09375 exp=550.762955 shr=5.26887361 lines=351 END +moesel call=159 dev=0 T=1 E=512 k=10 sel=6e558f11c4013434 minulps=2 mintok=0 x=1471.486 logit=3099.71875 exp=864.215507 shr=5.8337274 lines=352 END +moesel call=160 dev=0 T=1 E=512 k=10 sel=64ec13a3c0245f29 minulps=2 mintok=0 x=1317.34324 logit=3044.51562 exp=692.437795 shr=5.36932105 lines=353 END +moesel call=161 dev=0 T=1 E=512 k=10 sel=5f4f3c990d22bb31 minulps=2 mintok=0 x=1098.57551 logit=2963.70312 exp=524.686988 shr=5.79104155 lines=354 END +moesel call=162 dev=0 T=1 E=512 k=10 sel=868a69c6fe12189a minulps=1 mintok=0 x=1154.29364 logit=3161.8125 exp=625.385319 shr=7.06885046 lines=355 END +moesel call=163 dev=0 T=1 E=512 k=10 sel=bc12ac54ecf1f835 minulps=7 mintok=0 x=1273.37386 logit=3102.66406 exp=655.353976 shr=5.84881203 lines=356 END +moesel call=164 dev=0 T=1 E=512 k=10 sel=188ebb0c0705055f minulps=1 mintok=0 x=1614.69366 logit=3177.42188 exp=1057.78862 shr=14.9799776 lines=357 END +moesel call=165 dev=0 T=1 E=512 k=10 sel=80088ed263e76837 minulps=6 mintok=0 x=1467.51619 logit=3145.3125 exp=836.37563 shr=7.55925656 lines=358 END +moesel call=166 dev=0 T=1 E=512 k=10 sel=894f039c5a8b4ba4 minulps=2 mintok=0 x=1403.07147 logit=3254.46875 exp=800.448175 shr=7.98383042 lines=359 END +moesel call=167 dev=0 T=1 E=512 k=10 sel=eb8d8255967fecf4 minulps=4 mintok=0 x=1737.49875 logit=3127.09375 exp=1053.51103 shr=18.5937283 lines=360 END +moesel call=168 dev=0 T=1 E=512 k=10 sel=735f79b421393331 minulps=1 mintok=0 x=1177.85914 logit=3191.78125 exp=571.974418 shr=6.57680889 lines=361 END +moesel call=169 dev=0 T=1 E=512 k=10 sel=e6e2e22937550012 minulps=9 mintok=0 x=1153.27574 logit=3059.17188 exp=541.296287 shr=6.18206786 lines=362 END +moesel call=170 dev=0 T=1 E=512 k=10 sel=4077e415e23aed88 minulps=6 mintok=0 x=1091.12849 logit=3117.5625 exp=492.27847 shr=4.54085315 lines=363 END +moesel call=171 dev=0 T=1 E=512 k=10 sel=52d591d1166fcd66 minulps=11 mintok=0 x=1128.09003 logit=3078.46875 exp=490.031421 shr=4.4073793 lines=364 END +moesel call=172 dev=0 T=1 E=512 k=10 sel=d5b7d59b693addc7 minulps=1 mintok=0 x=1246.31363 logit=3108.90625 exp=628.32774 shr=6.3387332 lines=365 END +moesel call=173 dev=0 T=1 E=512 k=10 sel=a76f28caa9389c5c minulps=4 mintok=0 x=1237.95625 logit=3147.45312 exp=591.657497 shr=6.3675811 lines=366 END +moesel call=174 dev=0 T=1 E=512 k=10 sel=2a650cf963392c95 minulps=1 mintok=0 x=1344.51108 logit=3119.4375 exp=783.550203 shr=8.529039 lines=367 END +moesel call=175 dev=0 T=1 E=512 k=10 sel=67b69a4ebc1103c6 minulps=2 mintok=0 x=1763.59213 logit=3157.35938 exp=1111.8995 shr=7.47240212 lines=368 END +moesel call=176 dev=0 T=1 E=512 k=10 sel=b389ccc0f73e7d0a minulps=5 mintok=0 x=1747.07535 logit=3114.29688 exp=1104.01993 shr=9.85640764 lines=369 END +moesel call=177 dev=0 T=1 E=512 k=10 sel=d653c3f146c745db minulps=2 mintok=0 x=1576.18927 logit=3189.375 exp=1020.53586 shr=6.87306631 lines=370 END +moesel call=178 dev=0 T=1 E=512 k=10 sel=b33be1e81fca8628 minulps=0 mintok=0 x=1482.88848 logit=3291.57812 exp=902.234411 shr=7.58216667 lines=371 END +moesel call=179 dev=0 T=1 E=512 k=10 sel=126affb7f16f1935 minulps=1 mintok=0 x=1768.0564 logit=3204.6875 exp=1384.13685 shr=7.81168029 lines=372 END +moesel call=180 dev=0 T=1 E=512 k=10 sel=a581d13c9085d7bd minulps=1 mintok=0 x=2256.299 logit=3255.92188 exp=2356.2561 shr=22.8342528 lines=373 END +moesel call=181 dev=0 T=1 E=512 k=10 sel=1fc2a0f7cc1d336a minulps=0 mintok=0 x=2140.51995 logit=3245.95312 exp=1706.35723 shr=18.3786964 lines=374 END +moesel call=182 dev=0 T=1 E=512 k=10 sel=ff4fd74bb3dc6518 minulps=0 mintok=0 x=2056.02953 logit=3287.51562 exp=2182.11039 shr=22.1919606 lines=375 END +moesel call=183 dev=0 T=1 E=512 k=10 sel=1378d58df87832ca minulps=6 mintok=0 x=2265.28398 logit=3070.65625 exp=1908.95659 shr=10.7998445 lines=376 END +moesel call=184 dev=0 T=1 E=512 k=10 sel=eaa3ad093aa15a51 minulps=1 mintok=0 x=2321.61115 logit=3115.9375 exp=1922.93207 shr=14.6389322 lines=377 END +moesel call=185 dev=0 T=1 E=512 k=10 sel=e640c6edc8997ed6 minulps=5 mintok=0 x=2479.74604 logit=3166.71875 exp=2057.65835 shr=14.9026581 lines=378 END +moesel call=186 dev=0 T=1 E=512 k=10 sel=5c54b1dbef71949b minulps=1 mintok=0 x=2383.45681 logit=3251.1875 exp=2245.25949 shr=21.1885175 lines=379 END +moesel call=187 dev=0 T=1 E=512 k=10 sel=f949b3520df67388 minulps=0 mintok=0 x=1985.20757 logit=3131.78125 exp=1416.58352 shr=13.3448344 lines=380 END +moesel call=188 dev=0 T=1 E=512 k=10 sel=8ffcb0ae131665b6 minulps=2 mintok=0 x=2426.90604 logit=3164.84375 exp=1845.1169 shr=21.1736459 lines=381 END +moesel call=189 dev=0 T=1 E=512 k=10 sel=4b3c8e6dc667dd7f minulps=2 mintok=0 x=2495.70905 logit=3197.51562 exp=1765.98206 shr=20.5617286 lines=382 END +moesel call=190 dev=0 T=1 E=512 k=10 sel=b0a8ea541f8bd42e minulps=1 mintok=0 x=2445.78047 logit=3187.85938 exp=1627.32222 shr=33.7317171 lines=383 END +moesel call=191 dev=0 T=1 E=512 k=10 sel=b72a0bf0cb2ae7f4 minulps=7 mintok=0 x=2395.1438 logit=3300.82812 exp=1680.61398 shr=89.5329192 lines=384 END +moesel call=192 dev=0 T=1 E=512 k=10 sel=4a58e59eedd05614 minulps=6 mintok=0 x=713.292795 logit=3547.25 exp=495.988081 shr=46.8861551 lines=385 END +moesel call=193 dev=0 T=1 E=512 k=10 sel=3c10167bba9572f3 minulps=5 mintok=0 x=809.619069 logit=3187.96875 exp=437.949026 shr=6.4195975 lines=386 END +moesel call=194 dev=0 T=1 E=512 k=10 sel=f3629bf962962ed4 minulps=1 mintok=0 x=842.832673 logit=3179.79688 exp=376.089623 shr=8.64113953 lines=387 END +moesel call=195 dev=0 T=1 E=512 k=10 sel=d55bf8389a54c9fb minulps=1 mintok=0 x=1078.46139 logit=3033.20312 exp=590.782449 shr=5.92302778 lines=388 END +moesel call=196 dev=0 T=1 E=512 k=10 sel=36dc03036fc7c978 minulps=0 mintok=0 x=1060.07721 logit=3011.92188 exp=582.046325 shr=5.94607417 lines=389 END +moesel call=197 dev=0 T=1 E=512 k=10 sel=a88e6a7d9a5a07dd minulps=1 mintok=0 x=949.684791 logit=3075.35938 exp=394.444535 shr=5.0281306 lines=390 END +moesel call=198 dev=0 T=1 E=512 k=10 sel=1bde0ab8790b3a9f minulps=5 mintok=0 x=1144.69436 logit=3118.64062 exp=691.466931 shr=6.24584097 lines=391 END +moesel call=199 dev=0 T=1 E=512 k=10 sel=a5901fcade5aeb5a minulps=6 mintok=0 x=1084.59545 logit=3181.70312 exp=514.253633 shr=4.58468512 lines=392 END +moesel call=200 dev=0 T=1 E=512 k=10 sel=651be32f40ba2189 minulps=2 mintok=0 x=1094.96036 logit=3051.125 exp=592.097759 shr=5.58841326 lines=393 END +moesel call=201 dev=0 T=1 E=512 k=10 sel=4417b1db3d817d3e minulps=0 mintok=0 x=1118.3233 logit=3082.14062 exp=602.553823 shr=5.69393443 lines=394 END +moesel call=202 dev=0 T=1 E=512 k=10 sel=58ed711438b36896 minulps=0 mintok=0 x=1041.59428 logit=3158.1875 exp=505.892227 shr=4.74559257 lines=395 END +moesel call=203 dev=0 T=1 E=512 k=10 sel=2798c511b0ca9ac7 minulps=1 mintok=0 x=1051.18178 logit=3081.125 exp=525.968272 shr=4.96797041 lines=396 END +moesel call=204 dev=0 T=1 E=512 k=10 sel=d6dbe34036838bfe minulps=6 mintok=0 x=1101.62494 logit=3145.875 exp=496.542189 shr=5.36030903 lines=397 END +moesel call=205 dev=0 T=1 E=512 k=10 sel=267cf3e13ec62502 minulps=9 mintok=0 x=1023.10753 logit=3136.07812 exp=503.564436 shr=4.55933059 lines=398 END +moesel call=206 dev=0 T=1 E=512 k=10 sel=4d0bd04cb548f074 minulps=3 mintok=0 x=1128.66985 logit=3069.54688 exp=595.28454 shr=6.81580678 lines=399 END +moesel call=207 dev=0 T=1 E=512 k=10 sel=0018511c7cea8f6f minulps=1 mintok=0 x=1461.56197 logit=3111.71875 exp=804.730988 shr=7.45506278 lines=400 END +moesel call=208 dev=0 T=1 E=512 k=10 sel=038c91e55b73dfd6 minulps=6 mintok=0 x=1295.11488 logit=3057.35938 exp=653.777343 shr=5.7832199 lines=401 END +moesel call=209 dev=0 T=1 E=512 k=10 sel=20af3977d7677193 minulps=1 mintok=0 x=1080.79119 logit=2994.48438 exp=479.642704 shr=5.36518809 lines=402 END +moesel call=210 dev=0 T=1 E=512 k=10 sel=ee831f5d0cf832ce minulps=2 mintok=0 x=1135.78924 logit=3196.26562 exp=548.740983 shr=5.72230443 lines=403 END +moesel call=211 dev=0 T=1 E=512 k=10 sel=e2ce040888d1bb11 minulps=1 mintok=0 x=1256.81571 logit=3183.99219 exp=654.013566 shr=5.91730656 lines=404 END +moesel call=212 dev=0 T=1 E=512 k=10 sel=447e7ca666c6e126 minulps=0 mintok=0 x=1552.87338 logit=3278.48438 exp=1149.22229 shr=12.3917163 lines=405 END +moesel call=213 dev=0 T=1 E=512 k=10 sel=9d5fa6a60280a40a minulps=1 mintok=0 x=1460.57157 logit=3178.59375 exp=837.561507 shr=6.90216361 lines=406 END +moesel call=214 dev=0 T=1 E=512 k=10 sel=f0dbba8e2b188530 minulps=11 mintok=0 x=1435.35947 logit=3347.32812 exp=807.203559 shr=6.9085494 lines=407 END +moesel call=215 dev=0 T=1 E=512 k=10 sel=66c0f1f0479ffadb minulps=1 mintok=0 x=1606.52447 logit=3155.17188 exp=1033.16949 shr=15.3148386 lines=408 END +moesel call=216 dev=0 T=1 E=512 k=10 sel=cc510b9c49ed2d00 minulps=1 mintok=0 x=1161.21321 logit=3186.01562 exp=583.139586 shr=5.90356094 lines=409 END +moesel call=217 dev=0 T=1 E=512 k=10 sel=ac32ad67d4bd751a minulps=3 mintok=0 x=1120.68004 logit=3077.76562 exp=534.875565 shr=5.12656626 lines=410 END +moesel call=218 dev=0 T=1 E=512 k=10 sel=959ffa2010515604 minulps=9 mintok=0 x=1126.36991 logit=3185.39062 exp=536.596135 shr=4.70694034 lines=411 END +moesel call=219 dev=0 T=1 E=512 k=10 sel=6b67e7dc07b53f5b minulps=3 mintok=0 x=1124.05293 logit=3063.07812 exp=487.673697 shr=4.76643467 lines=412 END +moesel call=220 dev=0 T=1 E=512 k=10 sel=3ac2ba29dde59d39 minulps=3 mintok=0 x=1191.57999 logit=3152.03125 exp=561.666117 shr=4.72839332 lines=413 END +moesel call=221 dev=0 T=1 E=512 k=10 sel=ce99874228f254b0 minulps=5 mintok=0 x=1197.95257 logit=3152.48438 exp=596.825586 shr=5.33076208 lines=414 END +moesel call=222 dev=0 T=1 E=512 k=10 sel=3af0aa8ba3838d70 minulps=0 mintok=0 x=1273.0381 logit=3064.42188 exp=721.419823 shr=9.51686171 lines=415 END +moesel call=223 dev=0 T=1 E=512 k=10 sel=ad77c8d45c7ce9c0 minulps=1 mintok=0 x=1523.41024 logit=3128.51562 exp=887.931278 shr=10.2387804 lines=416 END +moesel call=224 dev=0 T=1 E=512 k=10 sel=252c7c0de8c512b2 minulps=4 mintok=0 x=1497.84688 logit=3076.26562 exp=823.353848 shr=11.8247422 lines=417 END +moesel call=225 dev=0 T=1 E=512 k=10 sel=c320c05ede9fc421 minulps=1 mintok=0 x=1461.94109 logit=3160.96875 exp=918.791436 shr=7.01439513 lines=418 END +moesel call=226 dev=0 T=1 E=512 k=10 sel=89b01295228b1b96 minulps=2 mintok=0 x=1368.63893 logit=3252.625 exp=827.248699 shr=8.87961981 lines=419 END +moesel call=227 dev=0 T=1 E=512 k=10 sel=c198c77e26eea249 minulps=1 mintok=0 x=1678.00058 logit=3194.26562 exp=1209.90576 shr=11.5028279 lines=420 END +moesel call=228 dev=0 T=1 E=512 k=10 sel=e18856bce8825f99 minulps=1 mintok=0 x=2098.54796 logit=3199.03125 exp=2244.56434 shr=22.519245 lines=421 END +moesel call=229 dev=0 T=1 E=512 k=10 sel=869de8075d997083 minulps=0 mintok=0 x=1962.59503 logit=3198.96875 exp=1661.12753 shr=15.560072 lines=422 END +moesel call=230 dev=0 T=1 E=512 k=10 sel=733d8575121d937d minulps=3 mintok=0 x=1927.04253 logit=3310.73438 exp=2137.46422 shr=19.1797356 lines=423 END +moesel call=231 dev=0 T=1 E=512 k=10 sel=8eae380edd67ae35 minulps=4 mintok=0 x=1951.92702 logit=3038.76562 exp=1906.98826 shr=20.5522859 lines=424 END +moesel call=232 dev=0 T=1 E=512 k=10 sel=5cf08d1ad6f1fb7a minulps=0 mintok=0 x=1954.75556 logit=3016.32812 exp=1524.3282 shr=22.0171233 lines=425 END +moesel call=233 dev=0 T=1 E=512 k=10 sel=57b5aaeccae2daf3 minulps=0 mintok=0 x=2030.80098 logit=3131.5625 exp=1501.26161 shr=17.2168976 lines=426 END +moesel call=234 dev=0 T=1 E=512 k=10 sel=2a899308e2b70ab3 minulps=1 mintok=0 x=2148.25961 logit=3215.17188 exp=2059.23845 shr=22.0976569 lines=427 END +moesel call=235 dev=0 T=1 E=512 k=10 sel=fe1acb1931b52830 minulps=0 mintok=0 x=1667.36501 logit=3090.6875 exp=1054.48358 shr=18.6527985 lines=428 END +moesel call=236 dev=0 T=1 E=512 k=10 sel=38220bd5b93d987a minulps=0 mintok=0 x=2100.92886 logit=3219.29688 exp=1510.38157 shr=18.487669 lines=429 END +moesel call=237 dev=0 T=1 E=512 k=10 sel=16b9ebe672a7440b minulps=0 mintok=0 x=2252.26691 logit=3289.04688 exp=1527.21831 shr=20.6977258 lines=430 END +moesel call=238 dev=0 T=1 E=512 k=10 sel=4585ce64901c1120 minulps=1 mintok=0 x=2225.3584 logit=3155.09375 exp=1539.90151 shr=28.5997086 lines=431 END +moesel call=239 dev=0 T=1 E=512 k=10 sel=aeab7eb7aaad1d90 minulps=3 mintok=0 x=2125.31551 logit=3224.57812 exp=1246.8143 shr=108.693364 lines=432 END +moesel call=240 dev=0 T=1 E=512 k=10 sel=313e8aa1f8b2371d minulps=0 mintok=0 x=679.889969 logit=3463.76562 exp=567.10042 shr=37.5760253 lines=433 END +moesel call=241 dev=0 T=1 E=512 k=10 sel=1a1dea2cb93dc617 minulps=3 mintok=0 x=860.278532 logit=3185.25 exp=463.900769 shr=5.37393649 lines=434 END +moesel call=242 dev=0 T=1 E=512 k=10 sel=80d46976e8832350 minulps=6 mintok=0 x=874.798262 logit=3168.625 exp=399.728898 shr=7.68230708 lines=435 END +moesel call=243 dev=0 T=1 E=512 k=10 sel=d65b645151f3cadd minulps=3 mintok=0 x=1092.09847 logit=3000.07812 exp=660.582582 shr=8.39706464 lines=436 END +moesel call=244 dev=0 T=1 E=512 k=10 sel=ade897a8ca71a377 minulps=2 mintok=0 x=1134.79967 logit=3062.0625 exp=625.690093 shr=10.9471523 lines=437 END +moesel call=245 dev=0 T=1 E=512 k=10 sel=3337f7727d302517 minulps=3 mintok=0 x=1007.48689 logit=3076.28125 exp=437.289432 shr=9.92289427 lines=438 END +moesel call=246 dev=0 T=1 E=512 k=10 sel=6ced9e0fbb9734bc minulps=4 mintok=0 x=1147.92157 logit=3179.40625 exp=681.492524 shr=6.15881637 lines=439 END +moesel call=247 dev=0 T=1 E=512 k=10 sel=8c3ef3b0d0ae7913 minulps=2 mintok=0 x=1107.90611 logit=3146.95312 exp=569.927384 shr=6.37451947 lines=440 END +moesel call=248 dev=0 T=1 E=512 k=10 sel=8cdfdfb77e408218 minulps=0 mintok=0 x=1115.51341 logit=3023.89062 exp=583.387254 shr=6.30009348 lines=441 END +moesel call=249 dev=0 T=1 E=512 k=10 sel=1a5ebbf8bccb68ee minulps=2 mintok=0 x=1162.72114 logit=3037.25 exp=655.106069 shr=5.92989672 lines=442 END +moesel call=250 dev=0 T=1 E=512 k=10 sel=5c7702307f31fdc8 minulps=1 mintok=0 x=1063.1795 logit=3063.39062 exp=565.082634 shr=4.69016513 lines=443 END +moesel call=251 dev=0 T=1 E=512 k=10 sel=7d3020228600a698 minulps=0 mintok=0 x=1074.89305 logit=3028.03125 exp=531.726752 shr=4.95197909 lines=444 END +moesel call=252 dev=0 T=1 E=512 k=10 sel=33d71b1a2e5ef978 minulps=0 mintok=0 x=1116.43097 logit=3098.85938 exp=513.639663 shr=6.05464247 lines=445 END +moesel call=253 dev=0 T=1 E=512 k=10 sel=5dc669be1db0220c minulps=2 mintok=0 x=1001.71599 logit=3008.79688 exp=462.334784 shr=5.82557359 lines=446 END +moesel call=254 dev=0 T=1 E=512 k=10 sel=d8cce58214282a56 minulps=1 mintok=0 x=1126.47991 logit=3002.5625 exp=560.081899 shr=7.61708035 lines=447 END +moesel call=255 dev=0 T=1 E=512 k=10 sel=d606b23cad97fc89 minulps=5 mintok=0 x=1676.88349 logit=3134.4375 exp=1156.8899 shr=7.20779438 lines=448 END +moesel call=256 dev=0 T=1 E=512 k=10 sel=55d784c5cd482218 minulps=8 mintok=0 x=1369.21154 logit=3017.04688 exp=753.026495 shr=6.38711438 lines=449 END +moesel call=257 dev=0 T=1 E=512 k=10 sel=2647c20ba371219d minulps=0 mintok=0 x=1191.20086 logit=3023.98438 exp=562.921191 shr=5.45886013 lines=450 END +moesel call=258 dev=0 T=1 E=512 k=10 sel=db85e3b84703f782 minulps=2 mintok=0 x=1192.62422 logit=3201.9375 exp=617.203346 shr=6.92112707 lines=451 END +moesel call=259 dev=0 T=1 E=512 k=10 sel=d554b88e910dd8f9 minulps=1 mintok=0 x=1313.76158 logit=3109.86719 exp=654.178819 shr=6.85460453 lines=452 END +moesel call=260 dev=0 T=1 E=512 k=10 sel=3cebd5841feff49c minulps=1 mintok=0 x=1748.36034 logit=3199.75 exp=1200.58362 shr=14.2438044 lines=453 END +moesel call=261 dev=0 T=1 E=512 k=10 sel=ff271e465447dc87 minulps=7 mintok=0 x=1475.9109 logit=3105.01562 exp=882.76117 shr=11.476743 lines=454 END +moesel call=262 dev=0 T=1 E=512 k=10 sel=2e554904dd76e57e minulps=3 mintok=0 x=1405.51104 logit=3312.8125 exp=765.863996 shr=9.73312861 lines=455 END +moesel call=263 dev=0 T=1 E=512 k=10 sel=585837905fa10257 minulps=7 mintok=0 x=1760.61483 logit=3180.46875 exp=1084.86851 shr=11.8500076 lines=456 END +moesel call=264 dev=0 T=1 E=512 k=10 sel=b63cd6085d2a092e minulps=0 mintok=0 x=1230.74654 logit=3169.21875 exp=635.532898 shr=7.00597061 lines=457 END +moesel call=265 dev=0 T=1 E=512 k=10 sel=223462b18d423d6e minulps=9 mintok=0 x=1138.43392 logit=2991.09375 exp=511.542922 shr=5.27611259 lines=458 END +moesel call=266 dev=0 T=1 E=512 k=10 sel=69fa182ee332548d minulps=9 mintok=0 x=1172.15502 logit=3157.23438 exp=546.535292 shr=5.93225051 lines=459 END +moesel call=267 dev=0 T=1 E=512 k=10 sel=8e36dc807fb6d776 minulps=2 mintok=0 x=1198.90388 logit=3112 exp=550.252909 shr=5.44168846 lines=460 END +moesel call=268 dev=0 T=1 E=512 k=10 sel=c908d12bb6ded22f minulps=11 mintok=0 x=1248.57796 logit=3207.45312 exp=584.063421 shr=5.4245027 lines=461 END +moesel call=269 dev=0 T=1 E=512 k=10 sel=951bf44f7b6351d4 minulps=8 mintok=0 x=1254.04182 logit=3142.57031 exp=609.852381 shr=7.00630525 lines=462 END +moesel call=270 dev=0 T=1 E=512 k=10 sel=4a6c62a92417fb07 minulps=0 mintok=0 x=1443.55998 logit=3153.78125 exp=831.783232 shr=8.63710169 lines=463 END +moesel call=271 dev=0 T=1 E=512 k=10 sel=464daf12dd534cb9 minulps=0 mintok=0 x=1728.59673 logit=3136.89062 exp=1032.99495 shr=8.16373086 lines=464 END +moesel call=272 dev=0 T=1 E=512 k=10 sel=1c55e95e26d65235 minulps=2 mintok=0 x=1672.97059 logit=3087.75 exp=966.479639 shr=10.3026717 lines=465 END +moesel call=273 dev=0 T=1 E=512 k=10 sel=5d6e0fbdc5007cb5 minulps=2 mintok=0 x=1541.22878 logit=3202.98438 exp=991.586527 shr=7.00148208 lines=466 END +moesel call=274 dev=0 T=1 E=512 k=10 sel=40b206eb41aaee1b minulps=1 mintok=0 x=1450.42547 logit=3270.01562 exp=967.231181 shr=9.64170421 lines=467 END +moesel call=275 dev=0 T=1 E=512 k=10 sel=74fac48c1e2c31f6 minulps=1 mintok=0 x=1719.49511 logit=3271.76562 exp=1267.3607 shr=9.73379068 lines=468 END +moesel call=276 dev=0 T=1 E=512 k=10 sel=67aeeae4008e2417 minulps=1 mintok=0 x=2059.33632 logit=3192.67188 exp=1810.40256 shr=20.807757 lines=469 END +moesel call=277 dev=0 T=1 E=512 k=10 sel=2e20000d94e9b920 minulps=1 mintok=0 x=1968.24071 logit=3161.92188 exp=1820.24125 shr=17.4798741 lines=470 END +moesel call=278 dev=0 T=1 E=512 k=10 sel=fafa110131150eab minulps=3 mintok=0 x=1889.36966 logit=3200.65625 exp=1755.15098 shr=23.6713966 lines=471 END +moesel call=279 dev=0 T=1 E=512 k=10 sel=756c12fd46a83e14 minulps=0 mintok=0 x=2077.86097 logit=3063.20312 exp=1900.61443 shr=19.1389007 lines=472 END +moesel call=280 dev=0 T=1 E=512 k=10 sel=1d6e1ee1d12b1dbb minulps=0 mintok=0 x=2124.35651 logit=3064.42188 exp=1825.3641 shr=19.3226681 lines=473 END +moesel call=281 dev=0 T=1 E=512 k=10 sel=3d86cc3ed6bc1119 minulps=2 mintok=0 x=2252.72728 logit=3196.1875 exp=1895.10492 shr=20.5761556 lines=474 END +moesel call=282 dev=0 T=1 E=512 k=10 sel=7c852893544cd14d minulps=0 mintok=0 x=1977.10798 logit=3096.79688 exp=2173.13381 shr=26.9104897 lines=475 END +moesel call=283 dev=0 T=1 E=512 k=10 sel=02b087e5c0a5b528 minulps=2 mintok=0 x=1771.88104 logit=3096.0625 exp=1201.84482 shr=15.8606838 lines=476 END +moesel call=284 dev=0 T=1 E=512 k=10 sel=9b240facc3be5e58 minulps=1 mintok=0 x=2235.23505 logit=3167.04688 exp=1786.89934 shr=22.6928716 lines=477 END +moesel call=285 dev=0 T=1 E=512 k=10 sel=8560a07d382f356b minulps=0 mintok=0 x=2439.82397 logit=3225.67188 exp=1591.14833 shr=23.9335559 lines=478 END +moesel call=286 dev=0 T=1 E=512 k=10 sel=dc5a384394b3345e minulps=0 mintok=0 x=2250.02311 logit=3232.1875 exp=1454.84173 shr=33.1123497 lines=479 END +moesel call=287 dev=0 T=1 E=512 k=10 sel=25f74f64ea240606 minulps=0 mintok=0 x=1986.24271 logit=3244.15625 exp=1190.10885 shr=110.485694 lines=480 END +moesel call=288 dev=0 T=1 E=512 k=10 sel=65c4a1664e07f708 minulps=1 mintok=0 x=725.714377 logit=3479.07812 exp=574.053736 shr=54.5889122 lines=481 END +moesel call=289 dev=0 T=1 E=512 k=10 sel=d9553908682b329c minulps=2 mintok=0 x=698.395152 logit=3100.6875 exp=395.024348 shr=4.60022465 lines=482 END +moesel call=290 dev=0 T=1 E=512 k=10 sel=283ad63a8ed7b442 minulps=0 mintok=0 x=781.103161 logit=3124.78125 exp=355.528568 shr=5.40893444 lines=483 END +moesel call=291 dev=0 T=1 E=512 k=10 sel=9b9c4f4a845881ed minulps=6 mintok=0 x=1026.58546 logit=3054.29688 exp=624.412025 shr=5.17255085 lines=484 END +moesel call=292 dev=0 T=1 E=512 k=10 sel=7a1ea1e6ba54a448 minulps=8 mintok=0 x=1050.63661 logit=2992.46875 exp=508.878341 shr=5.27879809 lines=485 END +moesel call=293 dev=0 T=1 E=512 k=10 sel=16b57f7a8b628201 minulps=4 mintok=0 x=904.385968 logit=3091.84375 exp=405.241234 shr=3.92045931 lines=486 END +moesel call=294 dev=0 T=1 E=512 k=10 sel=ac8bbfbea67faeff minulps=3 mintok=0 x=1070.35198 logit=3126.20312 exp=586.737313 shr=6.37898166 lines=487 END +moesel call=295 dev=0 T=1 E=512 k=10 sel=d9343c9619a6bfec minulps=4 mintok=0 x=1042.55654 logit=3152.40625 exp=524.855719 shr=4.48317985 lines=488 END +moesel call=296 dev=0 T=1 E=512 k=10 sel=8f70a9093f81cd98 minulps=1 mintok=0 x=1043.43202 logit=3019.78125 exp=537.966236 shr=5.05144785 lines=489 END +moesel call=297 dev=0 T=1 E=512 k=10 sel=d2035c730d97d8bb minulps=3 mintok=0 x=1075.2371 logit=3033.375 exp=596.580955 shr=6.06830943 lines=490 END +moesel call=298 dev=0 T=1 E=512 k=10 sel=8152ea1786d24b85 minulps=5 mintok=0 x=1040.69753 logit=3057.45312 exp=501.279944 shr=4.24556739 lines=491 END +moesel call=299 dev=0 T=1 E=512 k=10 sel=17af1b8fdc35b64e minulps=0 mintok=0 x=1073.69147 logit=3067.95312 exp=525.250522 shr=4.67117937 lines=492 END +moesel call=300 dev=0 T=1 E=512 k=10 sel=9a5dcd8f9cd8779c minulps=4 mintok=0 x=1100.9368 logit=3071.82812 exp=544.323194 shr=5.98198618 lines=493 END +moesel call=301 dev=0 T=1 E=512 k=10 sel=2969ce41bf41fb50 minulps=4 mintok=0 x=1057.58238 logit=3039.5625 exp=531.945031 shr=5.90877565 lines=494 END +moesel call=302 dev=0 T=1 E=512 k=10 sel=db7200e9d5b99318 minulps=3 mintok=0 x=1104.32088 logit=3009.09375 exp=558.763368 shr=7.39087255 lines=495 END +moesel call=303 dev=0 T=1 E=512 k=10 sel=1d93d36f3c11d1c8 minulps=1 mintok=0 x=1502.52884 logit=3103.98438 exp=1063.21384 shr=7.18970741 lines=496 END +moesel call=304 dev=0 T=1 E=512 k=10 sel=87a86dfce31576b9 minulps=3 mintok=0 x=1347.74365 logit=3053.71875 exp=753.566094 shr=7.00670701 lines=497 END +moesel call=305 dev=0 T=1 E=512 k=10 sel=41cc0b6a1718c880 minulps=4 mintok=0 x=1164.14633 logit=2986.84375 exp=547.457562 shr=5.73651992 lines=498 END +moesel call=306 dev=0 T=1 E=512 k=10 sel=de1b62ef14aa78de minulps=2 mintok=0 x=1134.59466 logit=3167.23438 exp=574.346282 shr=6.03064606 lines=499 END +moesel call=307 dev=0 T=1 E=512 k=10 sel=1b3784779dc0af8f minulps=0 mintok=0 x=1316.92364 logit=3119.42188 exp=694.134603 shr=7.14131635 lines=500 END +moesel call=308 dev=0 T=1 E=512 k=10 sel=2a23e5d32125c71f minulps=0 mintok=0 x=1702.7851 logit=3166.625 exp=1230.6785 shr=16.4940931 lines=501 END +moesel call=309 dev=0 T=1 E=512 k=10 sel=0e96d181fbe0f662 minulps=0 mintok=0 x=1454.35589 logit=3214.42188 exp=819.497698 shr=9.12751558 lines=502 END +moesel call=310 dev=0 T=1 E=512 k=10 sel=20d48bff19c4622b minulps=1 mintok=0 x=1365.67586 logit=3303.09375 exp=706.827515 shr=9.78021464 lines=503 END +moesel call=311 dev=0 T=1 E=512 k=10 sel=41be0d7ea6bfae2a minulps=1 mintok=0 x=1705.86607 logit=3131.96875 exp=1070.42255 shr=17.436403 lines=504 END +moesel call=312 dev=0 T=1 E=512 k=10 sel=f07b08f0b824624f minulps=1 mintok=0 x=1175.55225 logit=3134.71875 exp=543.637867 shr=6.6405838 lines=505 END +moesel call=313 dev=0 T=1 E=512 k=10 sel=306067407f578c6c minulps=4 mintok=0 x=1151.1226 logit=3062.95312 exp=544.949956 shr=4.31586302 lines=506 END +moesel call=314 dev=0 T=1 E=512 k=10 sel=af889cc77de28889 minulps=3 mintok=0 x=1118.60958 logit=3112.25 exp=532.277276 shr=4.5413089 lines=507 END +moesel call=315 dev=0 T=1 E=512 k=10 sel=fbb4003d2cf903e5 minulps=1 mintok=0 x=1146.01284 logit=3064.59375 exp=510.087216 shr=4.25406353 lines=508 END +moesel call=316 dev=0 T=1 E=512 k=10 sel=7b5eb38026a67fa0 minulps=5 mintok=0 x=1202.11007 logit=3088.65625 exp=591.727532 shr=4.87267454 lines=509 END +moesel call=317 dev=0 T=1 E=512 k=10 sel=1438151a99fdf1fc minulps=7 mintok=0 x=1206.8433 logit=3162.17969 exp=591.162721 shr=5.94734176 lines=510 END +moesel call=318 dev=0 T=1 E=512 k=10 sel=afa719036db2c0f5 minulps=1 mintok=0 x=1372.0859 logit=3136.40625 exp=718.917334 shr=6.30778893 lines=511 END +moesel call=319 dev=0 T=1 E=512 k=10 sel=a981fd53e22fef1e minulps=5 mintok=0 x=1769.45648 logit=3175.03125 exp=1089.9384 shr=6.01519078 lines=512 END +moesel call=320 dev=0 T=1 E=512 k=10 sel=f512022075ab967a minulps=4 mintok=0 x=1677.24164 logit=3127.76562 exp=959.01701 shr=6.77459425 lines=513 END +moesel call=321 dev=0 T=1 E=512 k=10 sel=e3e83d079774bac0 minulps=1 mintok=0 x=1543.26454 logit=3190.46094 exp=942.540585 shr=5.17241367 lines=514 END +moesel call=322 dev=0 T=1 E=512 k=10 sel=d83b85ddd5d2e1f0 minulps=2 mintok=0 x=1387.29871 logit=3252.98438 exp=794.058034 shr=7.04065768 lines=515 END +moesel call=323 dev=0 T=1 E=512 k=10 sel=22861aec8ff4b678 minulps=3 mintok=0 x=1706.48158 logit=3243.46875 exp=1256.44632 shr=8.59600396 lines=516 END +moesel call=324 dev=0 T=1 E=512 k=10 sel=e9cdf6e7585ee84c minulps=2 mintok=0 x=2122.00943 logit=3170.73438 exp=2229.17721 shr=18.9982709 lines=517 END +moesel call=325 dev=0 T=1 E=512 k=10 sel=cb523256829ff045 minulps=0 mintok=0 x=1952.80993 logit=3240.79688 exp=1593.96826 shr=13.937657 lines=518 END +moesel call=326 dev=0 T=1 E=512 k=10 sel=d5906ad53d3a6484 minulps=1 mintok=0 x=1826.51982 logit=3330.98438 exp=1620.15082 shr=20.6728923 lines=519 END +moesel call=327 dev=0 T=1 E=512 k=10 sel=98f926914da9d6ae minulps=3 mintok=0 x=2110.17197 logit=3134.96875 exp=1890.36801 shr=15.6439174 lines=520 END +moesel call=328 dev=0 T=1 E=512 k=10 sel=7ae646d409f66388 minulps=1 mintok=0 x=2126.12093 logit=3055.54688 exp=1829.18977 shr=21.8441796 lines=521 END +moesel call=329 dev=0 T=1 E=512 k=10 sel=754474e0ad3b3d7a minulps=1 mintok=0 x=2351.24663 logit=3231.59375 exp=2071.29422 shr=14.9193765 lines=522 END +moesel call=330 dev=0 T=1 E=512 k=10 sel=24c522327b640a75 minulps=0 mintok=0 x=1999.79314 logit=3185.20312 exp=1894.82306 shr=24.8555946 lines=523 END +moesel call=331 dev=0 T=1 E=512 k=10 sel=8e5f507371f0d7b0 minulps=3 mintok=0 x=1933.59995 logit=3132.17188 exp=1420.90853 shr=16.1825735 lines=524 END +moesel call=332 dev=0 T=1 E=512 k=10 sel=270eebc98e4bead2 minulps=3 mintok=0 x=2184.76522 logit=3141.25 exp=1842.80225 shr=20.4444085 lines=525 END +moesel call=333 dev=0 T=1 E=512 k=10 sel=5354b8cea9dba18b minulps=3 mintok=0 x=2459.68673 logit=3237.875 exp=1626.42225 shr=27.8580039 lines=526 END +moesel call=334 dev=0 T=1 E=512 k=10 sel=1142e3534969a49e minulps=0 mintok=0 x=2406.31829 logit=3322.17188 exp=1799.39596 shr=40.4937158 lines=527 END +moesel call=335 dev=0 T=1 E=512 k=10 sel=c94a22fd88631213 minulps=3 mintok=0 x=2081.18664 logit=3374.34375 exp=1217.60359 shr=122.361991 lines=528 END +moesel call=336 dev=0 T=1 E=512 k=10 sel=b291daf010588d28 minulps=1 mintok=0 x=659.597758 logit=3447.39062 exp=469.159108 shr=45.7341356 lines=529 END +moesel call=337 dev=0 T=1 E=512 k=10 sel=ee32bc9736b6cf00 minulps=0 mintok=0 x=744.214747 logit=3119.85938 exp=405.268057 shr=6.88231257 lines=530 END +moesel call=338 dev=0 T=1 E=512 k=10 sel=50f50dedf5022755 minulps=6 mintok=0 x=820.894992 logit=3177.5 exp=397.777142 shr=5.8978748 lines=531 END +moesel call=339 dev=0 T=1 E=512 k=10 sel=c4132c5ffb9e3f9c minulps=1 mintok=0 x=976.810976 logit=3053.20312 exp=555.519444 shr=4.75735583 lines=532 END +moesel call=340 dev=0 T=1 E=512 k=10 sel=9c9cbc587c4e17e2 minulps=1 mintok=0 x=1021.63636 logit=3012.84375 exp=512.161756 shr=4.92166897 lines=533 END +moesel call=341 dev=0 T=1 E=512 k=10 sel=ecdbd26eba1207f3 minulps=1 mintok=0 x=895.423302 logit=3059.6875 exp=386.687547 shr=5.24265565 lines=534 END +moesel call=342 dev=0 T=1 E=512 k=10 sel=c2aec880ac22d520 minulps=6 mintok=0 x=1103.41817 logit=3091.20312 exp=562.54641 shr=6.19814041 lines=535 END +moesel call=343 dev=0 T=1 E=512 k=10 sel=2548dd9c77c6e7e4 minulps=0 mintok=0 x=1043.25394 logit=3116.15625 exp=487.197401 shr=6.63845846 lines=536 END +moesel call=344 dev=0 T=1 E=512 k=10 sel=4ad94ca019845820 minulps=7 mintok=0 x=1072.25548 logit=3064.90625 exp=503.086349 shr=6.67753621 lines=537 END +moesel call=345 dev=0 T=1 E=512 k=10 sel=8c3e238035b2e00c minulps=2 mintok=0 x=1084.56166 logit=3044.3125 exp=580.709931 shr=5.82407161 lines=538 END +moesel call=346 dev=0 T=1 E=512 k=10 sel=52893f4b249024aa minulps=1 mintok=0 x=1022.18437 logit=3088.23438 exp=455.926706 shr=5.36088835 lines=539 END +moesel call=347 dev=0 T=1 E=512 k=10 sel=d21e98016e134761 minulps=1 mintok=0 x=1029.04039 logit=3036.70312 exp=442.093493 shr=4.50964108 lines=540 END +moesel call=348 dev=0 T=1 E=512 k=10 sel=1b76e7d9385e7722 minulps=0 mintok=0 x=1087.00528 logit=3137.04688 exp=486.450697 shr=5.53218515 lines=541 END +moesel call=349 dev=0 T=1 E=512 k=10 sel=30013c162d01f2fe minulps=1 mintok=0 x=1003.66151 logit=3078.34375 exp=400.118767 shr=5.05321492 lines=542 END +moesel call=350 dev=0 T=1 E=512 k=10 sel=f11772ae6c3a563c minulps=2 mintok=0 x=1091.84171 logit=3083 exp=568.548157 shr=5.41433711 lines=543 END +moesel call=351 dev=0 T=1 E=512 k=10 sel=9f05bf74205c0721 minulps=7 mintok=0 x=1519.62847 logit=3129.46875 exp=906.670658 shr=6.11613493 lines=544 END +moesel call=352 dev=0 T=1 E=512 k=10 sel=eb54668ec73370b0 minulps=2 mintok=0 x=1376.96477 logit=3132.51562 exp=693.985085 shr=5.12042207 lines=545 END +moesel call=353 dev=0 T=1 E=512 k=10 sel=b6f816b2af49cdba minulps=3 mintok=0 x=1178.19841 logit=3098.24219 exp=564.683437 shr=3.89768215 lines=546 END +moesel call=354 dev=0 T=1 E=512 k=10 sel=26b79864e98625c0 minulps=0 mintok=0 x=1191.77339 logit=3240.42188 exp=572.89218 shr=5.64262325 lines=547 END +moesel call=355 dev=0 T=1 E=512 k=10 sel=f1d72110a36eaffc minulps=1 mintok=0 x=1327.32575 logit=3190.39844 exp=676.942388 shr=5.84557788 lines=548 END +moesel call=356 dev=0 T=1 E=512 k=10 sel=c2c0fc9410fbbe29 minulps=2 mintok=0 x=1675.41935 logit=3261.26562 exp=1132.40968 shr=12.6039939 lines=549 END +moesel call=357 dev=0 T=1 E=512 k=10 sel=1ce725bd05554cae minulps=5 mintok=0 x=1560.32608 logit=3228.60938 exp=818.527346 shr=8.10074924 lines=550 END +moesel call=358 dev=0 T=1 E=512 k=10 sel=63488d7e9d2c4504 minulps=4 mintok=0 x=1479.46151 logit=3321.84375 exp=878.031076 shr=9.61285484 lines=551 END +moesel call=359 dev=0 T=1 E=512 k=10 sel=0eee9341edc422b7 minulps=1 mintok=0 x=1755.06528 logit=3166.26562 exp=1101.81624 shr=17.2246516 lines=552 END +moesel call=360 dev=0 T=1 E=512 k=10 sel=a6b071237faf4939 minulps=0 mintok=0 x=1208.2373 logit=3162.96875 exp=533.579158 shr=7.38802949 lines=553 END +moesel call=361 dev=0 T=1 E=512 k=10 sel=1ca8d72d91047dfa minulps=1 mintok=0 x=1230.13184 logit=3076.29688 exp=558.367758 shr=5.81657105 lines=554 END +moesel call=362 dev=0 T=1 E=512 k=10 sel=933cd14c3bd4b52e minulps=2 mintok=0 x=1140.70204 logit=3155.53125 exp=533.99655 shr=4.07003582 lines=555 END +moesel call=363 dev=0 T=1 E=512 k=10 sel=2dbdc55966c7dd8b minulps=7 mintok=0 x=1153.27789 logit=3073.625 exp=557.406727 shr=5.22911939 lines=556 END +moesel call=364 dev=0 T=1 E=512 k=10 sel=8b7f2d2d2c099b60 minulps=17 mintok=0 x=1207.63631 logit=3124.90625 exp=582.194724 shr=5.69001283 lines=557 END +moesel call=365 dev=0 T=1 E=512 k=10 sel=6643974331867f0f minulps=1 mintok=0 x=1233.70158 logit=3159.85156 exp=598.597766 shr=6.52653512 lines=558 END +moesel call=366 dev=0 T=1 E=512 k=10 sel=05fd9dc99c8030b0 minulps=0 mintok=0 x=1427.28672 logit=3160.21875 exp=878.17876 shr=9.61677593 lines=559 END +moesel call=367 dev=0 T=1 E=512 k=10 sel=4dd5c3066fd85d47 minulps=2 mintok=0 x=1735.77043 logit=3152.92188 exp=1124.61654 shr=8.02624789 lines=560 END +moesel call=368 dev=0 T=1 E=512 k=10 sel=0f3004f32e150574 minulps=2 mintok=0 x=1686.73968 logit=3083.09375 exp=1044.54516 shr=8.6367342 lines=561 END +moesel call=369 dev=0 T=1 E=512 k=10 sel=7f9ba315ca0d0e17 minulps=2 mintok=0 x=1526.28354 logit=3173.75 exp=962.364124 shr=5.67905901 lines=562 END +moesel call=370 dev=0 T=1 E=512 k=10 sel=a06dce1696f255a7 minulps=2 mintok=0 x=1456.22031 logit=3270.875 exp=1028.26673 shr=9.75792856 lines=563 END +moesel call=371 dev=0 T=1 E=512 k=10 sel=d8acbe0688ac969c minulps=0 mintok=0 x=1684.10358 logit=3224.45312 exp=1357.37747 shr=11.3340499 lines=564 END +moesel call=372 dev=0 T=1 E=512 k=10 sel=0bc097545ee84525 minulps=1 mintok=0 x=2024.9783 logit=3170.73438 exp=2188.19417 shr=24.8378046 lines=565 END +moesel call=373 dev=0 T=1 E=512 k=10 sel=d2176357f953daec minulps=1 mintok=0 x=1936.66635 logit=3245.34375 exp=1617.78563 shr=16.9567539 lines=566 END +moesel call=374 dev=0 T=1 E=512 k=10 sel=bafbe7a4b3084f1c minulps=1 mintok=0 x=2046.25077 logit=3364.85938 exp=2342.15202 shr=24.7374004 lines=567 END +moesel call=375 dev=0 T=1 E=512 k=10 sel=e683a36f83bc94f1 minulps=0 mintok=0 x=2122.94685 logit=3053.25 exp=2084.32494 shr=24.3336982 lines=568 END +moesel call=376 dev=0 T=1 E=512 k=10 sel=b157d56d3aafcca5 minulps=1 mintok=0 x=2061.46975 logit=3035.375 exp=1779.68694 shr=27.7318195 lines=569 END +moesel call=377 dev=0 T=1 E=512 k=10 sel=1e1dbb003a35f100 minulps=1 mintok=0 x=2239.2497 logit=3123.10938 exp=2039.06469 shr=19.410901 lines=570 END +moesel call=378 dev=0 T=1 E=512 k=10 sel=93a4e9383ea41440 minulps=1 mintok=0 x=2189.8285 logit=3234.20312 exp=2259.36728 shr=26.0850141 lines=571 END +moesel call=379 dev=0 T=1 E=512 k=10 sel=bb5aa0e989464ade minulps=4 mintok=0 x=1933.07185 logit=3144.82812 exp=1486.27811 shr=23.1175638 lines=572 END +moesel call=380 dev=0 T=1 E=512 k=10 sel=45b34bd15ceb4c30 minulps=0 mintok=0 x=2333.60364 logit=3283.95312 exp=2022.50347 shr=22.2763712 lines=573 END +moesel call=381 dev=0 T=1 E=512 k=10 sel=f4e46c128544fade minulps=1 mintok=0 x=2434.59585 logit=3230.5 exp=1760.3718 shr=28.0803697 lines=574 END +moesel call=382 dev=0 T=1 E=512 k=10 sel=8fbba3271d996794 minulps=1 mintok=0 x=2435.83194 logit=3288.67188 exp=1834.84836 shr=33.4673995 lines=575 END +moesel call=383 dev=0 T=1 E=512 k=10 sel=f50c7fe117d083f5 minulps=2 mintok=0 x=2213.45872 logit=3272.98438 exp=1350.7858 shr=128.38923 lines=576 END diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt new file mode 100644 index 000000000..3d1214d1c --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt @@ -0,0 +1,384 @@ +moesel call=0 dev=1 T=5 E=512 k=10 sel=f06a56dd5ffbf75d minulps=0 mintok=2 x=3613.74301 logit=17079.1641 exp=2540.45471 shr=250.794444 lines=5 END +moesel call=1 dev=1 T=5 E=512 k=10 sel=fb2a5272f1e4f8e0 minulps=0 mintok=2 x=3777.83058 logit=15567.3438 exp=2017.76373 shr=34.0138945 lines=10 END +moesel call=2 dev=1 T=5 E=512 k=10 sel=6ea2aef387496645 minulps=0 mintok=3 x=3842.47361 logit=15793.9375 exp=1800.37012 shr=51.3618547 lines=15 END +moesel call=3 dev=1 T=5 E=512 k=10 sel=eed17fad6513ba9d minulps=0 mintok=1 x=4873.4291 logit=15074.5156 exp=2582.47021 shr=37.7886879 lines=20 END +moesel call=4 dev=1 T=5 E=512 k=10 sel=4537bad749058f33 minulps=1 mintok=0 x=5355.60817 logit=15440.0156 exp=3064.05299 shr=47.2066198 lines=25 END +moesel call=5 dev=1 T=5 E=512 k=10 sel=ae1f95fee2b91838 minulps=0 mintok=4 x=4433.97312 logit=15627.6094 exp=2007.19558 shr=33.6438845 lines=30 END +moesel call=6 dev=1 T=5 E=512 k=10 sel=2ba8099d8f428dab minulps=0 mintok=0 x=5469.48399 logit=15911.25 exp=3096.6044 shr=46.8479666 lines=35 END +moesel call=7 dev=1 T=5 E=512 k=10 sel=4d910d271b635a3c minulps=0 mintok=4 x=5332.68826 logit=15908.0312 exp=2688.59794 shr=43.7803475 lines=40 END +moesel call=8 dev=1 T=5 E=512 k=10 sel=c0b7afc86cb49d16 minulps=0 mintok=2 x=5381.94492 logit=15746.7188 exp=3038.7866 shr=28.1943145 lines=45 END +moesel call=9 dev=1 T=5 E=512 k=10 sel=d27ac843303df465 minulps=0 mintok=0 x=5525.55351 logit=15657.6875 exp=2980.15014 shr=34.7109443 lines=50 END +moesel call=10 dev=1 T=5 E=512 k=10 sel=b14a7db9158b4e90 minulps=0 mintok=0 x=4963.28279 logit=15763.2812 exp=2419.79498 shr=26.9856972 lines=55 END +moesel call=11 dev=1 T=5 E=512 k=10 sel=a9f7fa4736c5c260 minulps=0 mintok=0 x=4866.5477 logit=15586.4688 exp=2152.86156 shr=18.6761832 lines=60 END +moesel call=12 dev=1 T=5 E=512 k=10 sel=98f4025615644ee6 minulps=0 mintok=0 x=4969.61794 logit=15637.7969 exp=2263.46059 shr=26.5210111 lines=65 END +moesel call=13 dev=1 T=5 E=512 k=10 sel=5ed47518079967d8 minulps=0 mintok=0 x=4648.40087 logit=15946.6172 exp=2030.92948 shr=27.4775673 lines=70 END +moesel call=14 dev=1 T=5 E=512 k=10 sel=031ae0561aec26c7 minulps=1 mintok=2 x=5256.95827 logit=15837.1875 exp=2909.15434 shr=22.7684577 lines=75 END +moesel call=15 dev=1 T=5 E=512 k=10 sel=88cb94c69381a618 minulps=0 mintok=0 x=6739.13267 logit=15811.4531 exp=3943.28047 shr=29.9124121 lines=80 END +moesel call=16 dev=1 T=5 E=512 k=10 sel=a8002b1c0688c414 minulps=0 mintok=1 x=6158.84255 logit=15758.3984 exp=3197.36537 shr=28.148731 lines=85 END +moesel call=17 dev=1 T=5 E=512 k=10 sel=cb0c0f7107d1b9f1 minulps=0 mintok=0 x=5371.43593 logit=15997.0938 exp=2598.86246 shr=23.7154928 lines=90 END +moesel call=18 dev=1 T=5 E=512 k=10 sel=f105feb5386cb177 minulps=1 mintok=2 x=5329.31178 logit=16324.9844 exp=2863.26153 shr=25.0691803 lines=95 END +moesel call=19 dev=1 T=5 E=512 k=10 sel=f099e75c3c364d86 minulps=0 mintok=0 x=5867.62627 logit=16197.375 exp=2835.76687 shr=28.2019645 lines=100 END +moesel call=20 dev=1 T=5 E=512 k=10 sel=7e2589d8f592e931 minulps=0 mintok=1 x=7148.21312 logit=16143 exp=4349.18548 shr=42.8663893 lines=105 END +moesel call=21 dev=1 T=5 E=512 k=10 sel=7155e9a4f0d1e731 minulps=0 mintok=0 x=6732.45336 logit=16193.1875 exp=3776.13145 shr=28.3413144 lines=110 END +moesel call=22 dev=1 T=5 E=512 k=10 sel=b3c62ea616442afa minulps=1 mintok=0 x=6492.17328 logit=16639.5312 exp=3611.11839 shr=34.8373965 lines=115 END +moesel call=23 dev=1 T=5 E=512 k=10 sel=a6e5fe051244ddf9 minulps=0 mintok=0 x=7579.43175 logit=16095.375 exp=4441.04099 shr=64.5485376 lines=120 END +moesel call=24 dev=1 T=5 E=512 k=10 sel=9540c28f286b90c6 minulps=0 mintok=1 x=5623.64631 logit=16329.8281 exp=2826.56102 shr=27.9391904 lines=125 END +moesel call=25 dev=1 T=5 E=512 k=10 sel=5817b743f19b664b minulps=0 mintok=0 x=5644.2011 logit=16026.5625 exp=2680.53693 shr=28.6913512 lines=130 END +moesel call=26 dev=1 T=5 E=512 k=10 sel=42c8ba089092b1b8 minulps=0 mintok=3 x=5242.63172 logit=16462.6016 exp=2432.41667 shr=22.653267 lines=135 END +moesel call=27 dev=1 T=5 E=512 k=10 sel=cb459862b970b77b minulps=0 mintok=1 x=5250.96584 logit=15928.7188 exp=2364.57499 shr=18.957759 lines=140 END +moesel call=28 dev=1 T=5 E=512 k=10 sel=3a31d0795bc6f47c minulps=1 mintok=3 x=5288.7323 logit=16112.25 exp=2343.11091 shr=23.6415624 lines=145 END +moesel call=29 dev=1 T=5 E=512 k=10 sel=6efd42d528ac6d8a minulps=0 mintok=2 x=5506.69599 logit=16535.8672 exp=2795.36289 shr=25.0596944 lines=150 END +moesel call=30 dev=1 T=5 E=512 k=10 sel=fb598e1355134959 minulps=0 mintok=4 x=6410.06056 logit=16661.2188 exp=3735.56547 shr=32.8470495 lines=155 END +moesel call=31 dev=1 T=5 E=512 k=10 sel=0050705d4ac202f3 minulps=0 mintok=1 x=8011.73438 logit=16345.3906 exp=4824.42603 shr=31.2061786 lines=160 END +moesel call=32 dev=1 T=5 E=512 k=10 sel=b7dded05065392b3 minulps=0 mintok=0 x=7319.41444 logit=16006.7109 exp=4399.70613 shr=32.7077647 lines=165 END +moesel call=33 dev=1 T=5 E=512 k=10 sel=4724818dacbff9f5 minulps=0 mintok=0 x=6685.99892 logit=16719.4062 exp=3823.74303 shr=25.313163 lines=170 END +moesel call=34 dev=1 T=5 E=512 k=10 sel=3188ef46a7207960 minulps=0 mintok=0 x=6111.76998 logit=16670.3125 exp=3806.79113 shr=34.2649005 lines=175 END +moesel call=35 dev=1 T=5 E=512 k=10 sel=a2b69c167a483a9f minulps=1 mintok=1 x=7041.42279 logit=16437.7656 exp=4584.31584 shr=34.4142891 lines=180 END +moesel call=36 dev=1 T=5 E=512 k=10 sel=f103fcb3508aa75b minulps=0 mintok=2 x=8744.37358 logit=16389.3984 exp=8149.37753 shr=88.5638159 lines=185 END +moesel call=37 dev=1 T=5 E=512 k=10 sel=8cffc9396f331d74 minulps=0 mintok=3 x=8264.8447 logit=16347.4062 exp=7002.85247 shr=65.2569962 lines=190 END +moesel call=38 dev=1 T=5 E=512 k=10 sel=bb350da9dcf52f46 minulps=0 mintok=3 x=8415.83688 logit=16815.75 exp=9246.79079 shr=109.602601 lines=195 END +moesel call=39 dev=1 T=5 E=512 k=10 sel=24d44eb6c44320f8 minulps=0 mintok=0 x=9002.68804 logit=15687.1641 exp=8439.7024 shr=96.6948645 lines=200 END +moesel call=40 dev=1 T=5 E=512 k=10 sel=ec50e4833d15dd6a minulps=0 mintok=0 x=9469.13997 logit=15723.0781 exp=7270.17214 shr=95.6658297 lines=205 END +moesel call=41 dev=1 T=5 E=512 k=10 sel=ee151309f1ded60c minulps=1 mintok=1 x=9563.14343 logit=16093.5781 exp=6882.67483 shr=100.706101 lines=210 END +moesel call=42 dev=1 T=5 E=512 k=10 sel=1a57c6c6a3bb19fb minulps=1 mintok=0 x=9784.81565 logit=16399.5625 exp=9223.56793 shr=131.682382 lines=215 END +moesel call=43 dev=1 T=5 E=512 k=10 sel=1a5af66d4fce33e6 minulps=0 mintok=1 x=8164.36896 logit=15766.6875 exp=5799.10439 shr=91.4694543 lines=220 END +moesel call=44 dev=1 T=5 E=512 k=10 sel=fbc17851e34f09a8 minulps=0 mintok=2 x=9133.89888 logit=15832.5312 exp=7374.70481 shr=105.2929 lines=225 END +moesel call=45 dev=1 T=5 E=512 k=10 sel=9fa59df33f617085 minulps=0 mintok=0 x=10331.2517 logit=16207.375 exp=7222.62607 shr=148.028945 lines=230 END +moesel call=46 dev=1 T=5 E=512 k=10 sel=1b55df0881bfc3af minulps=0 mintok=3 x=10065.2716 logit=16250.6719 exp=8490.58183 shr=158.074511 lines=235 END +moesel call=47 dev=1 T=5 E=512 k=10 sel=73ac3458f62c111f minulps=1 mintok=1 x=9562.57522 logit=16366.3281 exp=9053.50797 shr=825.618974 lines=240 END +moesel call=48 dev=1 T=1 E=512 k=10 sel=8fce961a87c895de minulps=7 mintok=0 x=723.470177 logit=3429.89062 exp=563.539422 shr=31.5357613 lines=241 END +moesel call=49 dev=1 T=1 E=512 k=10 sel=da67409db4906d8e minulps=2 mintok=0 x=816.576027 logit=3055.9375 exp=412.922141 shr=6.05394083 lines=242 END +moesel call=50 dev=1 T=1 E=512 k=10 sel=dc6a93971be9cf6b minulps=0 mintok=0 x=851.727616 logit=3039.20312 exp=411.391483 shr=9.24520291 lines=243 END +moesel call=51 dev=1 T=1 E=512 k=10 sel=da92804a7ed93391 minulps=2 mintok=0 x=1061.18465 logit=2983.54688 exp=515.079629 shr=5.82929541 lines=244 END +moesel call=52 dev=1 T=1 E=512 k=10 sel=79e503d24cec8ea1 minulps=10 mintok=0 x=1090.49174 logit=2982.1875 exp=544.015517 shr=5.63867059 lines=245 END +moesel call=53 dev=1 T=1 E=512 k=10 sel=8423f417266f13a9 minulps=1 mintok=0 x=941.708185 logit=3028.28125 exp=427.827993 shr=4.90885232 lines=246 END +moesel call=54 dev=1 T=1 E=512 k=10 sel=38d2a77ae773af17 minulps=2 mintok=0 x=1090.95048 logit=2902.09375 exp=691.67611 shr=6.25859495 lines=247 END +moesel call=55 dev=1 T=1 E=512 k=10 sel=e2b2fcd5dc379493 minulps=1 mintok=0 x=1110.22383 logit=2969.21875 exp=571.045208 shr=5.27463305 lines=248 END +moesel call=56 dev=1 T=1 E=512 k=10 sel=23e3af4450f40f94 minulps=5 mintok=0 x=1131.85893 logit=2973.25 exp=630.538729 shr=5.053668 lines=249 END +moesel call=57 dev=1 T=1 E=512 k=10 sel=c02b53ae5ee547bd minulps=2 mintok=0 x=1167.77365 logit=3020.34375 exp=599.142368 shr=7.05602749 lines=250 END +moesel call=58 dev=1 T=1 E=512 k=10 sel=b80e559c042768ff minulps=0 mintok=0 x=1063.17118 logit=2990.57812 exp=517.027451 shr=4.58549351 lines=251 END +moesel call=59 dev=1 T=1 E=512 k=10 sel=b978d5771b953316 minulps=2 mintok=0 x=1070.02329 logit=3021.53125 exp=477.88352 shr=3.21880716 lines=252 END +moesel call=60 dev=1 T=1 E=512 k=10 sel=5f147bce39d3d018 minulps=1 mintok=0 x=1108.20363 logit=3036.79688 exp=500.637898 shr=5.64360538 lines=253 END +moesel call=61 dev=1 T=1 E=512 k=10 sel=84b8a4ffe8160de6 minulps=0 mintok=0 x=998.216994 logit=3043.34375 exp=421.396003 shr=4.870466 lines=254 END +moesel call=62 dev=1 T=1 E=512 k=10 sel=9f55aa23ddc601de minulps=2 mintok=0 x=1127.07912 logit=3031.84375 exp=513.611865 shr=5.490476 lines=255 END +moesel call=63 dev=1 T=1 E=512 k=10 sel=e6abc54b43ce81a5 minulps=1 mintok=0 x=1455.34824 logit=3050.59375 exp=791.663157 shr=6.71446584 lines=256 END +moesel call=64 dev=1 T=1 E=512 k=10 sel=1255bc20aafb1161 minulps=1 mintok=0 x=1290.26098 logit=3052.90625 exp=622.36894 shr=5.60229019 lines=257 END +moesel call=65 dev=1 T=1 E=512 k=10 sel=c5ba50675c9f9d6d minulps=2 mintok=0 x=1140.36913 logit=3127.625 exp=593.306181 shr=4.68912232 lines=258 END +moesel call=66 dev=1 T=1 E=512 k=10 sel=cf9b2d4e92ee8295 minulps=2 mintok=0 x=1114.25081 logit=3188.0625 exp=577.227118 shr=4.53733855 lines=259 END +moesel call=67 dev=1 T=1 E=512 k=10 sel=7b0562becd0aba31 minulps=2 mintok=0 x=1281.4322 logit=3164.85938 exp=667.557706 shr=5.21218349 lines=260 END +moesel call=68 dev=1 T=1 E=512 k=10 sel=c8c8bfef3ac9cf11 minulps=1 mintok=0 x=1534.10805 logit=3156.46875 exp=945.605924 shr=9.54545289 lines=261 END +moesel call=69 dev=1 T=1 E=512 k=10 sel=fcd94a844c502abc minulps=0 mintok=0 x=1496.12201 logit=3201.54688 exp=848.053703 shr=7.07275957 lines=262 END +moesel call=70 dev=1 T=1 E=512 k=10 sel=ba1241b9927c42f3 minulps=5 mintok=0 x=1391.05154 logit=3276.57812 exp=828.893357 shr=4.11207919 lines=263 END +moesel call=71 dev=1 T=1 E=512 k=10 sel=5d8589e5e9b25070 minulps=5 mintok=0 x=1710.48188 logit=3154.39062 exp=1068.14446 shr=15.594517 lines=264 END +moesel call=72 dev=1 T=1 E=512 k=10 sel=91612b3fa6473fb0 minulps=1 mintok=0 x=1175.36515 logit=3137.07812 exp=584.083677 shr=5.80112762 lines=265 END +moesel call=73 dev=1 T=1 E=512 k=10 sel=b2ea85fa9a669cf8 minulps=11 mintok=0 x=1248.51615 logit=3123.1875 exp=596.386972 shr=5.07536156 lines=266 END +moesel call=74 dev=1 T=1 E=512 k=10 sel=9d9437086dced8c5 minulps=7 mintok=0 x=1149.56437 logit=3154.14062 exp=568.216715 shr=4.88735269 lines=267 END +moesel call=75 dev=1 T=1 E=512 k=10 sel=b943fedc5d203b58 minulps=10 mintok=0 x=1202.45795 logit=3088.84375 exp=639.258215 shr=5.79201026 lines=268 END +moesel call=76 dev=1 T=1 E=512 k=10 sel=000c6654b4417e5a minulps=4 mintok=0 x=1238.31239 logit=3142.3125 exp=650.707318 shr=5.53921388 lines=269 END +moesel call=77 dev=1 T=1 E=512 k=10 sel=ee80fbd9c62d5569 minulps=4 mintok=0 x=1271.90447 logit=3151.98438 exp=707.469367 shr=7.42523699 lines=270 END +moesel call=78 dev=1 T=1 E=512 k=10 sel=bd58136d2912631a minulps=1 mintok=0 x=1461.16225 logit=3129.1875 exp=884.908316 shr=7.92177817 lines=271 END +moesel call=79 dev=1 T=1 E=512 k=10 sel=eb29872faed407da minulps=4 mintok=0 x=1834.92 logit=3084.75 exp=1238.039 shr=9.50212419 lines=272 END +moesel call=80 dev=1 T=1 E=512 k=10 sel=292074bcffc58544 minulps=2 mintok=0 x=1759.32481 logit=3040.1875 exp=1064.99146 shr=9.66410056 lines=273 END +moesel call=81 dev=1 T=1 E=512 k=10 sel=4a574cc2fc7fb9fb minulps=4 mintok=0 x=1596.62545 logit=3309.0625 exp=1104.52725 shr=6.23227578 lines=274 END +moesel call=82 dev=1 T=1 E=512 k=10 sel=b3eaa9c6c6eb3130 minulps=5 mintok=0 x=1430.16452 logit=3348 exp=993.164888 shr=5.87929016 lines=275 END +moesel call=83 dev=1 T=1 E=512 k=10 sel=2bb0197f9528dae3 minulps=1 mintok=0 x=1680.11528 logit=3312.57812 exp=1237.64796 shr=6.98264772 lines=276 END +moesel call=84 dev=1 T=1 E=512 k=10 sel=cd4c4051f92b4d23 minulps=2 mintok=0 x=2020.4353 logit=3222.48438 exp=1912.06777 shr=17.9209241 lines=277 END +moesel call=85 dev=1 T=1 E=512 k=10 sel=76936aeb5eaaab51 minulps=6 mintok=0 x=1931.11175 logit=3290.15625 exp=1570.44779 shr=12.4351218 lines=278 END +moesel call=86 dev=1 T=1 E=512 k=10 sel=efc0b3e6db95dc18 minulps=0 mintok=0 x=1897.23602 logit=3249.34375 exp=2210.29989 shr=18.043677 lines=279 END +moesel call=87 dev=1 T=1 E=512 k=10 sel=2b804122fbd84b29 minulps=0 mintok=0 x=2039.54198 logit=3036.3125 exp=1692.76127 shr=26.3856548 lines=280 END +moesel call=88 dev=1 T=1 E=512 k=10 sel=c91c0963d866cf8c minulps=1 mintok=0 x=2063.36669 logit=3078.40625 exp=1645.13399 shr=23.3459053 lines=281 END +moesel call=89 dev=1 T=1 E=512 k=10 sel=6c1d2b803ded5a66 minulps=1 mintok=0 x=2106.95315 logit=3222.42188 exp=1856.79287 shr=14.4382533 lines=282 END +moesel call=90 dev=1 T=1 E=512 k=10 sel=130cc2b2b901b5c5 minulps=1 mintok=0 x=2103.03736 logit=3184 exp=2505.59213 shr=23.2879008 lines=283 END +moesel call=91 dev=1 T=1 E=512 k=10 sel=d72d542d6b6ef0ec minulps=0 mintok=0 x=1824.80401 logit=3087.8125 exp=1415.92651 shr=21.9134084 lines=284 END +moesel call=92 dev=1 T=1 E=512 k=10 sel=dc0e191167472a9a minulps=0 mintok=0 x=2065.04691 logit=3159.75 exp=1687.67223 shr=18.6924855 lines=285 END +moesel call=93 dev=1 T=1 E=512 k=10 sel=0770365869cd05cc minulps=7 mintok=0 x=2202.8219 logit=3269.01562 exp=1561.783 shr=23.8309534 lines=286 END +moesel call=94 dev=1 T=1 E=512 k=10 sel=3cac9ccd67a99bfc minulps=1 mintok=0 x=2221.9064 logit=3263.76562 exp=1527.75915 shr=32.8177135 lines=287 END +moesel call=95 dev=1 T=1 E=512 k=10 sel=2873dc2ce72e08bb minulps=2 mintok=0 x=2000.9953 logit=3182.67188 exp=1438.48454 shr=137.679196 lines=288 END +moesel call=96 dev=1 T=1 E=512 k=10 sel=0148dfd4911f327c minulps=0 mintok=0 x=711.156308 logit=3388.53125 exp=609.459642 shr=38.3518281 lines=289 END +moesel call=97 dev=1 T=1 E=512 k=10 sel=2a8a39b413aa2418 minulps=2 mintok=0 x=812.718665 logit=3055.0625 exp=457.39149 shr=5.11571234 lines=290 END +moesel call=98 dev=1 T=1 E=512 k=10 sel=7091a270949c8cf6 minulps=0 mintok=0 x=793.294253 logit=3057.59375 exp=304.46167 shr=7.27015826 lines=291 END +moesel call=99 dev=1 T=1 E=512 k=10 sel=835f7d56dda2a4d0 minulps=2 mintok=0 x=1148.8844 logit=3032.5625 exp=562.579764 shr=3.96235645 lines=292 END +moesel call=100 dev=1 T=1 E=512 k=10 sel=ca64ce98d54c003b minulps=2 mintok=0 x=1151.14993 logit=2937.5 exp=528.301102 shr=5.20830437 lines=293 END +moesel call=101 dev=1 T=1 E=512 k=10 sel=82c9513497c8dab5 minulps=2 mintok=0 x=1052.03611 logit=3041.64062 exp=469.374281 shr=8.19047566 lines=294 END +moesel call=102 dev=1 T=1 E=512 k=10 sel=fd2e2488b48a2177 minulps=2 mintok=0 x=1202.21154 logit=3076.23438 exp=646.929276 shr=6.38832703 lines=295 END +moesel call=103 dev=1 T=1 E=512 k=10 sel=b40e07a81fdf978f minulps=0 mintok=0 x=1098.26801 logit=3023.5625 exp=524.833726 shr=5.2388922 lines=296 END +moesel call=104 dev=1 T=1 E=512 k=10 sel=24e17d809db8c11a minulps=2 mintok=0 x=1118.6887 logit=3037.42188 exp=601.489528 shr=6.11610687 lines=297 END +moesel call=105 dev=1 T=1 E=512 k=10 sel=67c8568c46a49a4a minulps=0 mintok=0 x=1157.52346 logit=3072.96875 exp=540.229177 shr=5.98809193 lines=298 END +moesel call=106 dev=1 T=1 E=512 k=10 sel=a815754084dc4839 minulps=1 mintok=0 x=1068.68475 logit=3096.5625 exp=490.029281 shr=3.74977484 lines=299 END +moesel call=107 dev=1 T=1 E=512 k=10 sel=e31b42713f519e1a minulps=3 mintok=0 x=1055.69063 logit=3066.85938 exp=461.200665 shr=3.00659831 lines=300 END +moesel call=108 dev=1 T=1 E=512 k=10 sel=83b2d2fe7b7d991c minulps=2 mintok=0 x=1068.40184 logit=3072.42188 exp=498.168385 shr=6.41813868 lines=301 END +moesel call=109 dev=1 T=1 E=512 k=10 sel=7f5cc3e8d5161609 minulps=2 mintok=0 x=959.089637 logit=3026.51562 exp=419.919528 shr=6.11775106 lines=302 END +moesel call=110 dev=1 T=1 E=512 k=10 sel=98305dfd7917367d minulps=2 mintok=0 x=1192.49512 logit=3149.20312 exp=638.70187 shr=5.31779092 lines=303 END +moesel call=111 dev=1 T=1 E=512 k=10 sel=38ddb3334bf52011 minulps=12 mintok=0 x=1578.1546 logit=3117.59375 exp=927.968217 shr=7.89197639 lines=304 END +moesel call=112 dev=1 T=1 E=512 k=10 sel=d14c90fc283815e4 minulps=1 mintok=0 x=1321.545 logit=3179.85938 exp=670.672803 shr=4.59513244 lines=305 END +moesel call=113 dev=1 T=1 E=512 k=10 sel=5bf394db409c91de minulps=8 mintok=0 x=1084.32861 logit=3069.04688 exp=470.568587 shr=4.149268 lines=306 END +moesel call=114 dev=1 T=1 E=512 k=10 sel=db2f2b90ff1ca8c4 minulps=4 mintok=0 x=1062.79936 logit=3204.85938 exp=503.707939 shr=3.60143873 lines=307 END +moesel call=115 dev=1 T=1 E=512 k=10 sel=bf85a0dd154e12b1 minulps=1 mintok=0 x=1186.4054 logit=3132.625 exp=571.425364 shr=5.03959958 lines=308 END +moesel call=116 dev=1 T=1 E=512 k=10 sel=bd629352fffafcc6 minulps=0 mintok=0 x=1474.49221 logit=3201.65625 exp=814.89747 shr=7.91446456 lines=309 END +moesel call=117 dev=1 T=1 E=512 k=10 sel=4625653a1b19e96c minulps=2 mintok=0 x=1436.522 logit=3280.26562 exp=788.017591 shr=4.93058806 lines=310 END +moesel call=118 dev=1 T=1 E=512 k=10 sel=38c8a9ac128de4be minulps=2 mintok=0 x=1387.48718 logit=3292.9375 exp=765.92464 shr=3.74038844 lines=311 END +moesel call=119 dev=1 T=1 E=512 k=10 sel=9e60c6524f345583 minulps=0 mintok=0 x=1690.051 logit=3135.51562 exp=939.131987 shr=10.7073113 lines=312 END +moesel call=120 dev=1 T=1 E=512 k=10 sel=5fad91ec51de038f minulps=2 mintok=0 x=1188.62005 logit=3185.76562 exp=560.85911 shr=4.86120316 lines=313 END +moesel call=121 dev=1 T=1 E=512 k=10 sel=a033bbe69a732965 minulps=3 mintok=0 x=1223.35445 logit=3199.75 exp=606.992688 shr=4.56049468 lines=314 END +moesel call=122 dev=1 T=1 E=512 k=10 sel=0c39cc88655373dd minulps=2 mintok=0 x=1107.90326 logit=3252.17188 exp=558.658191 shr=4.66765702 lines=315 END +moesel call=123 dev=1 T=1 E=512 k=10 sel=1b10475baf5406c8 minulps=7 mintok=0 x=1193.38707 logit=3228.53125 exp=610.211403 shr=5.04859781 lines=316 END +moesel call=124 dev=1 T=1 E=512 k=10 sel=6bcf0c3c28679919 minulps=1 mintok=0 x=1201.07731 logit=3259.875 exp=585.112188 shr=5.20723253 lines=317 END +moesel call=125 dev=1 T=1 E=512 k=10 sel=1be218557f73f10e minulps=0 mintok=0 x=1197.28618 logit=3180.78125 exp=717.723588 shr=6.28841968 lines=318 END +moesel call=126 dev=1 T=1 E=512 k=10 sel=ab608f161fba4564 minulps=2 mintok=0 x=1454.99889 logit=3286.875 exp=989.59422 shr=9.29866271 lines=319 END +moesel call=127 dev=1 T=1 E=512 k=10 sel=2b9e8eeb37f55c79 minulps=1 mintok=0 x=1942.50397 logit=3202.98438 exp=1258.63799 shr=11.0705444 lines=320 END +moesel call=128 dev=1 T=1 E=512 k=10 sel=749568181cf50c4f minulps=6 mintok=0 x=1783.84903 logit=3163.57812 exp=1107.94875 shr=9.83864037 lines=321 END +moesel call=129 dev=1 T=1 E=512 k=10 sel=45d1e46900535c92 minulps=13 mintok=0 x=1534.101 logit=3155.03125 exp=1024.5957 shr=8.38383763 lines=322 END +moesel call=130 dev=1 T=1 E=512 k=10 sel=7ecefc81d491db2f minulps=1 mintok=0 x=1341.92368 logit=3231.20312 exp=791.356913 shr=5.80059241 lines=323 END +moesel call=131 dev=1 T=1 E=512 k=10 sel=5deb7233735a17d6 minulps=2 mintok=0 x=1643.58147 logit=3183.79688 exp=1020.21192 shr=8.19191471 lines=324 END +moesel call=132 dev=1 T=1 E=512 k=10 sel=bccd379b9a29dc4b minulps=2 mintok=0 x=1930.93375 logit=3198.125 exp=1744.26319 shr=22.8635413 lines=325 END +moesel call=133 dev=1 T=1 E=512 k=10 sel=0307bb7dc110b2b1 minulps=0 mintok=0 x=1780.71473 logit=3185.17188 exp=1323.80137 shr=11.6337567 lines=326 END +moesel call=134 dev=1 T=1 E=512 k=10 sel=823351c2efeb4459 minulps=1 mintok=0 x=1841.61302 logit=3173.40625 exp=1866.47121 shr=15.1784789 lines=327 END +moesel call=135 dev=1 T=1 E=512 k=10 sel=fe47f81894317833 minulps=2 mintok=0 x=2051.44921 logit=3084.29688 exp=1859.42941 shr=19.9916793 lines=328 END +moesel call=136 dev=1 T=1 E=512 k=10 sel=4caad7e0c0a547a3 minulps=0 mintok=0 x=2086.01662 logit=3108.67188 exp=1556.17088 shr=14.7403516 lines=329 END +moesel call=137 dev=1 T=1 E=512 k=10 sel=c8237987eca74795 minulps=2 mintok=0 x=2162.5299 logit=3213.9375 exp=1971.4318 shr=10.2028748 lines=330 END +moesel call=138 dev=1 T=1 E=512 k=10 sel=051106c8056901f3 minulps=0 mintok=0 x=2246.8277 logit=3174.71875 exp=1949.91024 shr=19.8890601 lines=331 END +moesel call=139 dev=1 T=1 E=512 k=10 sel=97a75e8e00d0b342 minulps=2 mintok=0 x=1882.35801 logit=3092.54688 exp=1340.40161 shr=18.3256835 lines=332 END +moesel call=140 dev=1 T=1 E=512 k=10 sel=713b8ce875e2c311 minulps=1 mintok=0 x=2155.21587 logit=3210.75 exp=1659.56717 shr=21.6259448 lines=333 END +moesel call=141 dev=1 T=1 E=512 k=10 sel=c367f92089653128 minulps=0 mintok=0 x=2231.77249 logit=3202.84375 exp=1724.72208 shr=29.3759355 lines=334 END +moesel call=142 dev=1 T=1 E=512 k=10 sel=ac6441082a2c0801 minulps=2 mintok=0 x=2130.89023 logit=3145.03125 exp=1735.93224 shr=34.0439154 lines=335 END +moesel call=143 dev=1 T=1 E=512 k=10 sel=ef919b7e56630278 minulps=1 mintok=0 x=1974.69898 logit=3181.9375 exp=1722.13843 shr=144.74768 lines=336 END +moesel call=144 dev=1 T=1 E=512 k=10 sel=09c0aa1fa6c8c877 minulps=0 mintok=0 x=709.483277 logit=3420.57812 exp=589.078646 shr=44.0248033 lines=337 END +moesel call=145 dev=1 T=1 E=512 k=10 sel=a916136dafb38ba5 minulps=2 mintok=0 x=805.992065 logit=3073.1875 exp=521.093776 shr=5.51094551 lines=338 END +moesel call=146 dev=1 T=1 E=512 k=10 sel=c3ae50cfa21dff8e minulps=0 mintok=0 x=834.956304 logit=3038.53125 exp=384.249757 shr=8.0188053 lines=339 END +moesel call=147 dev=1 T=1 E=512 k=10 sel=e9f1a22d10c9b363 minulps=1 mintok=0 x=1167.29771 logit=3167.39062 exp=533.255623 shr=3.74983719 lines=340 END +moesel call=148 dev=1 T=1 E=512 k=10 sel=a07ce5b4d7e4c0f4 minulps=0 mintok=0 x=1151.53777 logit=3034.25 exp=610.441006 shr=6.48309856 lines=341 END +moesel call=149 dev=1 T=1 E=512 k=10 sel=b19205a95b25a5a3 minulps=0 mintok=0 x=1082.42335 logit=3101.91406 exp=457.349422 shr=5.85124576 lines=342 END +moesel call=150 dev=1 T=1 E=512 k=10 sel=d705f8437be03c27 minulps=0 mintok=0 x=1201.75989 logit=3197 exp=640.053712 shr=7.00659662 lines=343 END +moesel call=151 dev=1 T=1 E=512 k=10 sel=55930248058bd417 minulps=1 mintok=0 x=1124.08582 logit=3105.375 exp=551.808847 shr=4.42178992 lines=344 END +moesel call=152 dev=1 T=1 E=512 k=10 sel=c8b64ec26f3ea291 minulps=1 mintok=0 x=1129.19646 logit=3045.6875 exp=561.122629 shr=4.68783407 lines=345 END +moesel call=153 dev=1 T=1 E=512 k=10 sel=ec36b67594288457 minulps=2 mintok=0 x=1172.88517 logit=3091.9375 exp=535.528177 shr=5.83415864 lines=346 END +moesel call=154 dev=1 T=1 E=512 k=10 sel=6ceb14ce65e1679f minulps=0 mintok=0 x=1051.11735 logit=3161.3125 exp=505.286261 shr=3.63972754 lines=347 END +moesel call=155 dev=1 T=1 E=512 k=10 sel=3b73bf134ebe2733 minulps=2 mintok=0 x=1057.84783 logit=3087.5 exp=455.834315 shr=3.05580962 lines=348 END +moesel call=156 dev=1 T=1 E=512 k=10 sel=d66b184040443e9e minulps=2 mintok=0 x=1101.54184 logit=3094.53125 exp=536.417945 shr=6.63558327 lines=349 END +moesel call=157 dev=1 T=1 E=512 k=10 sel=33c5d13ae10a34a4 minulps=1 mintok=0 x=979.399919 logit=3017.42188 exp=420.453636 shr=5.07466524 lines=350 END +moesel call=158 dev=1 T=1 E=512 k=10 sel=3901af792cd4064e minulps=2 mintok=0 x=1192.45647 logit=3166.73438 exp=586.611596 shr=4.74686272 lines=351 END +moesel call=159 dev=1 T=1 E=512 k=10 sel=ecea62446c550089 minulps=1 mintok=0 x=1616.97093 logit=3195.76562 exp=930.605066 shr=5.80639515 lines=352 END +moesel call=160 dev=1 T=1 E=512 k=10 sel=6dea3b8b688439cf minulps=3 mintok=0 x=1382.02513 logit=3142.1875 exp=656.488554 shr=4.61721072 lines=353 END +moesel call=161 dev=1 T=1 E=512 k=10 sel=1ab7e1e78d5fd1d8 minulps=13 mintok=0 x=1167.59768 logit=3055.20312 exp=537.666164 shr=5.2065028 lines=354 END +moesel call=162 dev=1 T=1 E=512 k=10 sel=2dcd573eb4fc01b4 minulps=1 mintok=0 x=1053.68328 logit=3182.125 exp=485.981195 shr=3.71340064 lines=355 END +moesel call=163 dev=1 T=1 E=512 k=10 sel=6b0da33229709cca minulps=0 mintok=0 x=1260.96437 logit=3168.07812 exp=612.015255 shr=4.95779561 lines=356 END +moesel call=164 dev=1 T=1 E=512 k=10 sel=c30e84a97933daad minulps=0 mintok=0 x=1504.6571 logit=3194.65625 exp=838.207693 shr=8.71455801 lines=357 END +moesel call=165 dev=1 T=1 E=512 k=10 sel=7bb39829cf126047 minulps=1 mintok=0 x=1409.34995 logit=3223.95312 exp=721.739153 shr=5.49940975 lines=358 END +moesel call=166 dev=1 T=1 E=512 k=10 sel=024eef52bb54717a minulps=3 mintok=0 x=1341.19608 logit=3351.73438 exp=705.514118 shr=4.06778384 lines=359 END +moesel call=167 dev=1 T=1 E=512 k=10 sel=eb52d195dc1e4d9d minulps=1 mintok=0 x=1621.8557 logit=3176.01562 exp=850.487498 shr=9.49747722 lines=360 END +moesel call=168 dev=1 T=1 E=512 k=10 sel=3b2fbe3418eff725 minulps=1 mintok=0 x=1190.36975 logit=3203.04688 exp=602.699007 shr=5.59183539 lines=361 END +moesel call=169 dev=1 T=1 E=512 k=10 sel=7ce39d832fb4a4c3 minulps=4 mintok=0 x=1280.78194 logit=3237.04688 exp=669.382044 shr=5.95203981 lines=362 END +moesel call=170 dev=1 T=1 E=512 k=10 sel=f8c3febace9a2e04 minulps=1 mintok=0 x=1134.26234 logit=3256.14844 exp=563.907842 shr=4.31081375 lines=363 END +moesel call=171 dev=1 T=1 E=512 k=10 sel=f1bd61005bc74eed minulps=1 mintok=0 x=1158.93305 logit=3194.48438 exp=552.877189 shr=4.51102593 lines=364 END +moesel call=172 dev=1 T=1 E=512 k=10 sel=208c78b6b20505f9 minulps=5 mintok=0 x=1209.99551 logit=3279.59375 exp=609.162565 shr=6.4372271 lines=365 END +moesel call=173 dev=1 T=1 E=512 k=10 sel=cdfc5eb8b1cdc02c minulps=10 mintok=0 x=1168.66896 logit=3203.35938 exp=700.878574 shr=6.8646412 lines=366 END +moesel call=174 dev=1 T=1 E=512 k=10 sel=97b8993e354862fe minulps=3 mintok=0 x=1437.9202 logit=3321.5625 exp=883.806022 shr=8.39914337 lines=367 END +moesel call=175 dev=1 T=1 E=512 k=10 sel=ad191e1c27426e2b minulps=1 mintok=0 x=1935.0825 logit=3242.09375 exp=1369.8418 shr=10.7598722 lines=368 END +moesel call=176 dev=1 T=1 E=512 k=10 sel=ddec8052fbe701ac minulps=1 mintok=0 x=1790.82479 logit=3217.04688 exp=1149.80673 shr=10.6727672 lines=369 END +moesel call=177 dev=1 T=1 E=512 k=10 sel=460ed40fbb28ce6e minulps=2 mintok=0 x=1555.90134 logit=3167.17188 exp=1039.83153 shr=8.08700437 lines=370 END +moesel call=178 dev=1 T=1 E=512 k=10 sel=01aec4439a260211 minulps=4 mintok=0 x=1429.73537 logit=3299.67188 exp=908.815103 shr=6.94476729 lines=371 END +moesel call=179 dev=1 T=1 E=512 k=10 sel=50154f5b367db687 minulps=2 mintok=0 x=1618.79316 logit=3195.625 exp=1038.68821 shr=9.20681597 lines=372 END +moesel call=180 dev=1 T=1 E=512 k=10 sel=dd994ce5c1673149 minulps=2 mintok=0 x=1874.35114 logit=3199.82812 exp=1654.29785 shr=23.3827562 lines=373 END +moesel call=181 dev=1 T=1 E=512 k=10 sel=3972ac1752b2eb41 minulps=0 mintok=0 x=1876.30117 logit=3315.125 exp=1407.24905 shr=11.709172 lines=374 END +moesel call=182 dev=1 T=1 E=512 k=10 sel=3f2d4a43d000d03c minulps=1 mintok=0 x=1911.34686 logit=3218.28125 exp=1845.18509 shr=14.4705107 lines=375 END +moesel call=183 dev=1 T=1 E=512 k=10 sel=8404ef6f85cd3288 minulps=0 mintok=0 x=1974.98183 logit=3112.85938 exp=1713.42116 shr=27.1394798 lines=376 END +moesel call=184 dev=1 T=1 E=512 k=10 sel=390caacb7024806b minulps=0 mintok=0 x=1954.29539 logit=3059.90625 exp=1365.72387 shr=16.5918299 lines=377 END +moesel call=185 dev=1 T=1 E=512 k=10 sel=46bbdd5729df570f minulps=1 mintok=0 x=2190.51382 logit=3202.73438 exp=1703.29756 shr=12.784988 lines=378 END +moesel call=186 dev=1 T=1 E=512 k=10 sel=fdd86d7f02a516c9 minulps=1 mintok=0 x=2351.97609 logit=3212.40625 exp=2130.96326 shr=24.0804143 lines=379 END +moesel call=187 dev=1 T=1 E=512 k=10 sel=1266f27edf621c1b minulps=0 mintok=0 x=1896.9653 logit=3094.79688 exp=1334.10299 shr=21.6554436 lines=380 END +moesel call=188 dev=1 T=1 E=512 k=10 sel=57f1e9475aff0171 minulps=0 mintok=0 x=2163.76679 logit=3197.3125 exp=1899.18018 shr=25.5749763 lines=381 END +moesel call=189 dev=1 T=1 E=512 k=10 sel=39a7a8e89337e51a minulps=4 mintok=0 x=2359.99121 logit=3273.03125 exp=1838.0856 shr=22.521865 lines=382 END +moesel call=190 dev=1 T=1 E=512 k=10 sel=861266fd7f73af51 minulps=1 mintok=0 x=2226.46057 logit=3210.28125 exp=1993.76646 shr=48.9299308 lines=383 END +moesel call=191 dev=1 T=1 E=512 k=10 sel=926e58e367c5382d minulps=5 mintok=0 x=2000.70777 logit=3205.5 exp=1690.58827 shr=161.124053 lines=384 END +moesel call=192 dev=1 T=1 E=512 k=10 sel=779637335a2e27ec minulps=0 mintok=0 x=716.147677 logit=3468.875 exp=487.75294 shr=51.390801 lines=385 END +moesel call=193 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=765.411524 logit=3042.59375 exp=447.925371 shr=5.44892272 lines=386 END +moesel call=194 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=0 mintok=0 x=790.436476 logit=3073.125 exp=368.452928 shr=6.00434699 lines=387 END +moesel call=195 dev=1 T=1 E=512 k=10 sel=715855688ae0f4c5 minulps=1 mintok=0 x=1146.69417 logit=3151.15625 exp=585.124857 shr=3.69654149 lines=388 END +moesel call=196 dev=1 T=1 E=512 k=10 sel=e75f20bfc53651de minulps=4 mintok=0 x=1099.03376 logit=2972.45312 exp=545.371254 shr=6.2964395 lines=389 END +moesel call=197 dev=1 T=1 E=512 k=10 sel=e199dea9a8bf81ec minulps=1 mintok=0 x=1101.94353 logit=3080.90625 exp=518.980939 shr=5.99029895 lines=390 END +moesel call=198 dev=1 T=1 E=512 k=10 sel=d07e4ef7dedbb018 minulps=0 mintok=0 x=1236.59581 logit=3145.20312 exp=672.002836 shr=7.11468908 lines=391 END +moesel call=199 dev=1 T=1 E=512 k=10 sel=b24da9a0b8ab6f1c minulps=0 mintok=0 x=1121.32949 logit=3087.5 exp=562.042839 shr=6.24267779 lines=392 END +moesel call=200 dev=1 T=1 E=512 k=10 sel=bf0a220efb56ee89 minulps=0 mintok=0 x=1105.57162 logit=3081.57812 exp=557.250946 shr=5.51314575 lines=393 END +moesel call=201 dev=1 T=1 E=512 k=10 sel=325e66aef4c1aa8a minulps=2 mintok=0 x=1177.02908 logit=3086.60938 exp=541.972698 shr=6.67578374 lines=394 END +moesel call=202 dev=1 T=1 E=512 k=10 sel=d2f4cd6d11682004 minulps=1 mintok=0 x=1060.25186 logit=3121.77344 exp=501.771266 shr=3.73486739 lines=395 END +moesel call=203 dev=1 T=1 E=512 k=10 sel=89bf9a12d44a0482 minulps=4 mintok=0 x=1063.31313 logit=3036.1875 exp=428.680316 shr=3.30325386 lines=396 END +moesel call=204 dev=1 T=1 E=512 k=10 sel=f3b171608a86ada7 minulps=3 mintok=0 x=1110.2053 logit=3080.90625 exp=539.927993 shr=6.87042566 lines=397 END +moesel call=205 dev=1 T=1 E=512 k=10 sel=5a340e8c3bc82a78 minulps=0 mintok=0 x=975.890717 logit=3075.875 exp=431.624742 shr=5.38765036 lines=398 END +moesel call=206 dev=1 T=1 E=512 k=10 sel=cb068b058d469c85 minulps=6 mintok=0 x=1176.68664 logit=3175.10938 exp=616.773291 shr=5.05151851 lines=399 END +moesel call=207 dev=1 T=1 E=512 k=10 sel=314184de6bfb7963 minulps=1 mintok=0 x=1546.15161 logit=3148.78125 exp=927.606893 shr=5.13259371 lines=400 END +moesel call=208 dev=1 T=1 E=512 k=10 sel=8b82a962f008f00e minulps=1 mintok=0 x=1366.02425 logit=3048.25 exp=682.192104 shr=3.56387435 lines=401 END +moesel call=209 dev=1 T=1 E=512 k=10 sel=120081ac18996dac minulps=0 mintok=0 x=1167.5068 logit=3007.96875 exp=524.051682 shr=4.2190643 lines=402 END +moesel call=210 dev=1 T=1 E=512 k=10 sel=25a8a4bc71d12bfa minulps=3 mintok=0 x=1100.30862 logit=3195.85938 exp=502.684609 shr=3.75043126 lines=403 END +moesel call=211 dev=1 T=1 E=512 k=10 sel=001509a3e0b4ce7e minulps=1 mintok=0 x=1287.11083 logit=3201.375 exp=613.999208 shr=4.0550633 lines=404 END +moesel call=212 dev=1 T=1 E=512 k=10 sel=41744aa8dbe9161f minulps=1 mintok=0 x=1570.91101 logit=3210.96875 exp=933.894274 shr=10.2405203 lines=405 END +moesel call=213 dev=1 T=1 E=512 k=10 sel=4287716436e44132 minulps=2 mintok=0 x=1437.59795 logit=3186.28125 exp=735.577279 shr=5.56163217 lines=406 END +moesel call=214 dev=1 T=1 E=512 k=10 sel=5dea9924948c0e3c minulps=0 mintok=0 x=1385.44258 logit=3246.20312 exp=708.992863 shr=3.78521363 lines=407 END +moesel call=215 dev=1 T=1 E=512 k=10 sel=9458619ca2d1b7a1 minulps=1 mintok=0 x=1692.75305 logit=3189.625 exp=944.580063 shr=10.9508233 lines=408 END +moesel call=216 dev=1 T=1 E=512 k=10 sel=24fdfa6aa86a5f52 minulps=5 mintok=0 x=1224.17387 logit=3162.39062 exp=613.257584 shr=5.39118691 lines=409 END +moesel call=217 dev=1 T=1 E=512 k=10 sel=f16f12b2e4688234 minulps=2 mintok=0 x=1306.69958 logit=3199.79688 exp=639.83127 shr=6.10713226 lines=410 END +moesel call=218 dev=1 T=1 E=512 k=10 sel=ac4f5ffd3e72d566 minulps=5 mintok=0 x=1143.23705 logit=3216.25781 exp=561.018667 shr=4.16465363 lines=411 END +moesel call=219 dev=1 T=1 E=512 k=10 sel=38dafeab32fd4282 minulps=1 mintok=0 x=1198.31271 logit=3162.76562 exp=592.81153 shr=4.26553735 lines=412 END +moesel call=220 dev=1 T=1 E=512 k=10 sel=d4e2b8a9288e46db minulps=2 mintok=0 x=1257.38946 logit=3255.40625 exp=696.537776 shr=5.32452663 lines=413 END +moesel call=221 dev=1 T=1 E=512 k=10 sel=2f65244a0856f622 minulps=1 mintok=0 x=1229.23446 logit=3239.89062 exp=690.785637 shr=7.22456931 lines=414 END +moesel call=222 dev=1 T=1 E=512 k=10 sel=fa0039720647820e minulps=2 mintok=0 x=1489.03941 logit=3324.375 exp=885.436165 shr=7.73208681 lines=415 END +moesel call=223 dev=1 T=1 E=512 k=10 sel=cf39959fe1a82ffc minulps=8 mintok=0 x=1920.93703 logit=3248.79688 exp=1267.45386 shr=9.17116281 lines=416 END +moesel call=224 dev=1 T=1 E=512 k=10 sel=a8f3ea369e9e86d6 minulps=5 mintok=0 x=1808.22214 logit=3195.85938 exp=1095.51868 shr=8.29888845 lines=417 END +moesel call=225 dev=1 T=1 E=512 k=10 sel=95cabe388a3fea46 minulps=1 mintok=0 x=1533.78564 logit=3201.78125 exp=944.96525 shr=7.15416351 lines=418 END +moesel call=226 dev=1 T=1 E=512 k=10 sel=aa45077b077b6962 minulps=2 mintok=0 x=1425.75299 logit=3382.39062 exp=853.198978 shr=4.64491395 lines=419 END +moesel call=227 dev=1 T=1 E=512 k=10 sel=5f6726bda4071304 minulps=1 mintok=0 x=1709.32 logit=3314.23438 exp=1157.89745 shr=8.22336807 lines=420 END +moesel call=228 dev=1 T=1 E=512 k=10 sel=30cb53ea52bfef17 minulps=1 mintok=0 x=2139.26325 logit=3304.17188 exp=1901.37701 shr=20.5340446 lines=421 END +moesel call=229 dev=1 T=1 E=512 k=10 sel=3da639862025f4d6 minulps=2 mintok=0 x=1999.99795 logit=3264.89062 exp=1386.41496 shr=11.2634649 lines=422 END +moesel call=230 dev=1 T=1 E=512 k=10 sel=cb19c87acd501fc0 minulps=0 mintok=0 x=1938.46453 logit=3120.98438 exp=1874.35078 shr=15.390727 lines=423 END +moesel call=231 dev=1 T=1 E=512 k=10 sel=b04de2f650660194 minulps=0 mintok=0 x=2085.96049 logit=3063.39062 exp=1603.72511 shr=21.0686349 lines=424 END +moesel call=232 dev=1 T=1 E=512 k=10 sel=e6d6f8a353fc5f0b minulps=1 mintok=0 x=2178.48807 logit=3108.0625 exp=1554.84994 shr=15.8733611 lines=425 END +moesel call=233 dev=1 T=1 E=512 k=10 sel=027cb00b19eaecc9 minulps=2 mintok=0 x=2198.12783 logit=3227.64062 exp=1867.98647 shr=11.2329605 lines=426 END +moesel call=234 dev=1 T=1 E=512 k=10 sel=89dfd7a13488d3e8 minulps=2 mintok=0 x=2255.41025 logit=3256.25 exp=1868.25884 shr=19.0055339 lines=427 END +moesel call=235 dev=1 T=1 E=512 k=10 sel=f8a7a5bd176b326e minulps=4 mintok=0 x=1909.94999 logit=3164.17188 exp=1401.34578 shr=18.5656592 lines=428 END +moesel call=236 dev=1 T=1 E=512 k=10 sel=b7744dee46c05f1a minulps=1 mintok=0 x=2226.24713 logit=3248.5625 exp=1988.42235 shr=23.1235812 lines=429 END +moesel call=237 dev=1 T=1 E=512 k=10 sel=762dcd493bb39ada minulps=1 mintok=0 x=2310.78488 logit=3242.46875 exp=1783.5975 shr=28.7035796 lines=430 END +moesel call=238 dev=1 T=1 E=512 k=10 sel=0dde70617e68873f minulps=5 mintok=0 x=2169.89524 logit=3210.82812 exp=1672.0583 shr=38.2316203 lines=431 END +moesel call=239 dev=1 T=1 E=512 k=10 sel=281cca5db19787bc minulps=0 mintok=0 x=2013.20563 logit=3224.53125 exp=1636.04632 shr=149.592793 lines=432 END +moesel call=240 dev=1 T=1 E=512 k=10 sel=61b49097fc368a4f minulps=1 mintok=0 x=720.28059 logit=3485.40625 exp=483.675413 shr=54.8119662 lines=433 END +moesel call=241 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=771.935771 logit=3070.79688 exp=417.276456 shr=5.94142606 lines=434 END +moesel call=242 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=3 mintok=0 x=787.540872 logit=3067.625 exp=402.074727 shr=7.02242934 lines=435 END +moesel call=243 dev=1 T=1 E=512 k=10 sel=fcadc8fa2617fb34 minulps=2 mintok=0 x=1124.66685 logit=3166.34375 exp=580.25582 shr=3.28912977 lines=436 END +moesel call=244 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1082.60521 logit=2976.375 exp=527.938656 shr=5.48257715 lines=437 END +moesel call=245 dev=1 T=1 E=512 k=10 sel=6c3ea50f741c1689 minulps=3 mintok=0 x=1125.42378 logit=3122.9375 exp=511.366771 shr=4.98313789 lines=438 END +moesel call=246 dev=1 T=1 E=512 k=10 sel=0aa7734f47636857 minulps=1 mintok=0 x=1323.94484 logit=3165.26562 exp=671.581911 shr=8.14461842 lines=439 END +moesel call=247 dev=1 T=1 E=512 k=10 sel=8e15ae330207bff0 minulps=1 mintok=0 x=1173.52005 logit=3075.85938 exp=571.254446 shr=6.22336972 lines=440 END +moesel call=248 dev=1 T=1 E=512 k=10 sel=0c140a8e49a781fd minulps=3 mintok=0 x=1183.84978 logit=3052.59375 exp=604.445688 shr=6.84836599 lines=441 END +moesel call=249 dev=1 T=1 E=512 k=10 sel=db717a9b30383fe4 minulps=3 mintok=0 x=1211.28881 logit=3059.46875 exp=580.742396 shr=5.14751594 lines=442 END +moesel call=250 dev=1 T=1 E=512 k=10 sel=59619e2e9965e888 minulps=1 mintok=0 x=1083.64493 logit=3153.60938 exp=503.252649 shr=3.60288797 lines=443 END +moesel call=251 dev=1 T=1 E=512 k=10 sel=74d1d7272204b827 minulps=5 mintok=0 x=1078.65814 logit=3063.03125 exp=474.361346 shr=2.34464275 lines=444 END +moesel call=252 dev=1 T=1 E=512 k=10 sel=72c14e01f28ffe9a minulps=0 mintok=0 x=1123.4959 logit=3052.09375 exp=527.66608 shr=5.18539597 lines=445 END +moesel call=253 dev=1 T=1 E=512 k=10 sel=d386a2016da631fd minulps=3 mintok=0 x=996.801906 logit=3047.92188 exp=417.467618 shr=4.93362171 lines=446 END +moesel call=254 dev=1 T=1 E=512 k=10 sel=e73cb09b45b079b4 minulps=6 mintok=0 x=1224.72177 logit=3078.6875 exp=587.41722 shr=4.98894918 lines=447 END +moesel call=255 dev=1 T=1 E=512 k=10 sel=3c461f614b5b675e minulps=1 mintok=0 x=1634.41036 logit=3118.46875 exp=900.135087 shr=8.71689782 lines=448 END +moesel call=256 dev=1 T=1 E=512 k=10 sel=f162918a3e59d49e minulps=1 mintok=0 x=1441.67168 logit=3060.29688 exp=718.891064 shr=7.18912839 lines=449 END +moesel call=257 dev=1 T=1 E=512 k=10 sel=5beb06c103212760 minulps=2 mintok=0 x=1211.66689 logit=3142.42188 exp=613.215454 shr=5.64889552 lines=450 END +moesel call=258 dev=1 T=1 E=512 k=10 sel=40423a978ab31a36 minulps=1 mintok=0 x=1087.29929 logit=3215.6875 exp=473.434067 shr=3.84199879 lines=451 END +moesel call=259 dev=1 T=1 E=512 k=10 sel=de4339573503bcd4 minulps=1 mintok=0 x=1264.18889 logit=3241.79688 exp=593.864389 shr=4.92873746 lines=452 END +moesel call=260 dev=1 T=1 E=512 k=10 sel=f790f933190f981e minulps=2 mintok=0 x=1497.17991 logit=3274.29688 exp=820.056975 shr=8.91404494 lines=453 END +moesel call=261 dev=1 T=1 E=512 k=10 sel=a6b70eb4e1969847 minulps=3 mintok=0 x=1415.73308 logit=3250.04688 exp=680.218217 shr=4.77076492 lines=454 END +moesel call=262 dev=1 T=1 E=512 k=10 sel=7b7a3168b2ff06a2 minulps=7 mintok=0 x=1350.50629 logit=3259.57812 exp=654.255479 shr=3.91195978 lines=455 END +moesel call=263 dev=1 T=1 E=512 k=10 sel=f5590947f667141f minulps=1 mintok=0 x=1614.07453 logit=3184.35938 exp=860.567514 shr=7.31600365 lines=456 END +moesel call=264 dev=1 T=1 E=512 k=10 sel=ef6c35ec164a8a49 minulps=0 mintok=0 x=1205.80305 logit=3202.09375 exp=586.778015 shr=5.23748323 lines=457 END +moesel call=265 dev=1 T=1 E=512 k=10 sel=fb96ce369453d56b minulps=2 mintok=0 x=1305.60037 logit=3266.23438 exp=606.43976 shr=5.04525727 lines=458 END +moesel call=266 dev=1 T=1 E=512 k=10 sel=933d7afaff8cc437 minulps=10 mintok=0 x=1109.77298 logit=3258.35156 exp=548.58202 shr=3.68036258 lines=459 END +moesel call=267 dev=1 T=1 E=512 k=10 sel=7cbbfaa10ca363ad minulps=2 mintok=0 x=1186.14458 logit=3144.14062 exp=535.223339 shr=5.36935256 lines=460 END +moesel call=268 dev=1 T=1 E=512 k=10 sel=100a651971679b44 minulps=2 mintok=0 x=1243.57518 logit=3286.28125 exp=637.414101 shr=6.02404639 lines=461 END +moesel call=269 dev=1 T=1 E=512 k=10 sel=a4fdb20db9cc56a8 minulps=0 mintok=0 x=1234.17061 logit=3215.42188 exp=652.155181 shr=9.21329071 lines=462 END +moesel call=270 dev=1 T=1 E=512 k=10 sel=d551b1ca5da9dbb5 minulps=1 mintok=0 x=1431.30979 logit=3378.96875 exp=738.425532 shr=7.25745194 lines=463 END +moesel call=271 dev=1 T=1 E=512 k=10 sel=dcef15817b363caa minulps=2 mintok=0 x=1662.58431 logit=3247.28125 exp=987.379433 shr=11.751852 lines=464 END +moesel call=272 dev=1 T=1 E=512 k=10 sel=1860028c3edaf876 minulps=1 mintok=0 x=1540.5087 logit=3175.40625 exp=837.511548 shr=10.8390061 lines=465 END +moesel call=273 dev=1 T=1 E=512 k=10 sel=1186c47d631d5f82 minulps=0 mintok=0 x=1431.05815 logit=3268.01562 exp=800.615634 shr=6.03382078 lines=466 END +moesel call=274 dev=1 T=1 E=512 k=10 sel=26c585a5a9a96de1 minulps=4 mintok=0 x=1368.66555 logit=3365.40625 exp=784.150686 shr=5.20184667 lines=467 END +moesel call=275 dev=1 T=1 E=512 k=10 sel=5447e5522c9978ba minulps=0 mintok=0 x=1576.16108 logit=3258.95312 exp=980.637357 shr=7.11631892 lines=468 END +moesel call=276 dev=1 T=1 E=512 k=10 sel=52face1eca90650c minulps=2 mintok=0 x=1852.7599 logit=3289.90625 exp=1242.96738 shr=17.6701261 lines=469 END +moesel call=277 dev=1 T=1 E=512 k=10 sel=c37a90b706905352 minulps=0 mintok=0 x=1826.76319 logit=3325.07812 exp=1238.72894 shr=11.1069868 lines=470 END +moesel call=278 dev=1 T=1 E=512 k=10 sel=72cfbe3454d4b6de minulps=1 mintok=0 x=1881.6622 logit=3186.0625 exp=1661.28122 shr=13.5915908 lines=471 END +moesel call=279 dev=1 T=1 E=512 k=10 sel=bfd9f1aaa5d844d7 minulps=1 mintok=0 x=1906.10385 logit=3046.67188 exp=1310.62947 shr=21.4008919 lines=472 END +moesel call=280 dev=1 T=1 E=512 k=10 sel=4f1e914b26764374 minulps=0 mintok=0 x=2037.13792 logit=3109.51562 exp=1402.99566 shr=14.5560947 lines=473 END +moesel call=281 dev=1 T=1 E=512 k=10 sel=5fba2b1dbb53abc0 minulps=1 mintok=0 x=1989.86975 logit=3266 exp=1452.78641 shr=9.4343206 lines=474 END +moesel call=282 dev=1 T=1 E=512 k=10 sel=18b6530e503683e3 minulps=4 mintok=0 x=2156.46802 logit=3193.28125 exp=1845.50453 shr=22.3346339 lines=475 END +moesel call=283 dev=1 T=1 E=512 k=10 sel=89b0aadca1054264 minulps=0 mintok=0 x=1886.75411 logit=3152.70312 exp=1358.00446 shr=18.8158455 lines=476 END +moesel call=284 dev=1 T=1 E=512 k=10 sel=b9719722eba427d0 minulps=1 mintok=0 x=2217.40372 logit=3220.92188 exp=1993.46938 shr=21.7836 lines=477 END +moesel call=285 dev=1 T=1 E=512 k=10 sel=71bbaf05cdb5130d minulps=0 mintok=0 x=2319.32773 logit=3235.875 exp=1773.67931 shr=25.0276712 lines=478 END +moesel call=286 dev=1 T=1 E=512 k=10 sel=0b57979055d8fdca minulps=1 mintok=0 x=2185.67904 logit=3188.79688 exp=1739.53171 shr=39.9090071 lines=479 END +moesel call=287 dev=1 T=1 E=512 k=10 sel=70924589f171ffe4 minulps=1 mintok=0 x=2032.05483 logit=3193.03125 exp=1692.53224 shr=153.420908 lines=480 END +moesel call=288 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=1 mintok=0 x=720.856046 logit=3492.75 exp=470.546575 shr=56.0767549 lines=481 END +moesel call=289 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=0 mintok=0 x=770.703693 logit=3051.84375 exp=392.605082 shr=6.61083775 lines=482 END +moesel call=290 dev=1 T=1 E=512 k=10 sel=19a4aa22fd55ceb9 minulps=0 mintok=0 x=797.399337 logit=3060.17188 exp=449.968985 shr=7.74540238 lines=483 END +moesel call=291 dev=1 T=1 E=512 k=10 sel=2a01e5c40d6cc0a3 minulps=1 mintok=0 x=1130.79295 logit=3148.03125 exp=630.561784 shr=3.20717122 lines=484 END +moesel call=292 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1107.32255 logit=3057.42188 exp=612.944122 shr=4.60172829 lines=485 END +moesel call=293 dev=1 T=1 E=512 k=10 sel=5bfbcc85bf27e03a minulps=0 mintok=0 x=1195.62512 logit=3226.40625 exp=568.449331 shr=5.80444346 lines=486 END +moesel call=294 dev=1 T=1 E=512 k=10 sel=872e07b269673330 minulps=0 mintok=0 x=1356.65116 logit=3237.53125 exp=760.423626 shr=8.0394378 lines=487 END +moesel call=295 dev=1 T=1 E=512 k=10 sel=5ca98857c1285dfe minulps=1 mintok=0 x=1237.19989 logit=3187.35938 exp=636.745745 shr=6.45760946 lines=488 END +moesel call=296 dev=1 T=1 E=512 k=10 sel=2808381ab44ad57f minulps=0 mintok=0 x=1263.79819 logit=3167.4375 exp=663.189906 shr=8.30687244 lines=489 END +moesel call=297 dev=1 T=1 E=512 k=10 sel=1e6ef2e2bd5bbe6e minulps=1 mintok=0 x=1308.03402 logit=3105.6875 exp=655.636429 shr=6.0368379 lines=490 END +moesel call=298 dev=1 T=1 E=512 k=10 sel=6d9a04e77a545573 minulps=0 mintok=0 x=1146.06395 logit=3228.01562 exp=597.569373 shr=4.32204409 lines=491 END +moesel call=299 dev=1 T=1 E=512 k=10 sel=5386a17f0a85355d minulps=0 mintok=0 x=1177.25779 logit=3092.0625 exp=561.144001 shr=3.26688719 lines=492 END +moesel call=300 dev=1 T=1 E=512 k=10 sel=6df890014a37fbfa minulps=2 mintok=0 x=1217.40623 logit=3067.8125 exp=577.586853 shr=4.4797349 lines=493 END +moesel call=301 dev=1 T=1 E=512 k=10 sel=a7d9e1f34c1f1e03 minulps=5 mintok=0 x=1056.97793 logit=3131.34375 exp=458.905468 shr=5.41991125 lines=494 END +moesel call=302 dev=1 T=1 E=512 k=10 sel=cbcf5808f868f2ee minulps=2 mintok=0 x=1295.18274 logit=3124.60938 exp=776.900305 shr=6.12398742 lines=495 END +moesel call=303 dev=1 T=1 E=512 k=10 sel=0b1314471336dbd0 minulps=1 mintok=0 x=1615.90277 logit=3082.39062 exp=928.556103 shr=10.2991807 lines=496 END +moesel call=304 dev=1 T=1 E=512 k=10 sel=81f99954a1527b57 minulps=1 mintok=0 x=1461.03125 logit=3065.98438 exp=717.857508 shr=6.48855598 lines=497 END +moesel call=305 dev=1 T=1 E=512 k=10 sel=e3d6b1ce22c8075e minulps=6 mintok=0 x=1269.97015 logit=3140.25 exp=619.990238 shr=4.45730279 lines=498 END +moesel call=306 dev=1 T=1 E=512 k=10 sel=d157df379a8be237 minulps=2 mintok=0 x=1184.60336 logit=3220.07812 exp=606.101411 shr=3.86488661 lines=499 END +moesel call=307 dev=1 T=1 E=512 k=10 sel=2a2ee430acd400cb minulps=5 mintok=0 x=1291.61908 logit=3141.15625 exp=661.987925 shr=4.67138139 lines=500 END +moesel call=308 dev=1 T=1 E=512 k=10 sel=c05e3ebbece289a1 minulps=1 mintok=0 x=1514.79625 logit=3150.875 exp=900.476488 shr=9.86281496 lines=501 END +moesel call=309 dev=1 T=1 E=512 k=10 sel=ef5beefc010fbc10 minulps=2 mintok=0 x=1490.35327 logit=3213.45312 exp=797.527022 shr=4.82737862 lines=502 END +moesel call=310 dev=1 T=1 E=512 k=10 sel=eda1b89e68411e03 minulps=6 mintok=0 x=1397.02388 logit=3237.34375 exp=733.108261 shr=4.77679757 lines=503 END +moesel call=311 dev=1 T=1 E=512 k=10 sel=075f24b2bb439374 minulps=1 mintok=0 x=1562.3066 logit=3135.10938 exp=756.076709 shr=12.6417682 lines=504 END +moesel call=312 dev=1 T=1 E=512 k=10 sel=afa97bd21171c051 minulps=0 mintok=0 x=1236.9966 logit=3235.8125 exp=623.439627 shr=5.93616664 lines=505 END +moesel call=313 dev=1 T=1 E=512 k=10 sel=cf66e769dbb64fc5 minulps=1 mintok=0 x=1318.87473 logit=3333.09375 exp=612.478401 shr=5.13968842 lines=506 END +moesel call=314 dev=1 T=1 E=512 k=10 sel=5689cecff7b5d5fa minulps=3 mintok=0 x=1129.20906 logit=3255.76562 exp=557.015241 shr=4.12521553 lines=507 END +moesel call=315 dev=1 T=1 E=512 k=10 sel=e4e002100e756a6a minulps=1 mintok=0 x=1203.42205 logit=3248.04688 exp=514.741366 shr=4.05207102 lines=508 END +moesel call=316 dev=1 T=1 E=512 k=10 sel=6f7356e1764238df minulps=0 mintok=0 x=1261.03246 logit=3370.92188 exp=619.130554 shr=4.29648276 lines=509 END +moesel call=317 dev=1 T=1 E=512 k=10 sel=ff0e8dba386525fc minulps=1 mintok=0 x=1235.21792 logit=3276.48438 exp=639.089688 shr=6.69108815 lines=510 END +moesel call=318 dev=1 T=1 E=512 k=10 sel=c1b1d02eae132273 minulps=0 mintok=0 x=1509.74702 logit=3320.64062 exp=797.675336 shr=4.56329502 lines=511 END +moesel call=319 dev=1 T=1 E=512 k=10 sel=afa51735733f0336 minulps=2 mintok=0 x=1753.00647 logit=3180.0625 exp=1001.70105 shr=7.61484663 lines=512 END +moesel call=320 dev=1 T=1 E=512 k=10 sel=0e0a4109a6700048 minulps=3 mintok=0 x=1699.79817 logit=3207.42188 exp=911.617586 shr=6.69794951 lines=513 END +moesel call=321 dev=1 T=1 E=512 k=10 sel=a6826b4479452d20 minulps=1 mintok=0 x=1546.95507 logit=3408.875 exp=895.016724 shr=4.04376189 lines=514 END +moesel call=322 dev=1 T=1 E=512 k=10 sel=8eda5f9a5229ddbb minulps=3 mintok=0 x=1436.64717 logit=3439.375 exp=840.710471 shr=4.31456018 lines=515 END +moesel call=323 dev=1 T=1 E=512 k=10 sel=d5ad71babb6fc5c7 minulps=0 mintok=0 x=1660.51322 logit=3350.73438 exp=977.062221 shr=5.02763242 lines=516 END +moesel call=324 dev=1 T=1 E=512 k=10 sel=dd12cf205d559b5f minulps=0 mintok=0 x=2002.32102 logit=3385.6875 exp=1631.89902 shr=15.2682286 lines=517 END +moesel call=325 dev=1 T=1 E=512 k=10 sel=703f6b74f64cf6b8 minulps=2 mintok=0 x=1954.07742 logit=3413.625 exp=1309.03405 shr=7.16108526 lines=518 END +moesel call=326 dev=1 T=1 E=512 k=10 sel=992f4482448a2e26 minulps=1 mintok=0 x=1946.47071 logit=3227.64062 exp=1703.92125 shr=16.0447938 lines=519 END +moesel call=327 dev=1 T=1 E=512 k=10 sel=afad6d7161729fbf minulps=0 mintok=0 x=1820.88432 logit=3014.78125 exp=1211.98775 shr=22.5960127 lines=520 END +moesel call=328 dev=1 T=1 E=512 k=10 sel=4b4b9ea682bbc8dd minulps=2 mintok=0 x=2039.01312 logit=3099.79688 exp=1363.61623 shr=16.1566821 lines=521 END +moesel call=329 dev=1 T=1 E=512 k=10 sel=d22f8948640f21f9 minulps=4 mintok=0 x=2040.4512 logit=3155.51562 exp=1436.01293 shr=11.5047986 lines=522 END +moesel call=330 dev=1 T=1 E=512 k=10 sel=5196aed810d04733 minulps=2 mintok=0 x=2173.83652 logit=3217.875 exp=1889.13136 shr=20.5893423 lines=523 END +moesel call=331 dev=1 T=1 E=512 k=10 sel=328a3aca9d96518b minulps=2 mintok=0 x=1944.52967 logit=3112.14062 exp=1430.88715 shr=21.6439633 lines=524 END +moesel call=332 dev=1 T=1 E=512 k=10 sel=e656fd5283a60b62 minulps=5 mintok=0 x=2199.25831 logit=3197.28125 exp=1847.42863 shr=23.1650873 lines=525 END +moesel call=333 dev=1 T=1 E=512 k=10 sel=987174c07c2caf33 minulps=1 mintok=0 x=2261.67968 logit=3255.39062 exp=1604.87861 shr=24.7171497 lines=526 END +moesel call=334 dev=1 T=1 E=512 k=10 sel=988d1ff90a540b9e minulps=0 mintok=0 x=2125.40543 logit=3217.32812 exp=1541.53301 shr=33.6606202 lines=527 END +moesel call=335 dev=1 T=1 E=512 k=10 sel=86f7247dd3bef976 minulps=4 mintok=0 x=1955.88045 logit=3172.40625 exp=1570.11116 shr=131.855621 lines=528 END +moesel call=336 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=0 mintok=0 x=720.338139 logit=3493.625 exp=464.303988 shr=56.7048903 lines=529 END +moesel call=337 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=1 mintok=0 x=769.10639 logit=3054.76562 exp=382.768493 shr=6.88952735 lines=530 END +moesel call=338 dev=1 T=1 E=512 k=10 sel=da8470c4d884deef minulps=1 mintok=0 x=782.368277 logit=3049.375 exp=451.578546 shr=8.05544319 lines=531 END +moesel call=339 dev=1 T=1 E=512 k=10 sel=d71ea9ac6dcf42f0 minulps=1 mintok=0 x=1124.95057 logit=3140.40625 exp=528.174591 shr=3.15244679 lines=532 END +moesel call=340 dev=1 T=1 E=512 k=10 sel=1d8b22207c3f012b minulps=2 mintok=0 x=1105.89104 logit=3068.79688 exp=613.785158 shr=4.04117057 lines=533 END +moesel call=341 dev=1 T=1 E=512 k=10 sel=dddd50c876a89191 minulps=0 mintok=0 x=1149.2405 logit=3142.51562 exp=509.933435 shr=5.5840464 lines=534 END +moesel call=342 dev=1 T=1 E=512 k=10 sel=b0359bc2c09a1b3b minulps=3 mintok=0 x=1298.50399 logit=3192.84375 exp=713.807125 shr=7.99223403 lines=535 END +moesel call=343 dev=1 T=1 E=512 k=10 sel=6a520a326b8a25b7 minulps=0 mintok=0 x=1194.55711 logit=3172.79688 exp=590.636703 shr=6.75776705 lines=536 END +moesel call=344 dev=1 T=1 E=512 k=10 sel=7d588424fcd98bcc minulps=1 mintok=0 x=1202.00878 logit=3131.35938 exp=603.862679 shr=8.54666037 lines=537 END +moesel call=345 dev=1 T=1 E=512 k=10 sel=31626831636724dd minulps=2 mintok=0 x=1226.89993 logit=3080.14062 exp=615.39382 shr=6.22751211 lines=538 END +moesel call=346 dev=1 T=1 E=512 k=10 sel=43f426a771e6476d minulps=2 mintok=0 x=1110.41097 logit=3227.71875 exp=600.466138 shr=3.9388314 lines=539 END +moesel call=347 dev=1 T=1 E=512 k=10 sel=9255641d811fa14b minulps=1 mintok=0 x=1198.20226 logit=3134.39062 exp=546.312478 shr=3.36125358 lines=540 END +moesel call=348 dev=1 T=1 E=512 k=10 sel=16b6bff91911aacc minulps=1 mintok=0 x=1235.28627 logit=3116.42188 exp=607.790884 shr=4.53681195 lines=541 END +moesel call=349 dev=1 T=1 E=512 k=10 sel=7b00267773828180 minulps=1 mintok=0 x=1093.17051 logit=3140.20312 exp=477.441844 shr=5.05125144 lines=542 END +moesel call=350 dev=1 T=1 E=512 k=10 sel=3c5c42aa762d7ed2 minulps=0 mintok=0 x=1297.56948 logit=3115.23438 exp=800.623956 shr=4.55729351 lines=543 END +moesel call=351 dev=1 T=1 E=512 k=10 sel=a1c98a491fdf8f53 minulps=1 mintok=0 x=1630.07224 logit=3116.92188 exp=847.179979 shr=9.87963495 lines=544 END +moesel call=352 dev=1 T=1 E=512 k=10 sel=36c2b92ae5213cf2 minulps=2 mintok=0 x=1486.85555 logit=3080.14062 exp=744.776459 shr=6.95439906 lines=545 END +moesel call=353 dev=1 T=1 E=512 k=10 sel=da733df3a073e796 minulps=3 mintok=0 x=1310.67543 logit=3194.54688 exp=623.855955 shr=4.44543065 lines=546 END +moesel call=354 dev=1 T=1 E=512 k=10 sel=1a70d14b8759a6a7 minulps=0 mintok=0 x=1191.35604 logit=3210.07812 exp=561.163461 shr=3.8106307 lines=547 END +moesel call=355 dev=1 T=1 E=512 k=10 sel=c7786e018fa0b376 minulps=0 mintok=0 x=1356.12887 logit=3178.14062 exp=635.307415 shr=4.51006264 lines=548 END +moesel call=356 dev=1 T=1 E=512 k=10 sel=bb5711b42a292ab8 minulps=0 mintok=0 x=1590.40884 logit=3208.67188 exp=901.43754 shr=7.99312962 lines=549 END +moesel call=357 dev=1 T=1 E=512 k=10 sel=1b95d55c3f33adbb minulps=1 mintok=0 x=1539.14113 logit=3263.67188 exp=785.560501 shr=4.4835915 lines=550 END +moesel call=358 dev=1 T=1 E=512 k=10 sel=bc6664ad83820499 minulps=2 mintok=0 x=1384.95764 logit=3234.65625 exp=734.015604 shr=4.67692428 lines=551 END +moesel call=359 dev=1 T=1 E=512 k=10 sel=935ebb549d36348a minulps=0 mintok=0 x=1625.68447 logit=3217.82812 exp=851.083263 shr=9.25576943 lines=552 END +moesel call=360 dev=1 T=1 E=512 k=10 sel=c14c2922f69377f4 minulps=3 mintok=0 x=1264.72816 logit=3181.32812 exp=646.316954 shr=5.49239905 lines=553 END +moesel call=361 dev=1 T=1 E=512 k=10 sel=3b98798a42003bd3 minulps=4 mintok=0 x=1367.85755 logit=3292.75 exp=605.989986 shr=4.37376307 lines=554 END +moesel call=362 dev=1 T=1 E=512 k=10 sel=9807414b2758b93b minulps=1 mintok=0 x=1156.36278 logit=3238.89844 exp=602.228519 shr=3.92336114 lines=555 END +moesel call=363 dev=1 T=1 E=512 k=10 sel=e91bee73ea7d500a minulps=2 mintok=0 x=1257.65634 logit=3272.20312 exp=580.206607 shr=3.96203512 lines=556 END +moesel call=364 dev=1 T=1 E=512 k=10 sel=5bf4960145709db9 minulps=2 mintok=0 x=1299.88866 logit=3405.14062 exp=642.298995 shr=3.61211545 lines=557 END +moesel call=365 dev=1 T=1 E=512 k=10 sel=4c383640b19fd6e2 minulps=0 mintok=0 x=1260.04282 logit=3294.4375 exp=663.644972 shr=6.15692358 lines=558 END +moesel call=366 dev=1 T=1 E=512 k=10 sel=016b1b4bfb070eb2 minulps=1 mintok=0 x=1536.37145 logit=3296.45312 exp=843.175764 shr=4.8903624 lines=559 END +moesel call=367 dev=1 T=1 E=512 k=10 sel=e7350d2ead76a3c0 minulps=1 mintok=0 x=1859.7461 logit=3306.5 exp=1200.1414 shr=9.07440132 lines=560 END +moesel call=368 dev=1 T=1 E=512 k=10 sel=6ca0cd00f758e308 minulps=1 mintok=0 x=1722.59768 logit=3267.75 exp=959.635834 shr=7.49882364 lines=561 END +moesel call=369 dev=1 T=1 E=512 k=10 sel=f8f91f0b2d22aeb1 minulps=0 mintok=0 x=1562.56308 logit=3393.4375 exp=889.603898 shr=4.85634382 lines=562 END +moesel call=370 dev=1 T=1 E=512 k=10 sel=882a94af44d71095 minulps=1 mintok=0 x=1454.78902 logit=3452.9375 exp=857.448726 shr=4.25973188 lines=563 END +moesel call=371 dev=1 T=1 E=512 k=10 sel=ff64b0bf2202b4fc minulps=0 mintok=0 x=1692.0218 logit=3291.65625 exp=1019.0725 shr=6.75417233 lines=564 END +moesel call=372 dev=1 T=1 E=512 k=10 sel=9b91a180a5678cdc minulps=1 mintok=0 x=2027.62852 logit=3403.25 exp=1435.13897 shr=14.9619538 lines=565 END +moesel call=373 dev=1 T=1 E=512 k=10 sel=1429782ecd13da9a minulps=1 mintok=0 x=2013.09555 logit=3381.17188 exp=1315.70277 shr=7.95567352 lines=566 END +moesel call=374 dev=1 T=1 E=512 k=10 sel=bf3fc9cb980fbaa3 minulps=2 mintok=0 x=1951.77497 logit=3209.48438 exp=1672.33791 shr=11.5469591 lines=567 END +moesel call=375 dev=1 T=1 E=512 k=10 sel=f33e480a7a6c8c87 minulps=0 mintok=0 x=2032.46027 logit=3127.0625 exp=1221.10053 shr=18.475357 lines=568 END +moesel call=376 dev=1 T=1 E=512 k=10 sel=21d1673aca0f1f26 minulps=3 mintok=0 x=2186.27908 logit=3095.125 exp=1615.30269 shr=15.3497127 lines=569 END +moesel call=377 dev=1 T=1 E=512 k=10 sel=222ff097af306e69 minulps=4 mintok=0 x=2135.07565 logit=3275.53125 exp=1590.75785 shr=8.49492601 lines=570 END +moesel call=378 dev=1 T=1 E=512 k=10 sel=1e01f620cc0dbe4e minulps=0 mintok=0 x=2322.02223 logit=3195.76562 exp=1922.07308 shr=18.8426979 lines=571 END +moesel call=379 dev=1 T=1 E=512 k=10 sel=52bd328d002baa24 minulps=1 mintok=0 x=1975.22079 logit=3130.84375 exp=1444.17408 shr=16.6541057 lines=572 END +moesel call=380 dev=1 T=1 E=512 k=10 sel=714d4354831b9d0c minulps=0 mintok=0 x=2107.97066 logit=3165.39062 exp=1600.94059 shr=17.4046731 lines=573 END +moesel call=381 dev=1 T=1 E=512 k=10 sel=d169c968f5828ca8 minulps=0 mintok=0 x=2147.22314 logit=3235.29688 exp=1470.35388 shr=19.0868192 lines=574 END +moesel call=382 dev=1 T=1 E=512 k=10 sel=2718c25d93354ec4 minulps=0 mintok=0 x=2065.08524 logit=3180.26562 exp=1359.24738 shr=28.9335856 lines=575 END +moesel call=383 dev=1 T=1 E=512 k=10 sel=764408d7e514a33e minulps=3 mintok=0 x=1877.81024 logit=3102.1875 exp=1514.72134 shr=103.995275 lines=576 END diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt new file mode 100644 index 000000000..f6fa37b49 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt @@ -0,0 +1,384 @@ +moesel call=0 dev=1 T=5 E=512 k=10 sel=5bbd84f0b21b362b minulps=0 mintok=4 x=3615.62777 logit=17083.2891 exp=2527.65707 shr=250.743384 lines=5 END +moesel call=1 dev=1 T=5 E=512 k=10 sel=993cb7ac2f9ebaf3 minulps=0 mintok=1 x=3778.69921 logit=15553.9531 exp=2031.5105 shr=34.0787401 lines=10 END +moesel call=2 dev=1 T=5 E=512 k=10 sel=6c216b6b2cfe27d8 minulps=0 mintok=2 x=3841.05556 logit=15788.7969 exp=1789.27763 shr=52.0076308 lines=15 END +moesel call=3 dev=1 T=5 E=512 k=10 sel=8f7b3342cf4416a7 minulps=0 mintok=4 x=4867.96077 logit=15090.2969 exp=2596.98063 shr=37.9476791 lines=20 END +moesel call=4 dev=1 T=5 E=512 k=10 sel=4537bad749058f33 minulps=1 mintok=0 x=5355.29174 logit=15451.1719 exp=3073.73295 shr=47.5011171 lines=25 END +moesel call=5 dev=1 T=5 E=512 k=10 sel=9158e7125f290134 minulps=0 mintok=4 x=4456.92829 logit=15641.9844 exp=2040.0845 shr=34.254536 lines=30 END +moesel call=6 dev=1 T=5 E=512 k=10 sel=82e18d30854928c2 minulps=0 mintok=0 x=5469.83739 logit=15914.7188 exp=3136.38253 shr=47.063142 lines=35 END +moesel call=7 dev=1 T=5 E=512 k=10 sel=d94a6478b8761de3 minulps=0 mintok=2 x=5334.0979 logit=15909.9688 exp=2703.35417 shr=44.1492911 lines=40 END +moesel call=8 dev=1 T=5 E=512 k=10 sel=372c48dfeade5c01 minulps=0 mintok=2 x=5380.46048 logit=15765.6094 exp=3055.85262 shr=28.1771415 lines=45 END +moesel call=9 dev=1 T=5 E=512 k=10 sel=496b7f503ecc0638 minulps=0 mintok=0 x=5517.20036 logit=15662.5 exp=2956.27156 shr=34.5520656 lines=50 END +moesel call=10 dev=1 T=5 E=512 k=10 sel=b14a7db9158b4e90 minulps=0 mintok=0 x=4967.56581 logit=15753.2188 exp=2433.28837 shr=27.2410728 lines=55 END +moesel call=11 dev=1 T=5 E=512 k=10 sel=f068ea276978664c minulps=0 mintok=0 x=4870.5168 logit=15590.9219 exp=2156.46224 shr=18.6210697 lines=60 END +moesel call=12 dev=1 T=5 E=512 k=10 sel=d7e678c7724801db minulps=0 mintok=0 x=4971.30472 logit=15636.0938 exp=2244.17018 shr=26.6883839 lines=65 END +moesel call=13 dev=1 T=5 E=512 k=10 sel=7fce6e907d8d40a9 minulps=0 mintok=0 x=4645.09897 logit=15943.6406 exp=2044.70843 shr=27.4566855 lines=70 END +moesel call=14 dev=1 T=5 E=512 k=10 sel=cec11c8b5014de56 minulps=1 mintok=4 x=5259.56363 logit=15817.8125 exp=2911.1454 shr=22.834866 lines=75 END +moesel call=15 dev=1 T=5 E=512 k=10 sel=88cb94c69381a618 minulps=0 mintok=0 x=6750.75098 logit=15817.3438 exp=3967.80807 shr=29.9183135 lines=80 END +moesel call=16 dev=1 T=5 E=512 k=10 sel=3e7f3d84739b8124 minulps=0 mintok=3 x=6160.80079 logit=15761.7188 exp=3181.57952 shr=28.3739225 lines=85 END +moesel call=17 dev=1 T=5 E=512 k=10 sel=cb0c0f7107d1b9f1 minulps=0 mintok=0 x=5372.15441 logit=15998.7812 exp=2587.95085 shr=23.7375047 lines=90 END +moesel call=18 dev=1 T=5 E=512 k=10 sel=f105feb5386cb177 minulps=0 mintok=3 x=5328.18034 logit=16318.9531 exp=2864.13171 shr=25.2442927 lines=95 END +moesel call=19 dev=1 T=5 E=512 k=10 sel=28ca1992b70a0310 minulps=1 mintok=0 x=5867.30701 logit=16202.9375 exp=2841.54083 shr=28.7731155 lines=100 END +moesel call=20 dev=1 T=5 E=512 k=10 sel=f735fa40c10add85 minulps=0 mintok=1 x=7151.09091 logit=16145.7344 exp=4341.68606 shr=43.5266942 lines=105 END +moesel call=21 dev=1 T=5 E=512 k=10 sel=e80a415a8a18321d minulps=0 mintok=0 x=6725.81356 logit=16177.8594 exp=3800.6881 shr=28.3593103 lines=110 END +moesel call=22 dev=1 T=5 E=512 k=10 sel=b3c62ea616442afa minulps=2 mintok=0 x=6497.52442 logit=16640.3281 exp=3627.13058 shr=35.1753135 lines=115 END +moesel call=23 dev=1 T=5 E=512 k=10 sel=74c07c747cc4d2cc minulps=0 mintok=0 x=7567.35111 logit=16092.75 exp=4452.38452 shr=64.4897626 lines=120 END +moesel call=24 dev=1 T=5 E=512 k=10 sel=a999bb8e78af4388 minulps=1 mintok=1 x=5616.96724 logit=16321.6562 exp=2815.24834 shr=27.9610005 lines=125 END +moesel call=25 dev=1 T=5 E=512 k=10 sel=8c1d1c68d8bc3aa6 minulps=0 mintok=0 x=5637.05077 logit=16019.5156 exp=2673.41674 shr=28.9463229 lines=130 END +moesel call=26 dev=1 T=5 E=512 k=10 sel=35fd18b3f1d9f02a minulps=0 mintok=1 x=5242.66694 logit=16425.1406 exp=2446.62897 shr=22.7984998 lines=135 END +moesel call=27 dev=1 T=5 E=512 k=10 sel=d5d4e03947aadf34 minulps=1 mintok=1 x=5245.88568 logit=15944.8594 exp=2370.57481 shr=19.0522638 lines=140 END +moesel call=28 dev=1 T=5 E=512 k=10 sel=827c2ba2af341d0e minulps=0 mintok=3 x=5277.3485 logit=16122.0156 exp=2359.31454 shr=23.4091644 lines=145 END +moesel call=29 dev=1 T=5 E=512 k=10 sel=3595d2909c9a67db minulps=1 mintok=2 x=5501.65057 logit=16555.7891 exp=2816.01307 shr=25.0176327 lines=150 END +moesel call=30 dev=1 T=5 E=512 k=10 sel=04758182a6a02d86 minulps=1 mintok=0 x=6430.20349 logit=16688.5312 exp=3843.51206 shr=32.5737284 lines=155 END +moesel call=31 dev=1 T=5 E=512 k=10 sel=90f3a853d44ec229 minulps=0 mintok=1 x=8011.68789 logit=16377.6406 exp=4867.48047 shr=30.3206993 lines=160 END +moesel call=32 dev=1 T=5 E=512 k=10 sel=bdedff50562962b8 minulps=0 mintok=1 x=7325.23582 logit=15981.2734 exp=4370.99278 shr=32.5776461 lines=165 END +moesel call=33 dev=1 T=5 E=512 k=10 sel=3c833769e5259f6d minulps=0 mintok=0 x=6701.2299 logit=16714.3281 exp=3827.1734 shr=25.0316927 lines=170 END +moesel call=34 dev=1 T=5 E=512 k=10 sel=3188ef46a7207960 minulps=1 mintok=0 x=6143.44704 logit=16665.9531 exp=3841.58992 shr=33.8632984 lines=175 END +moesel call=35 dev=1 T=5 E=512 k=10 sel=11d8430c7058002b minulps=2 mintok=1 x=7099.00982 logit=16456.9531 exp=4640.90574 shr=32.8907891 lines=180 END +moesel call=36 dev=1 T=5 E=512 k=10 sel=ce85453d6b312686 minulps=0 mintok=2 x=8781.90772 logit=16370.8594 exp=8225.27047 shr=88.246615 lines=185 END +moesel call=37 dev=1 T=5 E=512 k=10 sel=8bb5758a7850c86e minulps=0 mintok=1 x=8303.17253 logit=16354.3594 exp=7001.25341 shr=65.9959179 lines=190 END +moesel call=38 dev=1 T=5 E=512 k=10 sel=1088ca83dc40c8cf minulps=0 mintok=0 x=8461.53261 logit=16808.5312 exp=9278.23279 shr=109.359189 lines=195 END +moesel call=39 dev=1 T=5 E=512 k=10 sel=79bf51e408d85e86 minulps=0 mintok=0 x=9068.89016 logit=15714.6328 exp=8503.02438 shr=94.729246 lines=200 END +moesel call=40 dev=1 T=5 E=512 k=10 sel=5547dd9fd5bb6f4b minulps=0 mintok=0 x=9530.33126 logit=15709.2969 exp=7270.07352 shr=95.0038095 lines=205 END +moesel call=41 dev=1 T=5 E=512 k=10 sel=fc783871f0472791 minulps=1 mintok=1 x=9615.44181 logit=16108.7188 exp=6946.6853 shr=102.375323 lines=210 END +moesel call=42 dev=1 T=5 E=512 k=10 sel=d282c5f9fcddd1db minulps=0 mintok=0 x=9808.29724 logit=16385.2812 exp=9273.51059 shr=131.467709 lines=215 END +moesel call=43 dev=1 T=5 E=512 k=10 sel=21efbad6440ee049 minulps=0 mintok=3 x=8195.22645 logit=15776.9609 exp=5803.28836 shr=89.4814494 lines=220 END +moesel call=44 dev=1 T=5 E=512 k=10 sel=09168f1f28093967 minulps=0 mintok=1 x=9189.70934 logit=15894.25 exp=7386.05759 shr=105.226898 lines=225 END +moesel call=45 dev=1 T=5 E=512 k=10 sel=9fa59df33f617085 minulps=0 mintok=0 x=10345.0287 logit=16224.1719 exp=7223.67518 shr=145.161863 lines=230 END +moesel call=46 dev=1 T=5 E=512 k=10 sel=2ebdca3d3e7ecf8e minulps=1 mintok=2 x=10114.0947 logit=16271.3906 exp=8474.7606 shr=155.178569 lines=235 END +moesel call=47 dev=1 T=5 E=512 k=10 sel=b6add1408e56cc74 minulps=1 mintok=3 x=9601.74072 logit=16386.2812 exp=9116.37751 shr=815.846916 lines=240 END +moesel call=48 dev=1 T=1 E=512 k=10 sel=8fce961a87c895de minulps=7 mintok=0 x=723.664909 logit=3430.46875 exp=563.497883 shr=31.5413151 lines=241 END +moesel call=49 dev=1 T=1 E=512 k=10 sel=da67409db4906d8e minulps=2 mintok=0 x=815.778957 logit=3050.65625 exp=411.528591 shr=6.04054234 lines=242 END +moesel call=50 dev=1 T=1 E=512 k=10 sel=f1effeeba82642ba minulps=1 mintok=0 x=850.152067 logit=3036.60938 exp=419.765577 shr=9.20946218 lines=243 END +moesel call=51 dev=1 T=1 E=512 k=10 sel=da92804a7ed93391 minulps=0 mintok=0 x=1061.76275 logit=2973.15625 exp=522.752055 shr=5.76472539 lines=244 END +moesel call=52 dev=1 T=1 E=512 k=10 sel=79e503d24cec8ea1 minulps=4 mintok=0 x=1097.65658 logit=2991.65625 exp=552.884984 shr=5.48180634 lines=245 END +moesel call=53 dev=1 T=1 E=512 k=10 sel=8423f417266f13a9 minulps=0 mintok=0 x=959.375325 logit=3035.70312 exp=439.299158 shr=5.19935743 lines=246 END +moesel call=54 dev=1 T=1 E=512 k=10 sel=ac94027b28df13c6 minulps=0 mintok=0 x=1087.99149 logit=2885.35938 exp=719.38391 shr=6.26721723 lines=247 END +moesel call=55 dev=1 T=1 E=512 k=10 sel=a12e2a23cfcd41a7 minulps=0 mintok=0 x=1105.11678 logit=2956.125 exp=564.357492 shr=5.27389602 lines=248 END +moesel call=56 dev=1 T=1 E=512 k=10 sel=23e3af4450f40f94 minulps=10 mintok=0 x=1132.14298 logit=2970.45312 exp=636.273753 shr=4.96539277 lines=249 END +moesel call=57 dev=1 T=1 E=512 k=10 sel=4f5439affa3f7823 minulps=1 mintok=0 x=1169.21249 logit=3023.01562 exp=604.602861 shr=7.07437545 lines=250 END +moesel call=58 dev=1 T=1 E=512 k=10 sel=fa8a6c4748735536 minulps=1 mintok=0 x=1065.15538 logit=3002.625 exp=508.546191 shr=4.56790698 lines=251 END +moesel call=59 dev=1 T=1 E=512 k=10 sel=610e505750ac340d minulps=1 mintok=0 x=1074.04506 logit=3049.89062 exp=477.510162 shr=3.14345438 lines=252 END +moesel call=60 dev=1 T=1 E=512 k=10 sel=6e98fc1568dabe37 minulps=1 mintok=0 x=1099.14507 logit=3040.84375 exp=506.666758 shr=5.59145327 lines=253 END +moesel call=61 dev=1 T=1 E=512 k=10 sel=e556fe4cf283032d minulps=1 mintok=0 x=1004.17902 logit=3055.09375 exp=429.474333 shr=5.29509216 lines=254 END +moesel call=62 dev=1 T=1 E=512 k=10 sel=9f55aa23ddc601de minulps=1 mintok=0 x=1135.75445 logit=3041.59375 exp=528.632194 shr=5.57348994 lines=255 END +moesel call=63 dev=1 T=1 E=512 k=10 sel=b5fa1a7c51a11adb minulps=3 mintok=0 x=1481.94626 logit=3048.6875 exp=833.688153 shr=7.24646278 lines=256 END +moesel call=64 dev=1 T=1 E=512 k=10 sel=3f5ae586af492907 minulps=3 mintok=0 x=1308.29881 logit=3068.84375 exp=636.236269 shr=5.68035066 lines=257 END +moesel call=65 dev=1 T=1 E=512 k=10 sel=f0b2fb36fcbcebfc minulps=1 mintok=0 x=1135.90056 logit=3116.04688 exp=591.092324 shr=4.57442022 lines=258 END +moesel call=66 dev=1 T=1 E=512 k=10 sel=02878f048ba3b3cc minulps=0 mintok=0 x=1110.02275 logit=3212.3125 exp=565.944166 shr=4.58377909 lines=259 END +moesel call=67 dev=1 T=1 E=512 k=10 sel=44c730c5e4ad62b4 minulps=0 mintok=0 x=1277.21745 logit=3174.32812 exp=662.280042 shr=5.30693895 lines=260 END +moesel call=68 dev=1 T=1 E=512 k=10 sel=c8c8bfef3ac9cf11 minulps=3 mintok=0 x=1526.47765 logit=3156.125 exp=919.959777 shr=9.40919015 lines=261 END +moesel call=69 dev=1 T=1 E=512 k=10 sel=f0f1f9191ede49b5 minulps=1 mintok=0 x=1488.26378 logit=3190.21875 exp=857.027212 shr=8.0348592 lines=262 END +moesel call=70 dev=1 T=1 E=512 k=10 sel=ba1241b9927c42f3 minulps=8 mintok=0 x=1389.98263 logit=3250.26562 exp=826.856896 shr=4.24134622 lines=263 END +moesel call=71 dev=1 T=1 E=512 k=10 sel=a18a19bedd1d013f minulps=2 mintok=0 x=1725.68997 logit=3138.65625 exp=1067.90509 shr=14.7051098 lines=264 END +moesel call=72 dev=1 T=1 E=512 k=10 sel=f792ae08b7ac3a7b minulps=1 mintok=0 x=1169.60753 logit=3133.48438 exp=591.516186 shr=6.21655642 lines=265 END +moesel call=73 dev=1 T=1 E=512 k=10 sel=18c70210b6f7eecc minulps=5 mintok=0 x=1239.81597 logit=3114.59375 exp=591.640211 shr=5.13053793 lines=266 END +moesel call=74 dev=1 T=1 E=512 k=10 sel=ab65a2d9f6c06e5f minulps=0 mintok=0 x=1145.68917 logit=3141.35938 exp=564.744467 shr=4.88275191 lines=267 END +moesel call=75 dev=1 T=1 E=512 k=10 sel=b943fedc5d203b58 minulps=6 mintok=0 x=1200.04994 logit=3088.3125 exp=639.699755 shr=5.89255972 lines=268 END +moesel call=76 dev=1 T=1 E=512 k=10 sel=392b4ca5bc1fa11b minulps=0 mintok=0 x=1232.23133 logit=3132.15625 exp=643.189591 shr=5.68870719 lines=269 END +moesel call=77 dev=1 T=1 E=512 k=10 sel=c598bc5cb5506b7c minulps=3 mintok=0 x=1275.81649 logit=3159.20312 exp=740.54248 shr=7.27388824 lines=270 END +moesel call=78 dev=1 T=1 E=512 k=10 sel=6c8796420f9bb75c minulps=0 mintok=0 x=1470.86962 logit=3133.32031 exp=867.480005 shr=7.81114797 lines=271 END +moesel call=79 dev=1 T=1 E=512 k=10 sel=eb29872faed407da minulps=3 mintok=0 x=1827.32727 logit=3074.15625 exp=1186.14958 shr=10.3065294 lines=272 END +moesel call=80 dev=1 T=1 E=512 k=10 sel=21524f9969403d65 minulps=4 mintok=0 x=1763.7714 logit=3055.25 exp=1054.69468 shr=9.87494844 lines=273 END +moesel call=81 dev=1 T=1 E=512 k=10 sel=4a574cc2fc7fb9fb minulps=5 mintok=0 x=1602.81626 logit=3315.92188 exp=1103.81968 shr=6.46347584 lines=274 END +moesel call=82 dev=1 T=1 E=512 k=10 sel=b3eaa9c6c6eb3130 minulps=0 mintok=0 x=1433.33069 logit=3340.53125 exp=980.073327 shr=6.11113988 lines=275 END +moesel call=83 dev=1 T=1 E=512 k=10 sel=2e6166f480eec102 minulps=2 mintok=0 x=1680.14681 logit=3316.28125 exp=1233.40858 shr=7.08982886 lines=276 END +moesel call=84 dev=1 T=1 E=512 k=10 sel=e74550e15d7c5824 minulps=2 mintok=0 x=1999.0076 logit=3225.98438 exp=1862.59425 shr=19.151212 lines=277 END +moesel call=85 dev=1 T=1 E=512 k=10 sel=b589ab458c3fcf8f minulps=6 mintok=0 x=1922.31729 logit=3259.39062 exp=1476.00827 shr=13.1864017 lines=278 END +moesel call=86 dev=1 T=1 E=512 k=10 sel=f1fdb7b403ff173f minulps=2 mintok=0 x=1897.10037 logit=3267.20312 exp=2262.07617 shr=18.4758842 lines=279 END +moesel call=87 dev=1 T=1 E=512 k=10 sel=e281339c7be2009f minulps=2 mintok=0 x=2020.75961 logit=3055.8125 exp=1835.15941 shr=28.9277753 lines=280 END +moesel call=88 dev=1 T=1 E=512 k=10 sel=5d15905bdd77a9b8 minulps=3 mintok=0 x=2077.11619 logit=3079.07812 exp=1727.19587 shr=23.2208766 lines=281 END +moesel call=89 dev=1 T=1 E=512 k=10 sel=b5354b95743d475f minulps=0 mintok=0 x=2059.48909 logit=3224.5 exp=1896.18855 shr=12.8156054 lines=282 END +moesel call=90 dev=1 T=1 E=512 k=10 sel=e18d448a6e9d074f minulps=0 mintok=0 x=2088.69188 logit=3183.14062 exp=2517.64052 shr=23.6908675 lines=283 END +moesel call=91 dev=1 T=1 E=512 k=10 sel=196c9873cebb2a27 minulps=6 mintok=0 x=1798.19436 logit=3094.70312 exp=1263.77772 shr=21.8022825 lines=284 END +moesel call=92 dev=1 T=1 E=512 k=10 sel=ae96ba2bf0c60fc0 minulps=1 mintok=0 x=2015.2857 logit=3143.8125 exp=1677.70815 shr=18.9001883 lines=285 END +moesel call=93 dev=1 T=1 E=512 k=10 sel=0770365869cd05cc minulps=6 mintok=0 x=2180.65905 logit=3273.34375 exp=1535.39289 shr=23.4513273 lines=286 END +moesel call=94 dev=1 T=1 E=512 k=10 sel=195829ec6b1c4fae minulps=1 mintok=0 x=2207.75889 logit=3264.14062 exp=1543.09526 shr=31.3433605 lines=287 END +moesel call=95 dev=1 T=1 E=512 k=10 sel=2e60b1df07b0c22c minulps=0 mintok=0 x=1998.70799 logit=3184.03125 exp=1309.73513 shr=134.910462 lines=288 END +moesel call=96 dev=1 T=1 E=512 k=10 sel=0148dfd4911f327c minulps=0 mintok=0 x=711.229391 logit=3391.71875 exp=608.896065 shr=38.3388032 lines=289 END +moesel call=97 dev=1 T=1 E=512 k=10 sel=2a8a39b413aa2418 minulps=2 mintok=0 x=813.289912 logit=3060.14062 exp=458.857198 shr=5.10810651 lines=290 END +moesel call=98 dev=1 T=1 E=512 k=10 sel=7091a270949c8cf6 minulps=0 mintok=0 x=793.840794 logit=3058.3125 exp=305.347046 shr=7.26283523 lines=291 END +moesel call=99 dev=1 T=1 E=512 k=10 sel=d3488e89a2656e11 minulps=0 mintok=0 x=1148.27415 logit=3025.26562 exp=563.083738 shr=3.97451648 lines=292 END +moesel call=100 dev=1 T=1 E=512 k=10 sel=ca64ce98d54c003b minulps=2 mintok=0 x=1142.11562 logit=2944.34375 exp=527.39017 shr=4.94533142 lines=293 END +moesel call=101 dev=1 T=1 E=512 k=10 sel=82c9513497c8dab5 minulps=3 mintok=0 x=1046.18565 logit=3038.15625 exp=467.063415 shr=7.69106738 lines=294 END +moesel call=102 dev=1 T=1 E=512 k=10 sel=f49dbc59c7d107e1 minulps=1 mintok=0 x=1204.76578 logit=3084 exp=647.277392 shr=6.21173823 lines=295 END +moesel call=103 dev=1 T=1 E=512 k=10 sel=cb566956732e4c89 minulps=3 mintok=0 x=1098.74821 logit=3016 exp=524.098303 shr=5.28041228 lines=296 END +moesel call=104 dev=1 T=1 E=512 k=10 sel=b4292d57aaf8616d minulps=1 mintok=0 x=1126.13285 logit=3031.78125 exp=580.574672 shr=6.37354497 lines=297 END +moesel call=105 dev=1 T=1 E=512 k=10 sel=efc8328b96fbbf59 minulps=3 mintok=0 x=1151.31993 logit=3056.15625 exp=546.376722 shr=5.93742776 lines=298 END +moesel call=106 dev=1 T=1 E=512 k=10 sel=ecc57d51c9e6ad13 minulps=3 mintok=0 x=1062.67196 logit=3091.28125 exp=490.783311 shr=3.7371625 lines=299 END +moesel call=107 dev=1 T=1 E=512 k=10 sel=684bde3e5f1bf1aa minulps=3 mintok=0 x=1047.96809 logit=3081.14062 exp=455.223676 shr=3.03726482 lines=300 END +moesel call=108 dev=1 T=1 E=512 k=10 sel=bf127b4544c277d0 minulps=1 mintok=0 x=1065.00896 logit=3091.96875 exp=480.718076 shr=5.86843657 lines=301 END +moesel call=109 dev=1 T=1 E=512 k=10 sel=e598f26e2c516d21 minulps=1 mintok=0 x=952.62907 logit=3055.71875 exp=409.174747 shr=5.98165156 lines=302 END +moesel call=110 dev=1 T=1 E=512 k=10 sel=fee8e61f5b205d8d minulps=1 mintok=0 x=1190.50853 logit=3174.10938 exp=639.398977 shr=5.0496688 lines=303 END +moesel call=111 dev=1 T=1 E=512 k=10 sel=38ddb3334bf52011 minulps=7 mintok=0 x=1581.64626 logit=3088.82812 exp=915.981432 shr=7.10401408 lines=304 END +moesel call=112 dev=1 T=1 E=512 k=10 sel=197a929c90933e08 minulps=4 mintok=0 x=1315.26594 logit=3188.75 exp=665.381997 shr=4.12329758 lines=305 END +moesel call=113 dev=1 T=1 E=512 k=10 sel=5bf394db409c91de minulps=0 mintok=0 x=1086.86942 logit=3070.97656 exp=464.323578 shr=4.31994963 lines=306 END +moesel call=114 dev=1 T=1 E=512 k=10 sel=db2f2b90ff1ca8c4 minulps=6 mintok=0 x=1057.06034 logit=3195.5 exp=501.827474 shr=3.50922626 lines=307 END +moesel call=115 dev=1 T=1 E=512 k=10 sel=bf85a0dd154e12b1 minulps=1 mintok=0 x=1216.69313 logit=3169 exp=602.579264 shr=5.34388182 lines=308 END +moesel call=116 dev=1 T=1 E=512 k=10 sel=4df92e9c25fb4d51 minulps=5 mintok=0 x=1465.47069 logit=3190.48438 exp=796.757087 shr=8.43680063 lines=309 END +moesel call=117 dev=1 T=1 E=512 k=10 sel=cf786bdd0222e90f minulps=3 mintok=0 x=1447.98005 logit=3265.48438 exp=816.96201 shr=5.35224335 lines=310 END +moesel call=118 dev=1 T=1 E=512 k=10 sel=38c8a9ac128de4be minulps=1 mintok=0 x=1388.9577 logit=3321 exp=767.592594 shr=3.67117262 lines=311 END +moesel call=119 dev=1 T=1 E=512 k=10 sel=f85c4bb9161d1c08 minulps=1 mintok=0 x=1667.9041 logit=3130.51562 exp=928.467204 shr=9.59148346 lines=312 END +moesel call=120 dev=1 T=1 E=512 k=10 sel=4c1604415ca197b4 minulps=1 mintok=0 x=1192.32415 logit=3203.71875 exp=573.681941 shr=5.08476549 lines=313 END +moesel call=121 dev=1 T=1 E=512 k=10 sel=bd9f8afedc5e920f minulps=2 mintok=0 x=1218.67761 logit=3186.4375 exp=614.268691 shr=4.43214091 lines=314 END +moesel call=122 dev=1 T=1 E=512 k=10 sel=5ca6d5be21cfbf38 minulps=1 mintok=0 x=1105.24921 logit=3256.57812 exp=569.026838 shr=4.74037996 lines=315 END +moesel call=123 dev=1 T=1 E=512 k=10 sel=1b10475baf5406c8 minulps=24 mintok=0 x=1185.73088 logit=3211.54688 exp=593.23451 shr=5.2065296 lines=316 END +moesel call=124 dev=1 T=1 E=512 k=10 sel=d3fc82f065d25b5d minulps=3 mintok=0 x=1193.8325 logit=3253.59375 exp=564.272067 shr=5.23822185 lines=317 END +moesel call=125 dev=1 T=1 E=512 k=10 sel=1be218557f73f10e minulps=0 mintok=0 x=1199.97556 logit=3197.375 exp=687.191725 shr=6.24312269 lines=318 END +moesel call=126 dev=1 T=1 E=512 k=10 sel=f42323df10eb3bda minulps=1 mintok=0 x=1467.88619 logit=3279.4375 exp=972.377591 shr=8.87083261 lines=319 END +moesel call=127 dev=1 T=1 E=512 k=10 sel=d4ca388514099f7b minulps=0 mintok=0 x=1939.54626 logit=3197.4375 exp=1292.16474 shr=10.8208374 lines=320 END +moesel call=128 dev=1 T=1 E=512 k=10 sel=44d3f5dabd163fe4 minulps=0 mintok=0 x=1775.87708 logit=3192.46875 exp=1133.96296 shr=9.4649017 lines=321 END +moesel call=129 dev=1 T=1 E=512 k=10 sel=45d1e46900535c92 minulps=4 mintok=0 x=1562.9893 logit=3186.5625 exp=1062.0221 shr=8.61333513 lines=322 END +moesel call=130 dev=1 T=1 E=512 k=10 sel=7ecefc81d491db2f minulps=1 mintok=0 x=1355.86098 logit=3281.95312 exp=829.080054 shr=5.78759143 lines=323 END +moesel call=131 dev=1 T=1 E=512 k=10 sel=a1b8c880f2e38e41 minulps=4 mintok=0 x=1603.41966 logit=3165.34375 exp=989.260696 shr=7.96881433 lines=324 END +moesel call=132 dev=1 T=1 E=512 k=10 sel=ef214577e0fb46df minulps=2 mintok=0 x=1876.34944 logit=3195.42188 exp=1582.1276 shr=23.6369063 lines=325 END +moesel call=133 dev=1 T=1 E=512 k=10 sel=546bd831da771053 minulps=3 mintok=0 x=1755.0382 logit=3223.59375 exp=1334.19155 shr=11.9084479 lines=326 END +moesel call=134 dev=1 T=1 E=512 k=10 sel=4ba4625a67b48d5a minulps=3 mintok=0 x=1782.05633 logit=3165 exp=1762.52148 shr=15.7946019 lines=327 END +moesel call=135 dev=1 T=1 E=512 k=10 sel=33563f0a816e3ea5 minulps=3 mintok=0 x=2061.72572 logit=3058.8125 exp=1854.22584 shr=22.7734525 lines=328 END +moesel call=136 dev=1 T=1 E=512 k=10 sel=215e261e955d132a minulps=2 mintok=0 x=2105.59185 logit=3110.26562 exp=1579.11116 shr=17.9604087 lines=329 END +moesel call=137 dev=1 T=1 E=512 k=10 sel=c8237987eca74795 minulps=0 mintok=0 x=2075.90125 logit=3203.89062 exp=1873.71327 shr=11.5481218 lines=330 END +moesel call=138 dev=1 T=1 E=512 k=10 sel=be9e1f90de0ea44d minulps=0 mintok=0 x=2101.67663 logit=3193.0625 exp=1893.70736 shr=22.0316356 lines=331 END +moesel call=139 dev=1 T=1 E=512 k=10 sel=36ba76b551a2124f minulps=3 mintok=0 x=1861.83108 logit=3099.35938 exp=1341.23117 shr=21.6560817 lines=332 END +moesel call=140 dev=1 T=1 E=512 k=10 sel=0adbb6f1f0e23878 minulps=1 mintok=0 x=2019.18243 logit=3169.25 exp=1736.76268 shr=23.2480535 lines=333 END +moesel call=141 dev=1 T=1 E=512 k=10 sel=834801528ea79c8e minulps=1 mintok=0 x=2156.12024 logit=3188.3125 exp=1707.46686 shr=30.6757116 lines=334 END +moesel call=142 dev=1 T=1 E=512 k=10 sel=ac6441082a2c0801 minulps=0 mintok=0 x=2061.68134 logit=3139.15625 exp=1626.7381 shr=34.1050006 lines=335 END +moesel call=143 dev=1 T=1 E=512 k=10 sel=7576e41aaf7ace32 minulps=3 mintok=0 x=1918.08546 logit=3193.03125 exp=1586.48832 shr=159.199129 lines=336 END +moesel call=144 dev=1 T=1 E=512 k=10 sel=09c0aa1fa6c8c877 minulps=0 mintok=0 x=709.265829 logit=3419.98438 exp=588.563679 shr=44.0349371 lines=337 END +moesel call=145 dev=1 T=1 E=512 k=10 sel=a916136dafb38ba5 minulps=2 mintok=0 x=806.718358 logit=3072.34375 exp=521.929052 shr=5.52216164 lines=338 END +moesel call=146 dev=1 T=1 E=512 k=10 sel=c3ae50cfa21dff8e minulps=1 mintok=0 x=834.998063 logit=3038.6875 exp=383.940281 shr=8.0326896 lines=339 END +moesel call=147 dev=1 T=1 E=512 k=10 sel=a8708ad99713b1fa minulps=1 mintok=0 x=1166.77777 logit=3165.46875 exp=531.862158 shr=3.76528293 lines=340 END +moesel call=148 dev=1 T=1 E=512 k=10 sel=a07ce5b4d7e4c0f4 minulps=1 mintok=0 x=1143.24499 logit=3043.60938 exp=600.16254 shr=6.28855085 lines=341 END +moesel call=149 dev=1 T=1 E=512 k=10 sel=41ce2a00299c7b4e minulps=0 mintok=0 x=1077.6009 logit=3098.03125 exp=449.393738 shr=5.79224983 lines=342 END +moesel call=150 dev=1 T=1 E=512 k=10 sel=40068dc1a2211280 minulps=1 mintok=0 x=1197.02317 logit=3176.82812 exp=636.113586 shr=7.11001849 lines=343 END +moesel call=151 dev=1 T=1 E=512 k=10 sel=91b6b39d22f98b4f minulps=2 mintok=0 x=1124.00188 logit=3106.10938 exp=573.867174 shr=4.49665631 lines=344 END +moesel call=152 dev=1 T=1 E=512 k=10 sel=6249f6dc603c35af minulps=1 mintok=0 x=1121.45561 logit=3045.1875 exp=552.051332 shr=4.80011374 lines=345 END +moesel call=153 dev=1 T=1 E=512 k=10 sel=2718d09e060a4646 minulps=7 mintok=0 x=1165.85487 logit=3095.45312 exp=521.999165 shr=5.80828868 lines=346 END +moesel call=154 dev=1 T=1 E=512 k=10 sel=fc3615395040373a minulps=0 mintok=0 x=1045.45938 logit=3144.29688 exp=502.656306 shr=3.53277464 lines=347 END +moesel call=155 dev=1 T=1 E=512 k=10 sel=3b73bf134ebe2733 minulps=5 mintok=0 x=1054.21848 logit=3071.26562 exp=447.021474 shr=3.02292909 lines=348 END +moesel call=156 dev=1 T=1 E=512 k=10 sel=677f10c147284eaa minulps=1 mintok=0 x=1097.53947 logit=3078.78125 exp=529.79108 shr=6.68490016 lines=349 END +moesel call=157 dev=1 T=1 E=512 k=10 sel=8e80159dfedac81d minulps=1 mintok=0 x=980.618567 logit=3005.90625 exp=421.948717 shr=5.0479212 lines=350 END +moesel call=158 dev=1 T=1 E=512 k=10 sel=3901af792cd4064e minulps=0 mintok=0 x=1203.96706 logit=3163.9375 exp=603.212165 shr=4.67670434 lines=351 END +moesel call=159 dev=1 T=1 E=512 k=10 sel=ecea62446c550089 minulps=3 mintok=0 x=1616.5708 logit=3171.79688 exp=950.795252 shr=5.66884657 lines=352 END +moesel call=160 dev=1 T=1 E=512 k=10 sel=6dea3b8b688439cf minulps=3 mintok=0 x=1383.46333 logit=3119.78125 exp=659.9029 shr=4.44192258 lines=353 END +moesel call=161 dev=1 T=1 E=512 k=10 sel=1ab7e1e78d5fd1d8 minulps=13 mintok=0 x=1165.40147 logit=3048.26562 exp=542.088381 shr=4.76746796 lines=354 END +moesel call=162 dev=1 T=1 E=512 k=10 sel=2dcd573eb4fc01b4 minulps=1 mintok=0 x=1037.56161 logit=3133.90625 exp=471.669929 shr=3.77397194 lines=355 END +moesel call=163 dev=1 T=1 E=512 k=10 sel=dc42ee3aef8a215f minulps=1 mintok=0 x=1257.9455 logit=3177.53125 exp=625.847851 shr=5.1642314 lines=356 END +moesel call=164 dev=1 T=1 E=512 k=10 sel=c30e84a97933daad minulps=1 mintok=0 x=1477.93942 logit=3180.95312 exp=819.896465 shr=9.15391125 lines=357 END +moesel call=165 dev=1 T=1 E=512 k=10 sel=938a2967cafb3bb9 minulps=1 mintok=0 x=1389.86459 logit=3218.40625 exp=728.211013 shr=5.84739663 lines=358 END +moesel call=166 dev=1 T=1 E=512 k=10 sel=024eef52bb54717a minulps=6 mintok=0 x=1302.85019 logit=3321.03125 exp=674.083279 shr=4.14695835 lines=359 END +moesel call=167 dev=1 T=1 E=512 k=10 sel=eb52d195dc1e4d9d minulps=0 mintok=0 x=1583.46851 logit=3157.1875 exp=810.325841 shr=9.69447611 lines=360 END +moesel call=168 dev=1 T=1 E=512 k=10 sel=3b2fbe3418eff725 minulps=2 mintok=0 x=1167.98357 logit=3179.67188 exp=586.495793 shr=5.39254894 lines=361 END +moesel call=169 dev=1 T=1 E=512 k=10 sel=7ce39d832fb4a4c3 minulps=2 mintok=0 x=1282.43682 logit=3241.10938 exp=670.836109 shr=5.9029246 lines=362 END +moesel call=170 dev=1 T=1 E=512 k=10 sel=f8c3febace9a2e04 minulps=2 mintok=0 x=1120.09287 logit=3198.40625 exp=554.365411 shr=4.36842306 lines=363 END +moesel call=171 dev=1 T=1 E=512 k=10 sel=922aedd0aa6e7c97 minulps=3 mintok=0 x=1166.64225 logit=3210.28125 exp=550.763535 shr=4.82317674 lines=364 END +moesel call=172 dev=1 T=1 E=512 k=10 sel=88201025b87ff1aa minulps=1 mintok=0 x=1210.73197 logit=3291.85938 exp=608.724044 shr=6.43653818 lines=365 END +moesel call=173 dev=1 T=1 E=512 k=10 sel=242f52e1d2aea9b4 minulps=1 mintok=0 x=1174.16624 logit=3169.625 exp=684.683064 shr=7.11342804 lines=366 END +moesel call=174 dev=1 T=1 E=512 k=10 sel=97b8993e354862fe minulps=2 mintok=0 x=1443.55906 logit=3327.95312 exp=906.495923 shr=7.82015212 lines=367 END +moesel call=175 dev=1 T=1 E=512 k=10 sel=ecceaaf787f4e8a2 minulps=1 mintok=0 x=1907.92532 logit=3234.75 exp=1303.53765 shr=11.7175276 lines=368 END +moesel call=176 dev=1 T=1 E=512 k=10 sel=623fa0d91696b26a minulps=0 mintok=0 x=1772.42972 logit=3182.95312 exp=1102.16113 shr=12.0974883 lines=369 END +moesel call=177 dev=1 T=1 E=512 k=10 sel=a805094da310ac0d minulps=1 mintok=0 x=1562.21337 logit=3161.53125 exp=1013.94754 shr=8.14480484 lines=370 END +moesel call=178 dev=1 T=1 E=512 k=10 sel=ea7214f25d79013e minulps=6 mintok=0 x=1408.19263 logit=3322.875 exp=888.958483 shr=6.42585049 lines=371 END +moesel call=179 dev=1 T=1 E=512 k=10 sel=ca8d28b5f880044a minulps=3 mintok=0 x=1603.33182 logit=3187.57812 exp=981.019062 shr=9.40827595 lines=372 END +moesel call=180 dev=1 T=1 E=512 k=10 sel=5fa37274e14a4555 minulps=0 mintok=0 x=1830.96466 logit=3197.96875 exp=1497.51121 shr=24.2480717 lines=373 END +moesel call=181 dev=1 T=1 E=512 k=10 sel=0af4c7f5941e2562 minulps=4 mintok=0 x=1783.48712 logit=3279.78125 exp=1305.22535 shr=12.6527024 lines=374 END +moesel call=182 dev=1 T=1 E=512 k=10 sel=b2a982321d88e522 minulps=0 mintok=0 x=1841.23434 logit=3210.3125 exp=1817.83536 shr=16.2897464 lines=375 END +moesel call=183 dev=1 T=1 E=512 k=10 sel=985cef7fde795954 minulps=3 mintok=0 x=1939.8259 logit=3074.29688 exp=1636.95547 shr=29.4986321 lines=376 END +moesel call=184 dev=1 T=1 E=512 k=10 sel=e0639f29d57181b6 minulps=2 mintok=0 x=1947.12249 logit=3111.59375 exp=1375.78275 shr=18.5198702 lines=377 END +moesel call=185 dev=1 T=1 E=512 k=10 sel=083ec11d6668dd3b minulps=6 mintok=0 x=2043.42286 logit=3206.75 exp=1591.7568 shr=12.1486009 lines=378 END +moesel call=186 dev=1 T=1 E=512 k=10 sel=a5b8b376d111d99d minulps=1 mintok=0 x=2183.88591 logit=3210.40625 exp=1958.24461 shr=21.903844 lines=379 END +moesel call=187 dev=1 T=1 E=512 k=10 sel=6118f474e0432f43 minulps=5 mintok=0 x=1861.45516 logit=3113.1875 exp=1351.57273 shr=23.5180559 lines=380 END +moesel call=188 dev=1 T=1 E=512 k=10 sel=cbe071c09189f847 minulps=1 mintok=0 x=2028.40356 logit=3157.65625 exp=1554.31609 shr=25.4878428 lines=381 END +moesel call=189 dev=1 T=1 E=512 k=10 sel=132689ebfca1f5fe minulps=0 mintok=0 x=2282.88352 logit=3248.3125 exp=1786.49834 shr=23.95741 lines=382 END +moesel call=190 dev=1 T=1 E=512 k=10 sel=861266fd7f73af51 minulps=2 mintok=0 x=2142.89643 logit=3205.82812 exp=1843.56673 shr=45.3861183 lines=383 END +moesel call=191 dev=1 T=1 E=512 k=10 sel=4199bb65a00cd9d9 minulps=1 mintok=0 x=1941.61238 logit=3194.5 exp=1618.01128 shr=154.832057 lines=384 END +moesel call=192 dev=1 T=1 E=512 k=10 sel=779637335a2e27ec minulps=0 mintok=0 x=715.972754 logit=3468.28125 exp=487.24113 shr=51.3272271 lines=385 END +moesel call=193 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=765.311812 logit=3039.48438 exp=448.088648 shr=5.45426782 lines=386 END +moesel call=194 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=1 mintok=0 x=790.771747 logit=3074.59375 exp=368.835797 shr=6.01973237 lines=387 END +moesel call=195 dev=1 T=1 E=512 k=10 sel=55b427fc8ccf7832 minulps=0 mintok=0 x=1146.39526 logit=3150.625 exp=574.587783 shr=3.67641241 lines=388 END +moesel call=196 dev=1 T=1 E=512 k=10 sel=e75f20bfc53651de minulps=1 mintok=0 x=1086.01609 logit=2952.48438 exp=527.473318 shr=6.07862944 lines=389 END +moesel call=197 dev=1 T=1 E=512 k=10 sel=e199dea9a8bf81ec minulps=3 mintok=0 x=1096.07753 logit=3080 exp=518.36322 shr=6.0544898 lines=390 END +moesel call=198 dev=1 T=1 E=512 k=10 sel=d09df72dfb6f0590 minulps=0 mintok=0 x=1232.41076 logit=3124.51562 exp=686.97776 shr=6.92919293 lines=391 END +moesel call=199 dev=1 T=1 E=512 k=10 sel=cc090d387c768f94 minulps=1 mintok=0 x=1112.01891 logit=3108.51562 exp=549.316119 shr=6.26023278 lines=392 END +moesel call=200 dev=1 T=1 E=512 k=10 sel=aff67bb07407ae85 minulps=1 mintok=0 x=1098.923 logit=3076.9375 exp=563.466729 shr=5.53606613 lines=393 END +moesel call=201 dev=1 T=1 E=512 k=10 sel=325e66aef4c1aa8a minulps=3 mintok=0 x=1152.64936 logit=3080.92188 exp=520.811862 shr=6.24086326 lines=394 END +moesel call=202 dev=1 T=1 E=512 k=10 sel=c1e76752a2f342fb minulps=1 mintok=0 x=1049.53183 logit=3127.5 exp=494.600163 shr=3.69966551 lines=395 END +moesel call=203 dev=1 T=1 E=512 k=10 sel=a446f5b4aa564dce minulps=4 mintok=0 x=1059.41607 logit=3035.5 exp=429.340645 shr=3.23255975 lines=396 END +moesel call=204 dev=1 T=1 E=512 k=10 sel=6084dcd936031c37 minulps=3 mintok=0 x=1100.06998 logit=3073.10938 exp=535.282465 shr=7.49345727 lines=397 END +moesel call=205 dev=1 T=1 E=512 k=10 sel=9afbc29fbb70741f minulps=3 mintok=0 x=971.840316 logit=3056.85938 exp=427.407292 shr=5.83559805 lines=398 END +moesel call=206 dev=1 T=1 E=512 k=10 sel=e54efc1726ad547b minulps=4 mintok=0 x=1171.57439 logit=3153.32812 exp=611.380009 shr=5.64469967 lines=399 END +moesel call=207 dev=1 T=1 E=512 k=10 sel=7d0859b4f61a0cbc minulps=1 mintok=0 x=1560.24919 logit=3178.28125 exp=960.064218 shr=5.07720075 lines=400 END +moesel call=208 dev=1 T=1 E=512 k=10 sel=3ba030b2004253b6 minulps=0 mintok=0 x=1368.99124 logit=3038.89062 exp=700.1013 shr=4.16129598 lines=401 END +moesel call=209 dev=1 T=1 E=512 k=10 sel=1c33e4ec13c54172 minulps=11 mintok=0 x=1163.93996 logit=3016.45312 exp=531.942349 shr=4.26704559 lines=402 END +moesel call=210 dev=1 T=1 E=512 k=10 sel=25a8a4bc71d12bfa minulps=1 mintok=0 x=1086.59974 logit=3191.73438 exp=490.604975 shr=3.71147528 lines=403 END +moesel call=211 dev=1 T=1 E=512 k=10 sel=f5fbc184160044b4 minulps=1 mintok=0 x=1260.99685 logit=3202.71875 exp=612.229754 shr=4.33470666 lines=404 END +moesel call=212 dev=1 T=1 E=512 k=10 sel=c725a272191ace12 minulps=2 mintok=0 x=1533.60942 logit=3210.03125 exp=873.32988 shr=10.9670092 lines=405 END +moesel call=213 dev=1 T=1 E=512 k=10 sel=180fb3de7452b92a minulps=1 mintok=0 x=1426.39367 logit=3195.25 exp=709.281718 shr=5.5830484 lines=406 END +moesel call=214 dev=1 T=1 E=512 k=10 sel=f02ddca3898fa612 minulps=1 mintok=0 x=1375.95914 logit=3256.51562 exp=709.251355 shr=3.75101862 lines=407 END +moesel call=215 dev=1 T=1 E=512 k=10 sel=7bb2417988ed8e59 minulps=3 mintok=0 x=1674.38684 logit=3193.26562 exp=937.319845 shr=9.48965186 lines=408 END +moesel call=216 dev=1 T=1 E=512 k=10 sel=194d48a748c629c6 minulps=3 mintok=0 x=1206.20548 logit=3190.48438 exp=606.87991 shr=5.17114846 lines=409 END +moesel call=217 dev=1 T=1 E=512 k=10 sel=8d5fbf947b98670c minulps=1 mintok=0 x=1293.95078 logit=3205.15625 exp=650.469607 shr=5.57221315 lines=410 END +moesel call=218 dev=1 T=1 E=512 k=10 sel=ac4f5ffd3e72d566 minulps=2 mintok=0 x=1131.50172 logit=3202.33594 exp=550.584461 shr=4.01715562 lines=411 END +moesel call=219 dev=1 T=1 E=512 k=10 sel=82ef23facb2f8412 minulps=5 mintok=0 x=1201.47886 logit=3131.3125 exp=616.733163 shr=4.2928885 lines=412 END +moesel call=220 dev=1 T=1 E=512 k=10 sel=b644f72fad00f1ed minulps=2 mintok=0 x=1252.86975 logit=3241.01562 exp=669.556914 shr=5.77976474 lines=413 END +moesel call=221 dev=1 T=1 E=512 k=10 sel=93bd0ee37e07e15c minulps=1 mintok=0 x=1213.96096 logit=3224.45312 exp=694.328718 shr=7.16312608 lines=414 END +moesel call=222 dev=1 T=1 E=512 k=10 sel=eee468c72d623319 minulps=1 mintok=0 x=1494.2228 logit=3272.51562 exp=933.00448 shr=8.65537377 lines=415 END +moesel call=223 dev=1 T=1 E=512 k=10 sel=3a8eb23f98e69453 minulps=5 mintok=0 x=1945.89427 logit=3196.32812 exp=1337.74517 shr=9.30025354 lines=416 END +moesel call=224 dev=1 T=1 E=512 k=10 sel=8e7ef235d23ba453 minulps=1 mintok=0 x=1792.11004 logit=3157.46875 exp=1103.74508 shr=8.14367199 lines=417 END +moesel call=225 dev=1 T=1 E=512 k=10 sel=cd708bff21223dab minulps=1 mintok=0 x=1550.37602 logit=3131.53125 exp=943.768813 shr=6.73874525 lines=418 END +moesel call=226 dev=1 T=1 E=512 k=10 sel=a2592ab30223fbd7 minulps=1 mintok=0 x=1454.51285 logit=3359.14062 exp=880.687199 shr=4.52738642 lines=419 END +moesel call=227 dev=1 T=1 E=512 k=10 sel=7605e279ebbc9aa6 minulps=0 mintok=0 x=1718.42733 logit=3280.0625 exp=1116.49685 shr=7.29068101 lines=420 END +moesel call=228 dev=1 T=1 E=512 k=10 sel=0def7dfdc3e3fd69 minulps=1 mintok=0 x=2138.24944 logit=3269.1875 exp=1975.03493 shr=20.0015562 lines=421 END +moesel call=229 dev=1 T=1 E=512 k=10 sel=8be3456b4875ad2e minulps=0 mintok=0 x=2007.44902 logit=3257.98438 exp=1637.01024 shr=10.4202962 lines=422 END +moesel call=230 dev=1 T=1 E=512 k=10 sel=f9b2fcbf4e94b8c9 minulps=0 mintok=0 x=2039.25974 logit=3090.07812 exp=1880.78895 shr=14.8957555 lines=423 END +moesel call=231 dev=1 T=1 E=512 k=10 sel=44e72e18b98bed12 minulps=1 mintok=0 x=2088.80882 logit=3082.3125 exp=1784.81803 shr=22.1053065 lines=424 END +moesel call=232 dev=1 T=1 E=512 k=10 sel=09f0b82e8cf730f3 minulps=0 mintok=0 x=2106.75727 logit=3139.67188 exp=1539.96297 shr=14.7666921 lines=425 END +moesel call=233 dev=1 T=1 E=512 k=10 sel=0dff399b4770ee10 minulps=3 mintok=0 x=2241.38857 logit=3236.9375 exp=1788.18843 shr=12.8611548 lines=426 END +moesel call=234 dev=1 T=1 E=512 k=10 sel=e2b1245a1d963c28 minulps=0 mintok=0 x=2270.07075 logit=3207.75 exp=1944.56974 shr=23.0768353 lines=427 END +moesel call=235 dev=1 T=1 E=512 k=10 sel=34ecff52f133b24d minulps=1 mintok=0 x=1875.09357 logit=3110.0625 exp=1343.50007 shr=21.7689148 lines=428 END +moesel call=236 dev=1 T=1 E=512 k=10 sel=2ca538f02e32bb3c minulps=1 mintok=0 x=2224.76821 logit=3237.51562 exp=1791.08428 shr=27.2459818 lines=429 END +moesel call=237 dev=1 T=1 E=512 k=10 sel=3b5325c7ff30efba minulps=0 mintok=0 x=2308.26959 logit=3261.21875 exp=1658.76712 shr=30.9008512 lines=430 END +moesel call=238 dev=1 T=1 E=512 k=10 sel=4e3559090e65c272 minulps=0 mintok=0 x=2163.29301 logit=3203.39062 exp=1631.63319 shr=37.0887284 lines=431 END +moesel call=239 dev=1 T=1 E=512 k=10 sel=08c9ed7a4760f705 minulps=1 mintok=0 x=1978.85175 logit=3204.96875 exp=1603.01215 shr=144.048051 lines=432 END +moesel call=240 dev=1 T=1 E=512 k=10 sel=61b49097fc368a4f minulps=1 mintok=0 x=719.735275 logit=3487.75 exp=482.559906 shr=54.8279501 lines=433 END +moesel call=241 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=771.466531 logit=3071.32812 exp=416.046963 shr=5.94673173 lines=434 END +moesel call=242 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=3 mintok=0 x=787.406551 logit=3065.8125 exp=400.104052 shr=7.02664079 lines=435 END +moesel call=243 dev=1 T=1 E=512 k=10 sel=fcadc8fa2617fb34 minulps=1 mintok=0 x=1125.81332 logit=3166.0625 exp=581.746621 shr=3.30102856 lines=436 END +moesel call=244 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1082.08723 logit=2970.46875 exp=526.837637 shr=5.57845163 lines=437 END +moesel call=245 dev=1 T=1 E=512 k=10 sel=8bc65aa86c856784 minulps=1 mintok=0 x=1124.86482 logit=3119.34375 exp=504.797825 shr=5.05789583 lines=438 END +moesel call=246 dev=1 T=1 E=512 k=10 sel=62cc47e80cc73128 minulps=2 mintok=0 x=1321.76281 logit=3161.79688 exp=674.849031 shr=8.12079952 lines=439 END +moesel call=247 dev=1 T=1 E=512 k=10 sel=b7771ebaf5fb393a minulps=0 mintok=0 x=1168.06668 logit=3084.57812 exp=564.712501 shr=6.24083661 lines=440 END +moesel call=248 dev=1 T=1 E=512 k=10 sel=0c140a8e49a781fd minulps=3 mintok=0 x=1171.76134 logit=3066.1875 exp=599.744879 shr=6.866799 lines=441 END +moesel call=249 dev=1 T=1 E=512 k=10 sel=507f458d7a0dd61f minulps=1 mintok=0 x=1196.92192 logit=3047.9375 exp=566.048916 shr=5.14612947 lines=442 END +moesel call=250 dev=1 T=1 E=512 k=10 sel=11fefbb433e81ea5 minulps=0 mintok=0 x=1073.28715 logit=3136.92188 exp=504.051555 shr=3.50103059 lines=443 END +moesel call=251 dev=1 T=1 E=512 k=10 sel=b01f165a2aed8d4b minulps=3 mintok=0 x=1061.5559 logit=3038 exp=453.041076 shr=2.42076577 lines=444 END +moesel call=252 dev=1 T=1 E=512 k=10 sel=92e084d08d5d3f4e minulps=1 mintok=0 x=1111.23433 logit=3064.73438 exp=519.329495 shr=5.51540724 lines=445 END +moesel call=253 dev=1 T=1 E=512 k=10 sel=d906c5f6f48f0d41 minulps=1 mintok=0 x=984.467606 logit=3042.57812 exp=418.224721 shr=5.32077738 lines=446 END +moesel call=254 dev=1 T=1 E=512 k=10 sel=e73cb09b45b079b4 minulps=2 mintok=0 x=1224.14796 logit=3088.85938 exp=596.884094 shr=4.78430417 lines=447 END +moesel call=255 dev=1 T=1 E=512 k=10 sel=036ca995574452fa minulps=1 mintok=0 x=1623.95516 logit=3098.45312 exp=885.750901 shr=6.6510445 lines=448 END +moesel call=256 dev=1 T=1 E=512 k=10 sel=2c97bea4acec81a4 minulps=1 mintok=0 x=1440.77504 logit=3077.23438 exp=691.724926 shr=6.9816874 lines=449 END +moesel call=257 dev=1 T=1 E=512 k=10 sel=5beb06c103212760 minulps=2 mintok=0 x=1211.88206 logit=3146.01562 exp=610.742811 shr=5.3325901 lines=450 END +moesel call=258 dev=1 T=1 E=512 k=10 sel=cf56b1fef245c1b2 minulps=1 mintok=0 x=1087.62227 logit=3217.76562 exp=469.492844 shr=3.7592778 lines=451 END +moesel call=259 dev=1 T=1 E=512 k=10 sel=f1f15ef0786e7166 minulps=1 mintok=0 x=1260.5826 logit=3234.1875 exp=592.127562 shr=5.12547102 lines=452 END +moesel call=260 dev=1 T=1 E=512 k=10 sel=8cc149cf77964f24 minulps=1 mintok=0 x=1488.14397 logit=3273.53125 exp=789.547055 shr=8.75632502 lines=453 END +moesel call=261 dev=1 T=1 E=512 k=10 sel=a6b70eb4e1969847 minulps=3 mintok=0 x=1418.41927 logit=3256.96875 exp=685.589609 shr=4.91994519 lines=454 END +moesel call=262 dev=1 T=1 E=512 k=10 sel=7b7a3168b2ff06a2 minulps=6 mintok=0 x=1356.91032 logit=3266.42188 exp=664.668233 shr=4.00647543 lines=455 END +moesel call=263 dev=1 T=1 E=512 k=10 sel=f5590947f667141f minulps=0 mintok=0 x=1612.18464 logit=3188.35938 exp=874.37565 shr=7.22029203 lines=456 END +moesel call=264 dev=1 T=1 E=512 k=10 sel=7434b6bf2c085707 minulps=9 mintok=0 x=1213.43486 logit=3209.92188 exp=587.864345 shr=5.35805188 lines=457 END +moesel call=265 dev=1 T=1 E=512 k=10 sel=1d5b6fd1434dd984 minulps=4 mintok=0 x=1300.87791 logit=3272.32812 exp=610.208423 shr=5.07808681 lines=458 END +moesel call=266 dev=1 T=1 E=512 k=10 sel=933d7afaff8cc437 minulps=9 mintok=0 x=1113.05272 logit=3254.85156 exp=552.016545 shr=3.71943129 lines=459 END +moesel call=267 dev=1 T=1 E=512 k=10 sel=8ecaa8950e4d5f9d minulps=2 mintok=0 x=1190.53572 logit=3156.84375 exp=527.462512 shr=4.91369309 lines=460 END +moesel call=268 dev=1 T=1 E=512 k=10 sel=78e9d07fe08d6249 minulps=1 mintok=0 x=1243.89803 logit=3312.45312 exp=632.998852 shr=5.97065023 lines=461 END +moesel call=269 dev=1 T=1 E=512 k=10 sel=a4fdb20db9cc56a8 minulps=0 mintok=0 x=1238.8786 logit=3271.8125 exp=660.144664 shr=9.16139148 lines=462 END +moesel call=270 dev=1 T=1 E=512 k=10 sel=24eddf1056185207 minulps=2 mintok=0 x=1450.94733 logit=3379.29688 exp=770.611199 shr=7.09821257 lines=463 END +moesel call=271 dev=1 T=1 E=512 k=10 sel=6871cc1e230785ad minulps=2 mintok=0 x=1706.86826 logit=3224.35938 exp=1037.66972 shr=11.8690866 lines=464 END +moesel call=272 dev=1 T=1 E=512 k=10 sel=e3f5190e6d703653 minulps=2 mintok=0 x=1596.52113 logit=3170.98438 exp=911.640029 shr=11.5912727 lines=465 END +moesel call=273 dev=1 T=1 E=512 k=10 sel=2dfd19a0fc02b740 minulps=2 mintok=0 x=1459.34425 logit=3273.57812 exp=834.476069 shr=7.15768957 lines=466 END +moesel call=274 dev=1 T=1 E=512 k=10 sel=e08075eec512f324 minulps=6 mintok=0 x=1384.07997 logit=3403.46875 exp=800.853358 shr=5.02991112 lines=467 END +moesel call=275 dev=1 T=1 E=512 k=10 sel=5447e5522c9978ba minulps=0 mintok=0 x=1565.34497 logit=3261.125 exp=949.541952 shr=7.78171557 lines=468 END +moesel call=276 dev=1 T=1 E=512 k=10 sel=52face1eca90650c minulps=3 mintok=0 x=1849.35504 logit=3320.70312 exp=1248.77513 shr=18.1390751 lines=469 END +moesel call=277 dev=1 T=1 E=512 k=10 sel=aba9dd5f26da312b minulps=3 mintok=0 x=1773.53765 logit=3290.03125 exp=1154.50591 shr=9.91195093 lines=470 END +moesel call=278 dev=1 T=1 E=512 k=10 sel=06c460b9e38d97eb minulps=2 mintok=0 x=1860.26612 logit=3194.70312 exp=1618.3548 shr=12.8954418 lines=471 END +moesel call=279 dev=1 T=1 E=512 k=10 sel=ab07fba4d130b9e5 minulps=0 mintok=0 x=1894.73928 logit=3038.07812 exp=1262.50809 shr=21.319008 lines=472 END +moesel call=280 dev=1 T=1 E=512 k=10 sel=f51bd81f34330756 minulps=0 mintok=0 x=2039.29698 logit=3141.40625 exp=1378.85824 shr=14.5558079 lines=473 END +moesel call=281 dev=1 T=1 E=512 k=10 sel=38ba42a21cb96827 minulps=5 mintok=0 x=2034.52425 logit=3255.6875 exp=1556.42675 shr=10.1677739 lines=474 END +moesel call=282 dev=1 T=1 E=512 k=10 sel=caea884e4f8a421b minulps=3 mintok=0 x=2201.27892 logit=3181.125 exp=1767.24185 shr=21.6390068 lines=475 END +moesel call=283 dev=1 T=1 E=512 k=10 sel=85126cca847fe0ae minulps=0 mintok=0 x=1888.35798 logit=3154.57812 exp=1305.97696 shr=18.1589696 lines=476 END +moesel call=284 dev=1 T=1 E=512 k=10 sel=73334fc721c55436 minulps=3 mintok=0 x=2119.56377 logit=3171.78125 exp=1850.2893 shr=21.9173639 lines=477 END +moesel call=285 dev=1 T=1 E=512 k=10 sel=e51cb50a0f7c4c64 minulps=2 mintok=0 x=2235.23249 logit=3218.25 exp=1667.06572 shr=24.3306805 lines=478 END +moesel call=286 dev=1 T=1 E=512 k=10 sel=3ff312bf57506b4c minulps=0 mintok=0 x=2157.19093 logit=3175.09375 exp=1583.27752 shr=35.8604115 lines=479 END +moesel call=287 dev=1 T=1 E=512 k=10 sel=a71eb83b635761a7 minulps=2 mintok=0 x=1994.61945 logit=3186.25 exp=1613.20293 shr=139.88128 lines=480 END +moesel call=288 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=1 mintok=0 x=720.908846 logit=3493.3125 exp=470.299978 shr=56.0763303 lines=481 END +moesel call=289 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=0 mintok=0 x=770.535929 logit=3054.75 exp=390.753702 shr=6.59788024 lines=482 END +moesel call=290 dev=1 T=1 E=512 k=10 sel=19a4aa22fd55ceb9 minulps=1 mintok=0 x=797.194718 logit=3063.875 exp=449.164568 shr=7.77219334 lines=483 END +moesel call=291 dev=1 T=1 E=512 k=10 sel=2a01e5c40d6cc0a3 minulps=0 mintok=0 x=1130.49359 logit=3148.71875 exp=630.243819 shr=3.22358381 lines=484 END +moesel call=292 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1109.70885 logit=3062.89062 exp=617.032947 shr=4.58339378 lines=485 END +moesel call=293 dev=1 T=1 E=512 k=10 sel=25a7c1bf08883893 minulps=1 mintok=0 x=1195.05093 logit=3230.92188 exp=558.708276 shr=5.87885763 lines=486 END +moesel call=294 dev=1 T=1 E=512 k=10 sel=fe9752fc405e7f45 minulps=1 mintok=0 x=1351.35708 logit=3224.25 exp=758.855099 shr=8.13239998 lines=487 END +moesel call=295 dev=1 T=1 E=512 k=10 sel=23e39c29cd78e156 minulps=1 mintok=0 x=1231.14917 logit=3168.6875 exp=640.549042 shr=6.48122781 lines=488 END +moesel call=296 dev=1 T=1 E=512 k=10 sel=335dd0bd48b7fead minulps=0 mintok=0 x=1251.82667 logit=3166.5 exp=653.138235 shr=7.89882357 lines=489 END +moesel call=297 dev=1 T=1 E=512 k=10 sel=1e6ef2e2bd5bbe6e minulps=1 mintok=0 x=1291.58003 logit=3089.85938 exp=646.445662 shr=5.78292927 lines=490 END +moesel call=298 dev=1 T=1 E=512 k=10 sel=6d9a04e77a545573 minulps=1 mintok=0 x=1128.54297 logit=3227.82031 exp=573.681107 shr=4.35932608 lines=491 END +moesel call=299 dev=1 T=1 E=512 k=10 sel=431ad89c3a34fdb5 minulps=3 mintok=0 x=1160.22196 logit=3076.54688 exp=552.155695 shr=3.10065097 lines=492 END +moesel call=300 dev=1 T=1 E=512 k=10 sel=98cbb41200451908 minulps=2 mintok=0 x=1205.60989 logit=3072.40625 exp=588.010341 shr=4.60971649 lines=493 END +moesel call=301 dev=1 T=1 E=512 k=10 sel=cee73503e291c554 minulps=0 mintok=0 x=1049.94109 logit=3148.65625 exp=465.748932 shr=5.53982265 lines=494 END +moesel call=302 dev=1 T=1 E=512 k=10 sel=cbcf5808f868f2ee minulps=2 mintok=0 x=1300.08776 logit=3157.73438 exp=787.313637 shr=5.8626614 lines=495 END +moesel call=303 dev=1 T=1 E=512 k=10 sel=a176075303636b24 minulps=0 mintok=0 x=1626.80483 logit=3080.9375 exp=892.795173 shr=9.08575885 lines=496 END +moesel call=304 dev=1 T=1 E=512 k=10 sel=81f99954a1527b57 minulps=2 mintok=0 x=1465.01153 logit=3071.3125 exp=722.733711 shr=6.2803769 lines=497 END +moesel call=305 dev=1 T=1 E=512 k=10 sel=916f8776965ecba6 minulps=1 mintok=0 x=1268.95674 logit=3154.8125 exp=594.541258 shr=4.35259996 lines=498 END +moesel call=306 dev=1 T=1 E=512 k=10 sel=23826f8f1cfaf52c minulps=0 mintok=0 x=1170.08739 logit=3193.98438 exp=583.54809 shr=3.92149412 lines=499 END +moesel call=307 dev=1 T=1 E=512 k=10 sel=454397dc42f5b948 minulps=0 mintok=0 x=1291.48043 logit=3151.01562 exp=653.289927 shr=4.64947504 lines=500 END +moesel call=308 dev=1 T=1 E=512 k=10 sel=a693e98db6eaab69 minulps=1 mintok=0 x=1532.17177 logit=3179.4375 exp=895.965714 shr=9.50969727 lines=501 END +moesel call=309 dev=1 T=1 E=512 k=10 sel=e60de2e9e70d1f03 minulps=0 mintok=0 x=1490.63683 logit=3207.1875 exp=796.308669 shr=5.03227582 lines=502 END +moesel call=310 dev=1 T=1 E=512 k=10 sel=eda1b89e68411e03 minulps=5 mintok=0 x=1412.96221 logit=3241.04688 exp=750.6313 shr=4.91231269 lines=503 END +moesel call=311 dev=1 T=1 E=512 k=10 sel=492c4ee841cd0697 minulps=0 mintok=0 x=1547.72312 logit=3138.32812 exp=746.602557 shr=12.6945544 lines=504 END +moesel call=312 dev=1 T=1 E=512 k=10 sel=bb55ca07e4958d50 minulps=10 mintok=0 x=1232.44339 logit=3230.60938 exp=618.057561 shr=5.98809077 lines=505 END +moesel call=313 dev=1 T=1 E=512 k=10 sel=33bd5f8fd4274cde minulps=2 mintok=0 x=1307.53513 logit=3320.01562 exp=618.332616 shr=5.09537409 lines=506 END +moesel call=314 dev=1 T=1 E=512 k=10 sel=8d9f1ba184eaaf3c minulps=1 mintok=0 x=1146.61759 logit=3333.82812 exp=555.122777 shr=3.84388512 lines=507 END +moesel call=315 dev=1 T=1 E=512 k=10 sel=8f52b8b32a27c50c minulps=3 mintok=0 x=1203.73064 logit=3289.34375 exp=518.891168 shr=3.55087181 lines=508 END +moesel call=316 dev=1 T=1 E=512 k=10 sel=b06159110032b073 minulps=0 mintok=0 x=1267.07388 logit=3378.70312 exp=638.88724 shr=4.50843236 lines=509 END +moesel call=317 dev=1 T=1 E=512 k=10 sel=5ac221458ae3f815 minulps=0 mintok=0 x=1248.944 logit=3323.96875 exp=663.817585 shr=6.28959796 lines=510 END +moesel call=318 dev=1 T=1 E=512 k=10 sel=3692915ef113b95c minulps=6 mintok=0 x=1543.87815 logit=3373.25 exp=906.074797 shr=4.30320768 lines=511 END +moesel call=319 dev=1 T=1 E=512 k=10 sel=5654c4f757f5adc0 minulps=1 mintok=0 x=1784.87576 logit=3225.09375 exp=1127.58558 shr=6.99982764 lines=512 END +moesel call=320 dev=1 T=1 E=512 k=10 sel=cce9d411277ac7a0 minulps=2 mintok=0 x=1731.88059 logit=3257.48438 exp=954.358576 shr=6.33040473 lines=513 END +moesel call=321 dev=1 T=1 E=512 k=10 sel=df028f29b5993b25 minulps=2 mintok=0 x=1564.45086 logit=3376.35938 exp=917.965805 shr=4.72212958 lines=514 END +moesel call=322 dev=1 T=1 E=512 k=10 sel=c2f29cfbcc871c32 minulps=1 mintok=0 x=1470.7524 logit=3463.03125 exp=889.904336 shr=4.56328436 lines=515 END +moesel call=323 dev=1 T=1 E=512 k=10 sel=a08c6e7a5893f6cd minulps=10 mintok=0 x=1659.91617 logit=3319.4375 exp=1033.99195 shr=5.26986123 lines=516 END +moesel call=324 dev=1 T=1 E=512 k=10 sel=f9ee760e41a3c91d minulps=0 mintok=0 x=2027.81188 logit=3398.8125 exp=1750.47563 shr=13.1384588 lines=517 END +moesel call=325 dev=1 T=1 E=512 k=10 sel=e178771deea6c5ce minulps=7 mintok=0 x=1956.54692 logit=3331.42188 exp=1368.12519 shr=6.88059388 lines=518 END +moesel call=326 dev=1 T=1 E=512 k=10 sel=0fafc2fab555f6e9 minulps=1 mintok=0 x=1991.47472 logit=3239 exp=1614.7162 shr=13.9327306 lines=519 END +moesel call=327 dev=1 T=1 E=512 k=10 sel=6c7e393659972261 minulps=1 mintok=0 x=1906.79834 logit=3088.10938 exp=1370.20132 shr=17.0168733 lines=520 END +moesel call=328 dev=1 T=1 E=512 k=10 sel=8f36dc7be08e3931 minulps=2 mintok=0 x=2039.20587 logit=3143.9375 exp=1405.13947 shr=11.7286562 lines=521 END +moesel call=329 dev=1 T=1 E=512 k=10 sel=9e648ffe259dfbae minulps=3 mintok=0 x=2057.31357 logit=3195.73438 exp=1494.29283 shr=9.69284485 lines=522 END +moesel call=330 dev=1 T=1 E=512 k=10 sel=2982a17c543ebe92 minulps=0 mintok=0 x=2169.32461 logit=3203.21875 exp=1812.67176 shr=20.3597129 lines=523 END +moesel call=331 dev=1 T=1 E=512 k=10 sel=91e5443eedd29105 minulps=3 mintok=0 x=1929.6716 logit=3095.28125 exp=1407.73551 shr=16.6108131 lines=524 END +moesel call=332 dev=1 T=1 E=512 k=10 sel=e656fd5283a60b62 minulps=1 mintok=0 x=2128.8884 logit=3177 exp=1691.97369 shr=23.9393592 lines=525 END +moesel call=333 dev=1 T=1 E=512 k=10 sel=44a985e32c2f6607 minulps=1 mintok=0 x=2143.73433 logit=3217.01562 exp=1485.41742 shr=30.2132944 lines=526 END +moesel call=334 dev=1 T=1 E=512 k=10 sel=7ffb8d605828846f minulps=1 mintok=0 x=2044.79437 logit=3170.875 exp=1378.33216 shr=33.7946244 lines=527 END +moesel call=335 dev=1 T=1 E=512 k=10 sel=4ff4040682a26031 minulps=0 mintok=0 x=1870.00026 logit=3166.34375 exp=1537.97981 shr=141.048263 lines=528 END +moesel call=336 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=0 mintok=0 x=720.015304 logit=3494.46875 exp=464.331417 shr=56.7611626 lines=529 END +moesel call=337 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=1 mintok=0 x=769.239557 logit=3052.85938 exp=383.075755 shr=6.90563814 lines=530 END +moesel call=338 dev=1 T=1 E=512 k=10 sel=da8470c4d884deef minulps=2 mintok=0 x=781.241141 logit=3044.625 exp=450.116965 shr=8.03351162 lines=531 END +moesel call=339 dev=1 T=1 E=512 k=10 sel=d71ea9ac6dcf42f0 minulps=0 mintok=0 x=1123.97446 logit=3109.09375 exp=528.433402 shr=3.14524879 lines=532 END +moesel call=340 dev=1 T=1 E=512 k=10 sel=11e9c52f473f20ca minulps=1 mintok=0 x=1108.30387 logit=3072 exp=595.020402 shr=4.04507779 lines=533 END +moesel call=341 dev=1 T=1 E=512 k=10 sel=ccab85420d64391b minulps=0 mintok=0 x=1140.24286 logit=3136.26562 exp=521.452564 shr=5.89650467 lines=534 END +moesel call=342 dev=1 T=1 E=512 k=10 sel=b0359bc2c09a1b3b minulps=2 mintok=0 x=1284.18058 logit=3175.59375 exp=695.144514 shr=7.5284667 lines=535 END +moesel call=343 dev=1 T=1 E=512 k=10 sel=b8766a294ccc5e15 minulps=3 mintok=0 x=1179.17826 logit=3163.95312 exp=592.168702 shr=7.01145951 lines=536 END +moesel call=344 dev=1 T=1 E=512 k=10 sel=39dd045c2a0bf426 minulps=1 mintok=0 x=1173.72122 logit=3133.8125 exp=598.289271 shr=8.36464005 lines=537 END +moesel call=345 dev=1 T=1 E=512 k=10 sel=c8b450035b8bad2b minulps=0 mintok=0 x=1196.105 logit=3062.15625 exp=590.992329 shr=6.52784037 lines=538 END +moesel call=346 dev=1 T=1 E=512 k=10 sel=2a7287b05a0cab3f minulps=2 mintok=0 x=1093.39694 logit=3217.6875 exp=582.437427 shr=3.78586096 lines=539 END +moesel call=347 dev=1 T=1 E=512 k=10 sel=9f3b3343e48ffbf5 minulps=0 mintok=0 x=1171.20684 logit=3088.54688 exp=539.94768 shr=2.9270583 lines=540 END +moesel call=348 dev=1 T=1 E=512 k=10 sel=7fe4fe5d8951d439 minulps=3 mintok=0 x=1199.96908 logit=3061.85938 exp=600.276196 shr=4.67941533 lines=541 END +moesel call=349 dev=1 T=1 E=512 k=10 sel=7b00267773828180 minulps=0 mintok=0 x=1068.80267 logit=3080.96875 exp=474.210937 shr=5.04527323 lines=542 END +moesel call=350 dev=1 T=1 E=512 k=10 sel=3c5c42aa762d7ed2 minulps=4 mintok=0 x=1269.14098 logit=3085.70312 exp=730.125101 shr=4.51368469 lines=543 END +moesel call=351 dev=1 T=1 E=512 k=10 sel=deaab1e1a7541b87 minulps=2 mintok=0 x=1619.64559 logit=3075.14062 exp=874.925515 shr=8.89459471 lines=544 END +moesel call=352 dev=1 T=1 E=512 k=10 sel=6e5d044cf1fc7022 minulps=1 mintok=0 x=1484.82882 logit=3053.96875 exp=733.953595 shr=6.29550232 lines=545 END +moesel call=353 dev=1 T=1 E=512 k=10 sel=aa42bad017dc8cbc minulps=2 mintok=0 x=1289.43275 logit=3151.95312 exp=615.826015 shr=4.31546008 lines=546 END +moesel call=354 dev=1 T=1 E=512 k=10 sel=315b9f288ff0a182 minulps=3 mintok=0 x=1178.73817 logit=3183.14062 exp=547.426989 shr=3.78970516 lines=547 END +moesel call=355 dev=1 T=1 E=512 k=10 sel=bdb81852e778bc90 minulps=0 mintok=0 x=1329.85278 logit=3159.40625 exp=636.104008 shr=4.7916096 lines=548 END +moesel call=356 dev=1 T=1 E=512 k=10 sel=7034143c6c58be41 minulps=0 mintok=0 x=1552.82118 logit=3206.71875 exp=872.151475 shr=7.76180266 lines=549 END +moesel call=357 dev=1 T=1 E=512 k=10 sel=4cc169bd11ce9a33 minulps=0 mintok=0 x=1496.55086 logit=3206.70312 exp=733.508479 shr=5.00650799 lines=550 END +moesel call=358 dev=1 T=1 E=512 k=10 sel=4862310eebbd815e minulps=1 mintok=0 x=1348.99547 logit=3195.28125 exp=690.420928 shr=4.49610108 lines=551 END +moesel call=359 dev=1 T=1 E=512 k=10 sel=d5ecc24ee49916ac minulps=0 mintok=0 x=1622.07154 logit=3201.65625 exp=865.30765 shr=8.74974447 lines=552 END +moesel call=360 dev=1 T=1 E=512 k=10 sel=963b0fbb9cb6e81f minulps=0 mintok=0 x=1255.61019 logit=3156.42188 exp=659.038896 shr=5.84109655 lines=553 END +moesel call=361 dev=1 T=1 E=512 k=10 sel=1edc464c7723fe54 minulps=1 mintok=0 x=1336.06516 logit=3293.21875 exp=630.459126 shr=4.41336972 lines=554 END +moesel call=362 dev=1 T=1 E=512 k=10 sel=2626a43dbee38f3b minulps=3 mintok=0 x=1133.96029 logit=3218.69531 exp=583.583901 shr=4.19799819 lines=555 END +moesel call=363 dev=1 T=1 E=512 k=10 sel=80f08432dc533ac3 minulps=0 mintok=0 x=1200.27544 logit=3197.39062 exp=550.233979 shr=4.26960437 lines=556 END +moesel call=364 dev=1 T=1 E=512 k=10 sel=4de5de32929854dd minulps=1 mintok=0 x=1255.59562 logit=3333.96875 exp=631.117934 shr=3.87706444 lines=557 END +moesel call=365 dev=1 T=1 E=512 k=10 sel=7d8ae07aa246693f minulps=0 mintok=0 x=1252.15723 logit=3249.26562 exp=671.915036 shr=7.54002076 lines=558 END +moesel call=366 dev=1 T=1 E=512 k=10 sel=2c34bfad57a0f504 minulps=1 mintok=0 x=1507.71261 logit=3304.70312 exp=841.009994 shr=6.10757782 lines=559 END +moesel call=367 dev=1 T=1 E=512 k=10 sel=68714ec82c41b541 minulps=0 mintok=0 x=1763.37123 logit=3169.03125 exp=1111.12199 shr=12.374807 lines=560 END +moesel call=368 dev=1 T=1 E=512 k=10 sel=1f22675012ab1ea4 minulps=1 mintok=0 x=1654.11779 logit=3173.51562 exp=915.490735 shr=12.4345397 lines=561 END +moesel call=369 dev=1 T=1 E=512 k=10 sel=fd883978fa602ccb minulps=1 mintok=0 x=1490.08333 logit=3295.10938 exp=841.559377 shr=6.13698708 lines=562 END +moesel call=370 dev=1 T=1 E=512 k=10 sel=1298f8a8c115ca6d minulps=0 mintok=0 x=1382.55747 logit=3363.67188 exp=795.375256 shr=4.93426613 lines=563 END +moesel call=371 dev=1 T=1 E=512 k=10 sel=fafa416a41510e2b minulps=1 mintok=0 x=1588.0379 logit=3263.5625 exp=955.51882 shr=6.98780198 lines=564 END +moesel call=372 dev=1 T=1 E=512 k=10 sel=7fce82859a778218 minulps=1 mintok=0 x=1938.96774 logit=3336.21875 exp=1294.53322 shr=15.6524524 lines=565 END +moesel call=373 dev=1 T=1 E=512 k=10 sel=bcc944daed06c12b minulps=2 mintok=0 x=1902.17242 logit=3328.4375 exp=1293.49587 shr=8.27988695 lines=566 END +moesel call=374 dev=1 T=1 E=512 k=10 sel=9565301592511e79 minulps=1 mintok=0 x=1912.38708 logit=3257.60938 exp=1721.68549 shr=12.9984499 lines=567 END +moesel call=375 dev=1 T=1 E=512 k=10 sel=11e0a25c4aae080d minulps=2 mintok=0 x=1976.4549 logit=3087.9375 exp=1416.52003 shr=16.9319087 lines=568 END +moesel call=376 dev=1 T=1 E=512 k=10 sel=93e6ccc0425767d4 minulps=3 mintok=0 x=2087.62383 logit=3137.17188 exp=1523.74264 shr=11.2343423 lines=569 END +moesel call=377 dev=1 T=1 E=512 k=10 sel=34ffeb576a4d4586 minulps=2 mintok=0 x=2120.68981 logit=3262.29688 exp=1654.40233 shr=9.1849984 lines=570 END +moesel call=378 dev=1 T=1 E=512 k=10 sel=b5579a72498d18d8 minulps=0 mintok=0 x=2282.62504 logit=3218.20312 exp=1797.38524 shr=18.6403076 lines=571 END +moesel call=379 dev=1 T=1 E=512 k=10 sel=ccb1f591af99da75 minulps=0 mintok=0 x=1924.06312 logit=3161.35938 exp=1337.21027 shr=12.6647484 lines=572 END +moesel call=380 dev=1 T=1 E=512 k=10 sel=f239f060355c60fd minulps=0 mintok=0 x=2133.93015 logit=3152.23438 exp=1598.71725 shr=18.1373025 lines=573 END +moesel call=381 dev=1 T=1 E=512 k=10 sel=f83d76c3622c532f minulps=1 mintok=0 x=2125.87638 logit=3165.73438 exp=1455.09365 shr=19.723922 lines=574 END +moesel call=382 dev=1 T=1 E=512 k=10 sel=d81279237fcb3ff7 minulps=5 mintok=0 x=2100.57419 logit=3147.04688 exp=1383.72128 shr=29.6997827 lines=575 END +moesel call=383 dev=1 T=1 E=512 k=10 sel=dafd3703f38e1353 minulps=0 mintok=0 x=1889.96341 logit=3089.34375 exp=1556.76733 shr=114.923461 lines=576 END diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt new file mode 100644 index 000000000..af660cbf0 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt @@ -0,0 +1,75 @@ +RESULT GPU: NVIDIA Thor, 11.0, 595.78 +RESULT NVCC: Cuda compilation tools, release 13.0, V13.0.88 Build cuda_13.0.r13.0/compiler.36424714_0 +RESULT SOURCE TARBALL sha256=be5f31ec1d0b3c7ce4ed724647cd9295caa48db62dbff1689e501654d775fd44 +RESULT SOURCE TARBALL HEAD: 8e2ba6931dd5aae33955ffdd8e7258ff5d90eef0 +RESULT INSTRUMENT IN SOURCE: MoeSelFp=2(2) env=1(1) +RESULT ARTIFACT STAGE REUSED (no copy) +RESULT ARTIFACT VERIFIED: shard1 sha256=88a1420825a9304063e882ada29d438263617f51ac8923d438d927496693bafd bytes=72546461650 +RESULT CMAKE rc=0 +RESULT BUILD rc=0 wall=921s objects=585 +RESULT BINARY: /tmp/moediv-bld/examples/vllm-server sha256=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf built=2026-09-02T07:06:21Z +RESULT BINARY CARRIES THE INSTRUMENT: strings_hits=1 +RESULT TAP GATE OFF rc=0 [doctest] assertions: 121 | 121 passed | 0 failed | +RESULT TAP GATE ON rc=0 [doctest] assertions: 172 | 172 passed | 0 failed | +RESULT ARM CPU-CTRL START bin_sha=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf device=cpu env='VT_CPU_QUANT_REPACK=0' +RESULT ARM CPU-CTRL HEALTH up=1 load_wall=45s +RESULT ARM CPU-CTRL REQ http=200 curl_rc=0 wall=6s +RESULT ARM CPU-CTRL TOKEN IDS: 11751 13 15767 411 2029 11 1092 369 +RESULT ARM CPU-CTRL MATCHES CONTROL: YES +RESULT ARM CPU-CTRL MOESEL LINES: 960 +RESULT ARM CPU-CTRL MOESEL DIGESTS: 384 +RESULT ARM CPU-CTRL MOESEL LAST lines=: lines=576 +RESULT ARM CPU-CTRL PREFILL CALLS (T=5): 48 +RESULT ARM CPU-CTRL steps completed: 11 +RESULT ARM CPU-CTRL FIRST ERROR LINE: +RESULT ARM CPU-CTRL END +RESULT ARM CUDA-PROD START bin_sha=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf device=cuda env='VT_CPU_QUANT_REPACK=0' +RESULT ARM CUDA-PROD HEALTH up=1 load_wall=15s +RESULT ARM CUDA-PROD REQ http=200 curl_rc=0 wall=156s +RESULT ARM CUDA-PROD TOKEN IDS: 11751 271 271 271 271 271 0 0 +RESULT ARM CUDA-PROD MATCHES CONTROL: NO (control='11751 13 15767 411 2029 11 1092 369') +RESULT ARM CUDA-PROD MOESEL LINES: 960 +RESULT ARM CUDA-PROD MOESEL DIGESTS: 384 +RESULT ARM CUDA-PROD MOESEL LAST lines=: lines=576 +RESULT ARM CUDA-PROD PREFILL CALLS (T=5): 48 +RESULT ARM CUDA-PROD steps completed: 11 +RESULT ARM CUDA-PROD FIRST ERROR LINE: +RESULT ARM CUDA-PROD END +RESULT ARM CUDA-GDNSEQ START bin_sha=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf device=cuda env='VT_CPU_QUANT_REPACK=0 VT_GDN_CHUNKED=0' +RESULT ARM CUDA-GDNSEQ HEALTH up=1 load_wall=15s +RESULT ARM CUDA-GDNSEQ REQ http=200 curl_rc=0 wall=223s +RESULT ARM CUDA-GDNSEQ TOKEN IDS: 11751 271 271 11751 271 271 271 0 +RESULT ARM CUDA-GDNSEQ MATCHES CONTROL: NO (control='11751 13 15767 411 2029 11 1092 369') +RESULT ARM CUDA-GDNSEQ MOESEL LINES: 960 +RESULT ARM CUDA-GDNSEQ MOESEL DIGESTS: 384 +RESULT ARM CUDA-GDNSEQ MOESEL LAST lines=: lines=576 +RESULT ARM CUDA-GDNSEQ PREFILL CALLS (T=5): 48 +RESULT ARM CUDA-GDNSEQ steps completed: 11 +RESULT ARM CUDA-GDNSEQ FIRST ERROR LINE: +RESULT ARM CUDA-GDNSEQ END +RESULT CMP CUDA-PROD rc=0 lines=80 +RESULT CMP CUDA-PROD COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +RESULT CMP CUDA-PROD COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-PROD COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-PROD COUNT PREFILL calls=48 sel-hash MISMATCH=40 +RESULT CMP CUDA-PROD COUNT PREFILL token-slots=240 FLIPPED=78 +RESULT CMP CUDA-PROD FIRST PREFILL FLIP call=0 tok=2 +RESULT CMP CUDA-PROD MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +RESULT CMP CUDA-PROD COUNT DECODE calls=336 sel-hash MISMATCH=336 +RESULT CMP CUDA-PROD COUNT DECODE token-slots=336 FLIPPED=336 +RESULT CMP CUDA-PROD FIRST DECODE FLIP call=48 tok=0 +RESULT CMP CUDA-PROD MIN DECODE base minulps over calls: min=0 p10=0 median=1 +RESULT CMP CUDA-PROD VERDICT prefill expert-selection SET EQUALITY: DIFFERS (78 flipped token-slots) +RESULT CMP CUDA-GDNSEQ rc=0 lines=80 +RESULT CMP CUDA-GDNSEQ COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +RESULT CMP CUDA-GDNSEQ COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-GDNSEQ COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-GDNSEQ COUNT PREFILL calls=48 sel-hash MISMATCH=38 +RESULT CMP CUDA-GDNSEQ COUNT PREFILL token-slots=240 FLIPPED=75 +RESULT CMP CUDA-GDNSEQ FIRST PREFILL FLIP call=1 tok=2 +RESULT CMP CUDA-GDNSEQ MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +RESULT CMP CUDA-GDNSEQ COUNT DECODE calls=336 sel-hash MISMATCH=336 +RESULT CMP CUDA-GDNSEQ COUNT DECODE token-slots=336 FLIPPED=336 +RESULT CMP CUDA-GDNSEQ FIRST DECODE FLIP call=48 tok=0 +RESULT CMP CUDA-GDNSEQ MIN DECODE base minulps over calls: min=0 p10=0 median=1 +RESULT CMP CUDA-GDNSEQ VERDICT prefill expert-selection SET EQUALITY: DIFFERS (75 flipped token-slots)