diff --git a/.agents/specs/qwen4-exp-flash-next.md b/.agents/specs/qwen4-exp-flash-next.md index 0110ac6bc..7e0addb8a 100644 --- a/.agents/specs/qwen4-exp-flash-next.md +++ b/.agents/specs/qwen4-exp-flash-next.md @@ -8678,6 +8678,173 @@ product-behaviour change in this wave, because the measurement says there is no defect at the first divergence to change. +## Wave MOEDIV — is the MoE prefill divergence a SELECTION FLIP? (#2552) + +### The question, and why the existing instrument cannot answer it + +Wave PREFILLDIV ([#2547](https://github.com/mudler/vllm.cpp/issues/2547), +[#2554](https://github.com/mudler/vllm.cpp/pull/2554)) isolated TWO sources of +CPU/CUDA prefill divergence at decoder layer 0, the only layer whose input is +bit-identical on both CUDA arms. The first is the chunked Gated DeltaNet +prefill, and it is not a defect: it mirrors vLLM's vendored Flash-Linear-Attention +precision map at all five anchors. The second is the MoE block, which turns a +`2.1e-05` input difference into a `7.269e-05` output difference and which +`VT_GDN_CHUNKED=0` moves by only 1.7x, against 332x on the Gated DeltaNet tap. +[#2552](https://github.com/mudler/vllm.cpp/issues/2552) owns that second source. + +`VT_Q4EXP_LAYER_FP` taps VALUES. **A value tap cannot separate an +expert-selection flip from re-association inside the expert GEMM**, because a +discrete selection has bimodal error and not a tolerance: one token's top-k set +either changes, and that token's MoE output changes by an O(1) amount, or it +does not change at all and the residue is rounding. Averaging both into one +`rel(sum|x|)` destroys exactly the bit that decides which of the two happened. + +### What this wave lands + +`VT_MOE_SEL_FP=` — a **selected-expert-id tap** on the reference arm of +the shared sparse-MoE block (`MoeBlock`, `qwen3_5.cpp`), which is the arm +`RunQwen4ExpMoeBlock` reaches on a stacked keep-quant checkpoint. For the first +`` block invocations of the process it prints, per token: + +* the selected expert ids **SORTED**, because the assertion between two arms is + SET equality and the selection ORDER is not part of it (`vt::MoeCombine` sums + over the k slots and is order-invariant given the weights); +* `lo`, the smallest selected router logit, and `hi`, the largest rejected one, + each with its raw bf16 bit pattern; +* `margin = lo - hi` in **LOGIT** space rather than probability space, because + the softmax denominator is itself a device-order f32 reduction and differs + between the arms, while the bf16 logits are the values the selection is + actually a function of; +* `ulps`, the number of representable **bf16** steps between `hi` and `lo` + under the sign-magnitude total order. `ulps=0` means the two logits are the + SAME bf16 value and the boundary was decided by the lowest-index tie-break — + a knife edge that one ulp of re-association anywhere upstream will flip. + `ulps` is the honest margin for a bimodal error: a probability difference + reads as "small" for a gap of one representable step and for a gap of fifty. + +and, per call, one digest line carrying `sel` (an FNV-1a hash over every token's +sorted id list, so **selection-set equality between two arms is one string +comparison per layer**), the minimum `ulps` over the call's tokens, and four +`sum|x|` axes that decompose the block's own output: `x` (the block input), +`logit` (the router logits), `exp` (the assembled per-slot expert outputs before +the combine) and `shr` (the shared expert). With the selection sets equal, those +four say WHICH of the block's GEMMs carries the residue; with them unequal, the +per-token lines say which token flipped and how near the tie was. + +### The counted property + +`lines=` on every digest line is the running total of value lines the tap has +printed. **An instrument that never ran and two arms whose taps agreed look the +same in a diff**, and this row has already paid for that twice (a job that +searched `$BLD/vllm-server` while ninja links `examples/vllm-server`, and a +doctest case whose name contained a comma so `-tc` split it). The measuring job +asserts `lines == calls * T` and refuses to report a comparison otherwise. + +### Scope, and what this wave does NOT do + +It is a diagnostic tap and one measurement. It changes no math on any arm: the +tap reads host buffers the reference path had already downloaded (`h`, `logits`, +`ids`, `expert_out`) plus one guarded download of the shared-expert output, and +it is inert with the environment variable unset. It lands on the reference arm +only. The three fused CUDA arms (`MoeBlockFusedCuda`, `MoeBlockFusedMarlinCuda`, +`MoeBlockBf16Cuda`) keep their ids on device and are NOT tapped, which is +recorded rather than hidden: no `qwen4_exp` checkpoint reaches them, and adding +a device readback to a capturable path to instrument a model that cannot enter +it would be dead code by construction. + +### Gates + +* `ctest -R test_qwen4_exp_moe` at rc 0, with the tap's ids and `ulps` gated + against `MoeReference`, the suite's independent double-precision oracle + reimplementation, on both the bf16 and the keep-quant arm. +* The reachability mutation deletes the tap's production call site inside + `MoeBlock` and the tap case must RED. +* `scripts/agent-preflight.sh --fail-on-skip` at rc 0. + +### Evidence required + +One `rc` lease on `thor:gpu0`, one binary, the released +`unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S artifact (shard 1 sha256 +`88a1420825a9304063e882ada29d438263617f51ac8923d438d927496693bafd`), the same +prompt and server flags PREFILLDIV used, and three arms: `--device cpu` as the +control, `--device cuda`, and `--device cuda` with `VT_GDN_CHUNKED=0` so the +MoE residue is read with the first source removed. The result is the per-layer +`sel` hash comparison plus the `ulps` distribution. + +### Stop conditions + +Stop and report `PENDING` if the artifact cannot be staged; do not substitute +another checkpoint. Stop if the answer is that the CUDA arm mirrors vLLM and our +CPU arm is the outlier, exactly as PREFILLDIV concluded for the Gated DeltaNet: +say so with the upstream anchors and close #2552 as answered rather than fixed. +**Do not force a fix.** Changing a CUDA arm to agree with our CPU arm, when the +CUDA arm is the one that mirrors the oracle, moves this model AWAY from vLLM. + +### Outcome + +**MEASURED. The selections DIFFER, and it is NOT a defect.** Full result in +[the evidence file](../../docs/bench-evidence/qwen4exp-moe-selection-20260902.md); +`thor:gpu0` `sm_110`, job `a5bf074b-0f1a-490d-ab51-5d561857ef9e`, one binary +`b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf`, three arms, +the released UD-IQ1_S artifact verified inside the lease. + +**The number that answers the row.** At `E = 512` and `top_k = 10` the router's +top-k boundary is an **EXACT bf16 tie at 79 of 240 prefill token-slots (32.9%)** +on the CPU control, and inside one representable bf16 step at 55.8%. At a tie +the selection carries no information: the lowest-index tie-break decides it. The +flip rate IS the tie rate — 75 of 240 slots flip on the `VT_GDN_CHUNKED=0` arm +(31.3%), 78 on the production arm. + +**#2552's own headline number is NOT a flip, and this is the half the issue got +wrong.** At layer 0 with the Gated DeltaNet source removed — the arm its table +was taken on — both arms select the same experts for all five tokens. The +`7.269e-05` residue decomposes onto the expert GEMM: `x` 2.139e-05, `logit` +2.378e-05 (the router GEMM does not amplify), `exp` **1.421e-04** (6.6x), `shr` +4.310e-05. The keep-quant grouped GEMM is the amplifier, and +`cuda_quant_dot.cu:2158-2170` already says what it is — the integer core is +bit-identical and only the per-block float scale sum is reassociated, which +llama.cpp's own CPU/CUDA split does too. + +**Why it is not a defect.** vLLM routes this model on bf16 logits at 512 experts +as well: `Qwen4ExpSparseMoeBlock` inherits Qwen3Next's plain `ReplicatedLinear` +gate with no `params_dtype`, `moe_runner.py:897-902` runs it as a plain +`F.linear`, and the f32 widening happens inside `topk_softmax` — exactly our +polarity, our widening point and our lowest-index tie-break. Upstream sits on +the same knife edge. Read at `cdefd9d499`, a FORWARD REFERENCE 1566 commits past +the pin, which carries no `qwen4_exp` at all. **This is a source read, not a +measurement: no vLLM process was run on this checkpoint.** + +**One thing to WATCH rather than fix.** vLLM owns a fp32-capable router gate, +`GateLinear` (`gate_linear.py:18-33`), a `PluggableLayer` with three of five +tiers emitting fp32 logits. Qwen3Next does not opt in, so it is unreachable for +this architecture today. If upstream ever flips `qwen4_exp` onto it we mirror +that; widening ours first would move this model AWAY from the oracle, which is +what the stop condition above forbids. + +**The gate question, settled with a mechanism.** PREFILLDIV observed that token +agreement is not monotone in numerical distance and proposed no CPU-vs-CUDA +token gate. This says WHY: a third of the routing decisions carry zero margin, +so the emitted sequence is a function of tie-break order rather than of +accuracy. **No CPU-vs-CUDA token-exactness gate is well posed for `qwen4_exp`.** +The selection-set agreement rate and the tie-rate histogram ARE well posed and +are now measurable; a gate against vLLM stays the right target and stays OWED, +because vLLM's GGUF support is an out-of-tree plugin and every safetensors arm +of this model exceeds the largest fleet box. + +**What this run could NOT prove, and one thing it found by accident.** The +decode half is VOID: measured on `origin/main` `a99b9c69a`, the CUDA arm answers +`11751 271 271 271 271 271 0 0`, not the fluent sequence PREFILLDIV recorded, +because that wave measured a tree carrying +[#2550](https://github.com/mudler/vllm.cpp/pull/2550)'s decode fix and this one +did not. From step 1 the two arms therefore run different token sequences, so +the comparator's 336-of-336 decode flip count is a different-input artifact and +is published only to say so. **#2550 landed at `bb78d1ee8` while this wave was +writing up**, which is why this branch carries a merge of it; the numbers above +were taken BEFORE that merge and are not re-derived here. A decode-phase +selection comparison is now well posed and is NOT claimed by this wave — the +prefill result stands on its own, and re-running the tap on a post-#2550 tree is +the cheap next measurement. + ## Now `ACTIVE`. **THE COUNT IS THE TABLE, AND THIS SENTENCE NO LONGER RESTATES IT.** diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902.md b/docs/bench-evidence/qwen4exp-moe-selection-20260902.md new file mode 100644 index 000000000..f7b8d6bfd --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902.md @@ -0,0 +1,245 @@ +# The `qwen4_exp` MoE routers DO pick different experts, and the reason is that a third of the boundaries are EXACT TIES, 2 September 2026 + +Wave MOEDIV of [`MODEL-MM-QWEN4-EXP`](../../.agents/specs/qwen4-exp-flash-next.md), +[#2552](https://github.com/mudler/vllm.cpp/issues/2552). + +**The one-line result.** The two arms' expert selections **DIFFER** — 75 of 240 +prefill token-slots on the arm #2552 measured, 78 of 240 on the production CUDA +arm. **They are not a defect.** At `num_experts = 512` and +`num_experts_per_tok = 10` the router's top-k boundary is an EXACT bf16 tie at +**79 of 240 (32.9%)** of prefill token-slots on the CPU control, and within ONE +representable bf16 step at 55.8%. At a tie the selection carries no information +at all: it is decided by the lowest-index tie-break, and any perturbation +anywhere upstream reshuffles it. **The flip rate is the tie rate** — 31.3% of +slots flip against 32.9% tied. + +**And #2552's own headline number is NOT a flip.** At decoder layer 0 with the +Gated DeltaNet source removed, which is the arm the issue's table was taken on, +the two arms select the **same experts** for all five tokens. The `7.269e-05` +residue there is arithmetic, and this run says which GEMM produces it. + +## 1. What was run, and on what + +**Tree.** `8e2ba6931dd5aae33955ffdd8e7258ff5d90eef0` — `origin/main` `a99b9c69a` +plus this wave's `VT_MOE_SEL_FP` tap and nothing else. Source tarball sha256 +`be5f31ec1d0b3c7ce4ed724647cd9295caa48db62dbff1689e501654d775fd44`. + +**Host.** `thor:gpu0` under an `rc` lease, job +`a5bf074b-0f1a-490d-ab51-5d561857ef9e`, pod `rc-worker-n8smh`, aarch64, NVIDIA +Thor, compute capability 11.0, driver 595.78, nvcc 13.0.88, built `sm_110`. +`CMAKE rc=0`; `BUILD rc=0 wall=921s objects=585`; server +`b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf`, ONE binary +for all three arms. + +**Artifact.** The released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S, shard 1 +sha256 `88a1420825a9304063e882ada29d438263617f51ac8923d438d927496693bafd`, +72,546,461,650 bytes, verified inside the lease before any arm ran. + +**Request.** `examples/vllm-server`, greedy, `max_tokens=8`, prompt +`The capital of France is`, `--block-size 16 --num-blocks 128 --max-model-len +256`, `VT_CPU_QUANT_REPACK=0` on every arm — the same request PREFILLDIV used. +Geometry read off the tap itself: `T=5 E=512 k=10`, 48 MoE calls per prefill +step, so **the MoE call index IS the decoder layer index at step 0**. + +**The tap gate ran on this box and this toolchain**, not only on the development +machine: `TAP GATE OFF rc=0 … assertions: 121 | 121 passed | 0 failed` and +`TAP GATE ON rc=0 … assertions: 172 | 172 passed | 0 failed`. + +**The instrument asserts that it ran.** Every arm printed 960 `moesel` lines, +384 digests and 48 prefill calls, and every arm's last digest reads +`lines=576`, which is the derivable total (48 prefill calls x 5 tokens + 336 +decode calls x 1). The comparator re-derives it and prints +`COUNTED-PROPERTY … agree=True` for both sides of both comparisons. + +**Two independent instruments agree on the MoE block's input, bit for bit.** +This tap's `x` axis reads the same tensor `VT_Q4EXP_LAYER_FP` calls `L00 +mhc.mix`, from a different tree, and the numbers are identical: + +| arm | this tap, `x` at call 0 | [PREFILLDIV](qwen4exp-cuda-prefill-divergence-20260902.md), `L00 mhc.mix` | +|---|---|---| +| CPU-CTRL | 3613.82031 | 3613.82031 | +| CUDA-PROD | 3615.62777 | 3615.62777 | +| CUDA-GDNSEQ | 3613.74301 | 3613.74301 | + +Both `rel` values reproduce too: `4.999e-04` and `2.139e-05`. + +## 2. The selections + +`sel` is an FNV-1a hash over each token's SORTED selected ids, so one string +comparison per layer is the set-equality assertion. Step 0 only. + +| comparison | prefill calls | `sel` mismatched | token-slots | FLIPPED | first flip | +|---|---|---|---|---|---| +| CPU-CTRL vs CUDA-PROD | 48 | 40 | 240 | **78** | call 0 (layer 0), tok 2 | +| CPU-CTRL vs CUDA-GDNSEQ | 48 | 38 | 240 | **75** | **call 1 (layer 1)**, tok 2 | + +**Layer 0 separates the two sources here exactly as it did for the values.** On +the `VT_GDN_CHUNKED=0` arm — the one whose MoE residue #2552 named — layer 0's +selections AGREE on all five tokens, and the first flip is at layer 1. On the +production arm layer 0 already flips, because the chunked Gated DeltaNet has +put 23x more difference into that layer's MoE input (`4.999e-04` against +`2.139e-05`). So the flip threshold at layer 0 sits between those two numbers. + +## 3. Why they flip: the boundary is a TIE + +`ulps` is the number of representable **bf16** values between the largest +REJECTED logit and the smallest SELECTED one, under the sign-magnitude total +order. `ulps = 0` means they are the SAME bf16 value. + +| arm | slots | `ulps`=0 | 1 | 2 | 3 | 4 | 5 | >=6 | tied | <=1 ulp | +|---|---|---|---|---|---|---|---|---|---|---| +| CPU-CTRL | 240 | **79** | 55 | 31 | 32 | 16 | 13 | 14 | **32.9%** | 55.8% | +| CUDA-PROD | 240 | 63 | 62 | 48 | 33 | 9 | 13 | 12 | 26.2% | 52.1% | +| CUDA-GDNSEQ | 240 | 72 | 68 | 41 | 29 | 10 | 9 | 11 | 30.0% | 58.3% | + +The median MoE call's smallest margin is `minulps = 0` on every arm. The first +flip on the GDNSEQ arm reads + +``` +base ids=16,19,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) +other ids=16,48,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) +``` + +— identical bit patterns on both sides of the boundary, on both arms. Expert 19 +and expert 48 hold the same bf16 logit as the tenth-place expert, and which one +is selected is decided by nothing but which index the greedy scan reaches +first. **A discrete selection has bimodal error, and here a third of the +boundaries have no margin at all.** + +That is the whole mechanism, and the arithmetic is the trigger rather than the +cause. `logit` (the router GEMM's own output) differs by only `2.378e-05` at +layer 0 on the GDNSEQ arm — no amplification over its `2.139e-05` input — yet +by layer 5 the block INPUT differs by `1.630e-02`, three orders of magnitude +higher. That jump is flipped experts entering the residual stream, not rounding +compounding. + +## 4. Where the layer-0 residue actually comes from + +With the selections equal, the digest's four `sum|x|` axes decompose the block. +Layer 0, `VT_GDN_CHUNKED=0`, `rel` against the CPU control: + +| axis | what it is | rel | vs the `2.139e-05` input | +|---|---|---|---| +| `x` | the block input | 2.139e-05 | 1.0x | +| `logit` | the bf16 router logits | 2.378e-05 | 1.1x | +| `exp` | the assembled per-slot expert outputs | **1.421e-04** | **6.6x** | +| `shr` | the shared expert | 4.310e-05 | 2.0x | + +and PREFILLDIV's `L00 moe` (the combined block output) reads `7.269e-05`, +between `exp` and `shr` as the weighted combine of the two requires. **The +router GEMM is not the amplifier. The keep-quant grouped expert GEMM is.** + +## 5. Is it a defect? No, and here is the reading + +**The router mirrors vLLM.** Read at vLLM +`cdefd9d4997f00da72dc6245cc60678b50761b7e`, a **FORWARD REFERENCE** 1566 commits +beyond this row's parity pin `5559679229bc961848b121ccdeaa8fa5d79bec98`, which +has no `vllm/models/qwen4_exp/` at all: + +| anchor | what it says | +|---|---| +| `vllm/models/qwen4_exp/nvidia/model.py:160` | `class Qwen4ExpSparseMoeBlock(Qwen3NextSparseMoeBlock)` — it adds a sequence-parallel refusal and `n_shared_experts`, and inherits the router untouched | +| `vllm/model_executor/models/qwen3_next.py:170-176` | `self.gate = ReplicatedLinear(hidden_size, num_experts, bias=False, quant_config=None)` — **no `params_dtype`**, so the gate weight is the resolved model dtype | +| `vllm/model_executor/layers/fused_moe/runner/moe_runner.py:897-902` | `router_logits, _ = self.gate(hidden_states)` — plain `F.linear`, bf16 in, **bf16 out** | +| `vllm/model_executor/layers/fused_moe/router/fused_topk_router.py:26-41` -> `ops.topk_softmax` | the widening to f32 happens INSIDE the top-k kernel, after the GEMM | +| `vllm/model_executor/layers/fused_moe/experts/cpu_moe.py:69` | the CPU arm of the same: `gating_output = gating_output.float()` | + +Ours: `MoeBlock`'s reference path computes `logits = MatmulBf16(...)` into a +**bf16** `[T,E]` buffer and hands it to `vt::MoeRouterTopK`, whose CPU +(`src/vt/cpu/cpu_ops.cpp:2951`) and CUDA (`src/vt/cuda/cuda_moe.cu:64`) kernels +both widen into an f32 softmax and take a greedy top-k with a strict `>` +ascending scan, so the lowest expert index wins an exact tie. Same width, same +widening point, same tie-break. **vLLM routes this model on bf16 logits at 512 +experts too, so upstream sits on the same knife edge.** + +vLLM does own a fp32-capable gate — `GateLinear` +(`vllm/model_executor/layers/fused_moe/router/gate_linear.py:18-33`), a +`PluggableLayer` whose five dispatch tiers include three that emit fp32 logits. +**Qwen3Next does not opt into it**, so none of those tiers is reachable for this +architecture and Tier 5 (`F.linear`) is what runs. That is recorded as a thing +to mirror IF upstream ever flips this model onto `GateLinear`, and it is not a +licence to widen ours first: doing so would move us AWAY from the oracle. + +**The expert GEMM's arm-to-arm difference is a documented re-association, and +the secondary oracle has the same one.** `src/vt/cuda/cuda_quant_dot.cu:2158-2170` +states it outright — "The INTEGER core (sum x.qs*y.qs per 32-block) is +bit-identical; only the per-block float scale sum is reassociated (warp tree vs +CPU sequential)" — and the activation quantizer is a bit-exact port either way +(`cpu_quant_act.cpp:51-81` against `cuda_quant_dot.cu:1174-1195`: ternary amax, +`d = amax/127`, `F32ToF16(d)`, `roundf(x*id)`). The released UD-IQ1_S expert +towers are IQ4_NL / Q5_0, the 32-element Q8_0-activation lane +(`cuda_quant_dot.cu:2528-2534`). llama.cpp @ `237ad9b961f009ae19ac29dbce4cd0c1251f94b3` +splits the same way: `ggml/src/ggml-cuda/mmvq.cu:566-641` accumulates per-lane +partials and finishes with `warp_reduce_sum`, while the CPU `vec_dot` +(`ggml/src/ggml-cpu/arch/x86/quants.c:841-852`) sums sequentially. + +**So neither arm is wrong.** vLLM's own CPU and CUDA backends would disagree on +this model for the same two reasons, and at a 32.9% exact-tie rate the +disagreement is not a small one. #2552 is answered rather than fixed. + +## 6. What this run RETIRES, and what it says about a gate + +**#2552's own hypothesis is half right and the halves are separable.** The issue +asked whether the layer-0 MoE residue is a selection flip. It is **not** — layer +0 on the GDNSEQ arm agrees on every token, and the residue decomposes onto the +expert GEMM. But from layer 1 the flips are real, they are the dominant source +of everything downstream, and they were invisible to every value tap taken on +this row. + +**It also confirms PREFILLDIV's gate conclusion with a mechanism.** That wave +observed that token agreement is not monotone in numerical distance and proposed +no CPU-vs-CUDA token gate. This says why: a third of the routing decisions in +this model carry zero margin, so the token sequence is a function of tie-break +order, not of accuracy. **No CPU-vs-CUDA token-exactness gate can be well posed +for `qwen4_exp` at bf16 routing.** What IS well posed and is now measurable is +the selection-set agreement rate and the tie-rate histogram of §3, which are +properties of one arm and comparable across trees. A gate against **vLLM** stays +the right target and stays OWED: it needs an artifact vLLM can load, and vLLM's +GGUF support is an out-of-tree plugin (`docs/features/quantization/gguf.md`) +while every safetensors arm of this model exceeds the largest fleet box. + +## 7. The raw outputs + +Committed beside this file in +[`qwen4exp-moe-selection-20260902/`](qwen4exp-moe-selection-20260902/), so every +number above is checkable rather than transcribed: `results.txt` (the job's own +`RESULT` lines, including both `TAP GATE` rcs and every arm's counted property), +`cmp-CUDA-PROD.txt` and `cmp-CUDA-GDNSEQ.txt` (the full comparisons, with the +first twenty flips of each phase spelled out), and `digests-.txt` (all 384 +digest lines per arm — the `sel` hash, `minulps` and the four `sum|x|` axes for +every MoE call of every step). The per-token value lines are NOT committed: at +576 per arm they add nothing the digests and the flip lists do not already +carry. + +## 8. What this is NOT + +n = 1. One prompt, one artifact, one box, one repetition, greedy only, UD-IQ1_S +only, `num_reqs = 1`. **NOT a token gate:** no oracle decoded this prompt, and +the CPU arm is a CONTROL, not an oracle — nothing here says which arm is closer +to correct. No speed number: the tap synchronises once per MoE call for the +shared-expert readback, so the wall times are instrumented and compare to +nothing. The "vLLM sits on the same knife edge" conclusion in §5 is a **SOURCE +READ, not a measurement**; no vLLM process was run on this checkpoint. + +**THE DECODE COLUMNS ARE VOID AND ARE PUBLISHED ONLY TO SAY SO.** The comparator +reports 336 of 336 decode token-slots flipped, and that number means nothing: +on this tree the CUDA arm answers +`11751 271 271 271 271 271 0 0` (production) and +`11751 271 271 11751 271 271 271 0` (`VT_GDN_CHUNKED=0`) against the CPU +control's `11751 13 15767 411 2029 11 1092 369`, so from step 1 the two arms are +running different token sequences and a 100% flip rate is a different-INPUT +artifact rather than a numerics result. + +**That degeneracy is itself a finding, and it has a shelf life.** PREFILLDIV +recorded a fluent CUDA arm (`11751 13 15767 411 1928 11 628 567`) — but it +measured a tree that had +[#2550](https://github.com/mudler/vllm.cpp/pull/2550)'s decode fix merged in. +**On `origin/main` at `a99b9c69a`, which is what this run measured, `--device +cuda` does NOT reproduce that**; the fluency depended on a pull request that had +not landed. **#2550 landed at `bb78d1ee8` while this file was being written**, +so the branch carrying this evidence merges it and the sentence above is a +statement about the measured tree, not about `main` today. Nothing here is +re-derived on the post-#2550 tree: the prefill result does not depend on it (step +0 consumes the prompt, not a sampled token), and a decode-phase selection +comparison is now well posed and is left as the cheap next measurement rather +than claimed. diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt new file mode 100644 index 000000000..6e32efbaf --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-GDNSEQ.txt @@ -0,0 +1,80 @@ +=== CPU-CTRL vs CUDA-GDNSEQ === +COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +CALLS common=384 prefill(T=5)=48 decode(T=1)=336 +--- PREFILL --- +COUNT PREFILL calls=48 sel-hash MISMATCH=38 +COUNT PREFILL token-slots=240 FLIPPED=75 +FIRST PREFILL FLIP call=1 tok=2 + base ids=16,19,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) + other ids=16,48,56,58,61,230,250,320,426,493 ulps=0 lo=-4.125(0xc084) hi=-4.125(0xc084) + FLIP call=1 tok=2 base=16,19,56,58,61,230,250,320,426,493 other=16,48,56,58,61,230,250,320,426,493 base_ulps=0 other_ulps=0 + FLIP call=2 tok=2 base=75,83,84,226,354,381,403,408,415,473 other=83,84,226,354,370,381,403,408,415,473 base_ulps=0 other_ulps=1 + FLIP call=3 tok=1 base=26,122,186,253,302,352,377,405,418,476 other=26,122,186,253,302,352,377,405,421,476 base_ulps=0 other_ulps=0 + FLIP call=3 tok=2 base=40,118,153,252,253,302,418,421,432,476 other=118,122,153,252,253,302,418,421,432,476 base_ulps=1 other_ulps=0 + FLIP call=4 tok=1 base=112,133,205,252,273,309,360,419,469,488 other=133,159,205,252,273,309,360,419,469,488 base_ulps=1 other_ulps=3 + FLIP call=4 tok=4 base=41,48,54,63,133,205,252,309,393,469 other=41,48,54,133,205,252,309,360,393,469 base_ulps=0 other_ulps=1 + FLIP call=5 tok=4 base=34,162,220,236,254,421,423,450,460,493 other=34,162,220,236,254,336,366,421,460,493 base_ulps=2 other_ulps=0 + FLIP call=7 tok=2 base=66,97,269,292,303,337,346,439,458,470 other=66,97,269,301,303,337,346,439,458,470 base_ulps=1 other_ulps=2 + FLIP call=7 tok=3 base=66,104,148,225,247,303,346,435,481,510 other=66,104,148,225,303,346,419,435,481,510 base_ulps=0 other_ulps=1 + FLIP call=8 tok=3 base=94,167,224,231,280,289,333,355,382,455 other=94,167,224,231,280,333,355,382,455,462 base_ulps=0 other_ulps=2 + FLIP call=8 tok=4 base=94,167,224,231,275,308,365,414,448,462 other=94,167,224,231,275,308,414,422,448,462 base_ulps=0 other_ulps=3 + FLIP call=9 tok=3 base=17,75,318,356,406,424,453,462,486,509 other=17,52,318,356,406,424,453,462,486,509 base_ulps=0 other_ulps=0 + FLIP call=9 tok=4 base=52,63,75,139,230,238,406,424,462,509 other=17,52,63,75,139,238,406,424,462,509 base_ulps=1 other_ulps=1 + FLIP call=11 tok=1 base=90,108,163,201,238,261,268,350,397,477 other=31,90,108,163,201,261,268,350,397,477 base_ulps=1 other_ulps=1 + FLIP call=13 tok=4 base=148,162,250,273,282,290,296,385,387,488 other=148,162,250,273,282,290,385,387,418,488 base_ulps=5 other_ulps=1 + FLIP call=14 tok=4 base=106,138,160,173,299,400,409,415,460,500 other=106,138,160,173,299,365,400,409,415,460 base_ulps=1 other_ulps=1 + FLIP call=18 tok=4 base=3,92,93,116,139,194,201,226,314,428 other=3,39,92,93,116,139,194,201,226,428 base_ulps=2 other_ulps=4 + FLIP call=19 tok=0 base=31,173,293,375,406,413,434,462,474,500 other=8,31,173,293,375,406,413,434,474,500 base_ulps=1 other_ulps=0 + FLIP call=20 tok=4 base=63,65,73,155,240,248,258,265,323,496 other=63,73,78,155,240,248,258,265,323,496 base_ulps=1 other_ulps=1 + FLIP call=21 tok=4 base=67,98,167,189,195,225,329,412,473,488 other=98,142,167,189,195,225,329,412,473,488 base_ulps=0 other_ulps=1 +MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +AXIS PREFILL x first_nonzero_call=0 worst_rel=1.630e-02 at call=5 +AXIS PREFILL logit first_nonzero_call=0 worst_rel=2.938e-03 at call=44 +AXIS PREFILL exp first_nonzero_call=0 worst_rel=3.892e-02 at call=45 +AXIS PREFILL shr first_nonzero_call=0 worst_rel=5.065e-02 at call=39 +CALL0 PREFILL base x=3613.82031 logit=17079.5703 exp=2540.09374 shr=250.805253 +CALL0 PREFILL other x=3613.74301 logit=17079.1641 exp=2540.45471 shr=250.794444 +CALL0 PREFILL rel x = 2.139e-05 +CALL0 PREFILL rel logit = 2.378e-05 +CALL0 PREFILL rel exp = 1.421e-04 +CALL0 PREFILL rel shr = 4.310e-05 +--- DECODE --- +COUNT DECODE calls=336 sel-hash MISMATCH=336 +COUNT DECODE token-slots=336 FLIPPED=336 +FIRST DECODE FLIP call=48 tok=0 + base ids=91,106,308,366,367,386,434,466,503,505 ulps=2 lo=-4.9375(0xc09e) hi=-5(0xc0a0) + other ids=31,109,168,205,212,240,251,258,294,442 ulps=7 lo=-4.28125(0xc089) hi=-4.5(0xc090) + FLIP call=48 tok=0 base=91,106,308,366,367,386,434,466,503,505 other=31,109,168,205,212,240,251,258,294,442 base_ulps=2 other_ulps=7 + FLIP call=49 tok=0 base=83,113,114,222,266,298,320,357,500,504 other=19,57,95,113,125,320,370,383,389,475 base_ulps=1 other_ulps=2 + FLIP call=50 tok=0 base=29,75,100,163,218,327,446,473,484,487 other=11,32,71,75,137,150,222,280,403,502 base_ulps=2 other_ulps=0 + FLIP call=51 tok=0 base=26,83,143,156,224,227,295,302,366,476 other=82,190,202,205,237,243,253,335,394,476 base_ulps=1 other_ulps=2 + FLIP call=52 tok=0 base=48,78,133,160,205,239,280,309,313,415 other=41,68,163,252,294,346,382,388,410,496 base_ulps=1 other_ulps=10 + FLIP call=53 tok=0 base=6,34,104,216,220,254,338,421,460,476 other=20,68,110,137,163,220,236,326,420,435 base_ulps=2 other_ulps=1 + FLIP call=54 tok=0 base=15,18,92,169,177,228,270,344,403,424 other=66,92,168,220,237,285,326,397,403,409 base_ulps=4 other_ulps=2 + FLIP call=55 tok=0 base=31,69,104,151,225,303,389,435,448,510 other=22,52,194,208,242,325,346,351,458,499 base_ulps=0 other_ulps=1 + FLIP call=56 tok=0 base=81,94,167,183,231,280,355,362,455,462 other=210,224,227,231,244,308,316,422,426,462 base_ulps=7 other_ulps=5 + FLIP call=57 tok=0 base=17,52,159,230,274,281,321,406,463,509 other=71,75,111,139,230,238,318,396,424,452 base_ulps=1 other_ulps=2 + FLIP call=58 tok=0 base=139,194,207,267,296,307,393,399,405,461 other=203,207,225,296,320,322,357,393,433,461 base_ulps=3 other_ulps=0 + FLIP call=59 tok=0 base=30,86,90,201,238,246,261,310,371,500 other=30,69,84,98,108,261,273,397,477,495 base_ulps=0 other_ulps=2 + FLIP call=60 tok=0 base=10,36,139,180,216,281,344,366,381,436 other=67,82,139,180,192,219,392,418,421,505 base_ulps=3 other_ulps=1 + FLIP call=61 tok=0 base=36,92,162,250,259,282,290,292,453,488 other=148,162,250,273,282,381,387,400,463,480 base_ulps=0 other_ulps=0 + FLIP call=62 tok=0 base=2,106,135,181,297,299,300,400,415,501 other=75,77,82,86,94,106,160,365,407,454 base_ulps=4 other_ulps=2 + FLIP call=63 tok=0 base=120,251,317,319,378,432,443,470,490,496 other=12,100,199,283,327,351,432,470,490,501 base_ulps=2 other_ulps=1 + FLIP call=64 tok=0 base=40,141,206,228,270,274,314,350,489,497 other=92,191,228,272,274,277,320,392,480,489 base_ulps=1 other_ulps=1 + FLIP call=65 tok=0 base=3,107,131,152,291,299,351,363,412,498 other=68,84,105,160,172,299,363,407,469,498 base_ulps=2 other_ulps=2 + FLIP call=66 tok=0 base=3,194,201,278,314,350,401,402,418,442 other=11,28,89,101,194,201,264,275,398,475 base_ulps=2 other_ulps=2 + FLIP call=67 tok=0 base=25,31,45,81,84,140,202,328,343,434 other=30,64,84,225,246,263,364,403,434,454 base_ulps=0 other_ulps=2 +MIN DECODE base minulps over calls: min=0 p10=0 median=1 +AXIS DECODE x first_nonzero_call=48 worst_rel=3.448e-01 at call=149 +AXIS DECODE logit first_nonzero_call=48 worst_rel=8.444e-02 at call=55 +AXIS DECODE exp first_nonzero_call=48 worst_rel=4.142e-01 at call=375 +AXIS DECODE shr first_nonzero_call=48 worst_rel=6.083e-01 at call=243 +CALL0 DECODE base x=711.202027 logit=3398.40625 exp=412.329928 shr=43.5615003 +CALL0 DECODE other x=723.470177 logit=3429.89062 exp=563.539422 shr=31.5357613 +CALL0 DECODE rel x = 1.696e-02 +CALL0 DECODE rel logit = 9.179e-03 +CALL0 DECODE rel exp = 2.683e-01 +CALL0 DECODE rel shr = 2.761e-01 +VERDICT prefill expert-selection SET EQUALITY: DIFFERS (75 flipped token-slots) diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt new file mode 100644 index 000000000..3c73b84b4 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/cmp-CUDA-PROD.txt @@ -0,0 +1,80 @@ +=== CPU-CTRL vs CUDA-PROD === +COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +CALLS common=384 prefill(T=5)=48 decode(T=1)=336 +--- PREFILL --- +COUNT PREFILL calls=48 sel-hash MISMATCH=40 +COUNT PREFILL token-slots=240 FLIPPED=78 +FIRST PREFILL FLIP call=0 tok=2 + base ids=21,64,76,101,265,293,301,314,338,361 ulps=0 lo=-5(0xc0a0) hi=-5(0xc0a0) + other ids=21,76,101,265,293,301,314,338,361,369 ulps=1 lo=-5(0xc0a0) hi=-5.03125(0xc0a1) + FLIP call=0 tok=2 base=21,64,76,101,265,293,301,314,338,361 other=21,76,101,265,293,301,314,338,361,369 base_ulps=0 other_ulps=1 + FLIP call=2 tok=3 base=29,75,83,100,163,329,370,446,473,487 other=29,75,83,100,163,218,329,446,473,487 base_ulps=0 other_ulps=0 + FLIP call=3 tok=1 base=26,122,186,253,302,352,377,405,418,476 other=26,122,186,253,302,352,377,405,421,476 base_ulps=0 other_ulps=1 + FLIP call=4 tok=1 base=112,133,205,252,273,309,360,419,469,488 other=133,159,205,252,273,309,360,419,469,488 base_ulps=1 other_ulps=2 + FLIP call=4 tok=4 base=41,48,54,63,133,205,252,309,393,469 other=41,48,54,133,205,252,309,360,393,469 base_ulps=0 other_ulps=1 + FLIP call=5 tok=4 base=34,162,220,236,254,421,423,450,460,493 other=34,162,220,236,254,278,336,421,460,493 base_ulps=2 other_ulps=0 + FLIP call=6 tok=4 base=44,86,92,135,145,168,177,305,377,501 other=44,86,92,124,135,145,168,305,377,501 base_ulps=1 other_ulps=0 + FLIP call=7 tok=2 base=66,97,269,292,303,337,346,439,458,470 other=66,97,269,283,292,303,337,346,439,470 base_ulps=1 other_ulps=0 + FLIP call=7 tok=4 base=20,66,72,93,221,303,337,374,435,458 other=20,66,72,93,97,303,337,374,435,458 base_ulps=3 other_ulps=0 + FLIP call=8 tok=4 base=94,167,224,231,275,308,365,414,448,462 other=94,167,224,231,275,308,414,422,448,462 base_ulps=0 other_ulps=2 + FLIP call=9 tok=3 base=17,75,318,356,406,424,453,462,486,509 other=17,318,326,356,406,424,453,462,486,509 base_ulps=0 other_ulps=1 + FLIP call=9 tok=4 base=52,63,75,139,230,238,406,424,462,509 other=17,52,63,75,139,238,406,424,462,509 base_ulps=1 other_ulps=1 + FLIP call=11 tok=0 base=4,104,108,130,132,180,361,393,430,464 other=4,104,108,128,130,132,393,400,430,464 base_ulps=0 other_ulps=0 + FLIP call=11 tok=1 base=90,108,163,201,238,261,268,350,397,477 other=90,108,163,201,261,268,293,350,397,477 base_ulps=1 other_ulps=1 + FLIP call=12 tok=3 base=10,90,139,180,257,281,302,433,436,489 other=10,90,139,180,257,281,302,344,433,436 base_ulps=1 other_ulps=0 + FLIP call=13 tok=4 base=148,162,250,273,282,290,296,385,387,488 other=148,250,273,282,290,296,385,387,418,488 base_ulps=5 other_ulps=2 + FLIP call=16 tok=1 base=92,141,150,206,228,307,392,429,487,497 other=92,141,206,228,270,307,392,429,487,497 base_ulps=0 other_ulps=1 + FLIP call=16 tok=3 base=206,228,274,314,350,392,487,489,497,504 other=40,206,228,274,314,350,487,489,497,504 base_ulps=2 other_ulps=0 + FLIP call=18 tok=4 base=3,92,93,116,139,194,201,226,314,428 other=3,39,92,93,116,139,194,201,226,428 base_ulps=2 other_ulps=6 + FLIP call=19 tok=3 base=30,58,64,111,146,147,202,343,394,434 other=30,64,111,146,147,202,343,394,426,434 base_ulps=0 other_ulps=2 +MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +AXIS PREFILL x first_nonzero_call=0 worst_rel=1.121e-02 at call=5 +AXIS PREFILL logit first_nonzero_call=0 worst_rel=2.882e-03 at call=32 +AXIS PREFILL exp first_nonzero_call=0 worst_rel=3.878e-02 at call=45 +AXIS PREFILL shr first_nonzero_call=0 worst_rel=3.095e-02 at call=39 +CALL0 PREFILL base x=3613.82031 logit=17079.5703 exp=2540.09374 shr=250.805253 +CALL0 PREFILL other x=3615.62777 logit=17083.2891 exp=2527.65707 shr=250.743384 +CALL0 PREFILL rel x = 4.999e-04 +CALL0 PREFILL rel logit = 2.177e-04 +CALL0 PREFILL rel exp = 4.896e-03 +CALL0 PREFILL rel shr = 2.467e-04 +--- DECODE --- +COUNT DECODE calls=336 sel-hash MISMATCH=336 +COUNT DECODE token-slots=336 FLIPPED=336 +FIRST DECODE FLIP call=48 tok=0 + base ids=91,106,308,366,367,386,434,466,503,505 ulps=2 lo=-4.9375(0xc09e) hi=-5(0xc0a0) + other ids=31,109,168,205,212,240,251,258,294,442 ulps=7 lo=-4.28125(0xc089) hi=-4.5(0xc090) + FLIP call=48 tok=0 base=91,106,308,366,367,386,434,466,503,505 other=31,109,168,205,212,240,251,258,294,442 base_ulps=2 other_ulps=7 + FLIP call=49 tok=0 base=83,113,114,222,266,298,320,357,500,504 other=19,57,95,113,125,320,370,383,389,475 base_ulps=1 other_ulps=2 + FLIP call=50 tok=0 base=29,75,100,163,218,327,446,473,484,487 other=11,32,71,75,137,203,222,280,403,502 base_ulps=2 other_ulps=1 + FLIP call=51 tok=0 base=26,83,143,156,224,227,295,302,366,476 other=82,190,202,205,237,243,253,335,394,476 base_ulps=1 other_ulps=0 + FLIP call=52 tok=0 base=48,78,133,160,205,239,280,309,313,415 other=41,68,163,252,294,346,382,388,410,496 base_ulps=1 other_ulps=4 + FLIP call=53 tok=0 base=6,34,104,216,220,254,338,421,460,476 other=20,68,110,137,163,220,236,326,420,435 base_ulps=2 other_ulps=0 + FLIP call=54 tok=0 base=15,18,92,169,177,228,270,344,403,424 other=66,168,220,237,285,326,397,403,409,498 base_ulps=4 other_ulps=0 + FLIP call=55 tok=0 base=31,69,104,151,225,303,389,435,448,510 other=52,129,194,208,302,325,346,351,458,499 base_ulps=0 other_ulps=0 + FLIP call=56 tok=0 base=81,94,167,183,231,280,355,362,455,462 other=210,224,227,231,244,308,316,422,426,462 base_ulps=7 other_ulps=10 + FLIP call=57 tok=0 base=17,52,159,230,274,281,321,406,463,509 other=71,75,111,139,238,318,355,396,424,452 base_ulps=1 other_ulps=1 + FLIP call=58 tok=0 base=139,194,207,267,296,307,393,399,405,461 other=203,225,296,320,322,357,393,417,433,461 base_ulps=3 other_ulps=1 + FLIP call=59 tok=0 base=30,86,90,201,238,246,261,310,371,500 other=30,35,69,84,98,108,261,273,397,477 base_ulps=0 other_ulps=1 + FLIP call=60 tok=0 base=10,36,139,180,216,281,344,366,381,436 other=67,82,139,164,180,192,392,418,421,505 base_ulps=3 other_ulps=1 + FLIP call=61 tok=0 base=36,92,162,250,259,282,290,292,453,488 other=148,162,273,282,330,381,387,400,463,480 base_ulps=0 other_ulps=1 + FLIP call=62 tok=0 base=2,106,135,181,297,299,300,400,415,501 other=75,77,82,86,94,106,160,365,407,454 base_ulps=4 other_ulps=1 + FLIP call=63 tok=0 base=120,251,317,319,378,432,443,470,490,496 other=6,12,199,283,327,351,432,470,490,501 base_ulps=2 other_ulps=3 + FLIP call=64 tok=0 base=40,141,206,228,270,274,314,350,489,497 other=92,191,228,272,274,277,350,392,480,489 base_ulps=1 other_ulps=3 + FLIP call=65 tok=0 base=3,107,131,152,291,299,351,363,412,498 other=68,84,105,160,172,221,299,363,454,469 base_ulps=2 other_ulps=1 + FLIP call=66 tok=0 base=3,194,201,278,314,350,401,402,418,442 other=11,28,89,101,194,201,264,275,287,475 base_ulps=2 other_ulps=0 + FLIP call=67 tok=0 base=25,31,45,81,84,140,202,328,343,434 other=30,64,84,225,246,249,263,364,434,454 base_ulps=0 other_ulps=0 +MIN DECODE base minulps over calls: min=0 p10=0 median=1 +AXIS DECODE x first_nonzero_call=48 worst_rel=3.418e-01 at call=149 +AXIS DECODE logit first_nonzero_call=48 worst_rel=8.848e-02 at call=55 +AXIS DECODE exp first_nonzero_call=48 worst_rel=4.084e-01 at call=372 +AXIS DECODE shr first_nonzero_call=48 worst_rel=6.339e-01 at call=183 +CALL0 DECODE base x=711.202027 logit=3398.40625 exp=412.329928 shr=43.5615003 +CALL0 DECODE other x=723.664909 logit=3430.46875 exp=563.497883 shr=31.5413151 +CALL0 DECODE rel x = 1.722e-02 +CALL0 DECODE rel logit = 9.346e-03 +CALL0 DECODE rel exp = 2.683e-01 +CALL0 DECODE rel shr = 2.759e-01 +VERDICT prefill expert-selection SET EQUALITY: DIFFERS (78 flipped token-slots) diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt new file mode 100644 index 000000000..dfc59069d --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CPU-CTRL.txt @@ -0,0 +1,384 @@ +moesel call=0 dev=0 T=5 E=512 k=10 sel=f06a56dd5ffbf75d minulps=0 mintok=2 x=3613.82031 logit=17079.5703 exp=2540.09374 shr=250.805253 lines=5 END +moesel call=1 dev=0 T=5 E=512 k=10 sel=993cb7ac2f9ebaf3 minulps=0 mintok=2 x=3778.01525 logit=15559.4219 exp=2033.00741 shr=34.0071162 lines=10 END +moesel call=2 dev=0 T=5 E=512 k=10 sel=06f1af8ff5cd8f17 minulps=0 mintok=2 x=3839.48339 logit=15776.5469 exp=1795.80639 shr=51.7584638 lines=15 END +moesel call=3 dev=0 T=5 E=512 k=10 sel=d6bea8c3978b8bf0 minulps=0 mintok=1 x=4867.46251 logit=15097 exp=2573.91393 shr=37.8174276 lines=20 END +moesel call=4 dev=0 T=5 E=512 k=10 sel=379aaf5199791a68 minulps=0 mintok=4 x=5361.50879 logit=15455.75 exp=3039.10715 shr=47.7793482 lines=25 END +moesel call=5 dev=0 T=5 E=512 k=10 sel=c3f74a3051738847 minulps=0 mintok=1 x=4507.43522 logit=15666.25 exp=2064.90387 shr=34.3856177 lines=30 END +moesel call=6 dev=0 T=5 E=512 k=10 sel=2ba8099d8f428dab minulps=0 mintok=0 x=5471.63166 logit=15897.3125 exp=3109.92718 shr=46.9859591 lines=35 END +moesel call=7 dev=0 T=5 E=512 k=10 sel=b5644706ca226ea6 minulps=0 mintok=3 x=5342.22944 logit=15933.3281 exp=2723.02001 shr=44.1138059 lines=40 END +moesel call=8 dev=0 T=5 E=512 k=10 sel=e961aa0c5c947daa minulps=0 mintok=3 x=5381.78599 logit=15765.5 exp=3050.63449 shr=28.1738703 lines=45 END +moesel call=9 dev=0 T=5 E=512 k=10 sel=3059c42228399c4d minulps=0 mintok=0 x=5517.82366 logit=15659.5625 exp=2961.69517 shr=35.0320792 lines=50 END +moesel call=10 dev=0 T=5 E=512 k=10 sel=b14a7db9158b4e90 minulps=0 mintok=0 x=4960.85244 logit=15764.1406 exp=2429.17253 shr=27.1544901 lines=55 END +moesel call=11 dev=0 T=5 E=512 k=10 sel=e19210b1d108db0d minulps=0 mintok=0 x=4866.6894 logit=15596.3594 exp=2163.10668 shr=18.8600467 lines=60 END +moesel call=12 dev=0 T=5 E=512 k=10 sel=98f4025615644ee6 minulps=0 mintok=0 x=4971.42549 logit=15651.6406 exp=2259.01844 shr=26.7149538 lines=65 END +moesel call=13 dev=0 T=5 E=512 k=10 sel=d570a4cab1b7404e minulps=0 mintok=0 x=4644.05666 logit=15960.7734 exp=2035.5708 shr=27.478671 lines=70 END +moesel call=14 dev=0 T=5 E=512 k=10 sel=cec11c8b5014de56 minulps=1 mintok=4 x=5252.21439 logit=15834.5469 exp=2900.69116 shr=23.0321477 lines=75 END +moesel call=15 dev=0 T=5 E=512 k=10 sel=88cb94c69381a618 minulps=0 mintok=0 x=6735.20952 logit=15805.5938 exp=3958.20042 shr=29.6381709 lines=80 END +moesel call=16 dev=0 T=5 E=512 k=10 sel=a8002b1c0688c414 minulps=0 mintok=0 x=6149.08192 logit=15762.4844 exp=3181.60388 shr=28.2585843 lines=85 END +moesel call=17 dev=0 T=5 E=512 k=10 sel=cb0c0f7107d1b9f1 minulps=0 mintok=3 x=5359.06631 logit=15997.1562 exp=2583.30954 shr=23.7984908 lines=90 END +moesel call=18 dev=0 T=5 E=512 k=10 sel=541e04fa44573199 minulps=0 mintok=3 x=5321.07004 logit=16327.375 exp=2843.64529 shr=25.2339526 lines=95 END +moesel call=19 dev=0 T=5 E=512 k=10 sel=44be7bbc47cce29b minulps=0 mintok=3 x=5859.39894 logit=16217.5781 exp=2830.69161 shr=28.501454 lines=100 END +moesel call=20 dev=0 T=5 E=512 k=10 sel=0bcc4a933f992ede minulps=0 mintok=1 x=7145.87483 logit=16134.8125 exp=4328.92883 shr=43.0588427 lines=105 END +moesel call=21 dev=0 T=5 E=512 k=10 sel=7e056c1fcbffca34 minulps=0 mintok=0 x=6743.71262 logit=16218.3438 exp=3792.85942 shr=28.5697935 lines=110 END +moesel call=22 dev=0 T=5 E=512 k=10 sel=b3c62ea616442afa minulps=0 mintok=0 x=6487.36654 logit=16644.1406 exp=3624.94315 shr=35.2794219 lines=115 END +moesel call=23 dev=0 T=5 E=512 k=10 sel=0099e3fd59b42d18 minulps=0 mintok=0 x=7552.71469 logit=16095.1641 exp=4454.02452 shr=62.8391198 lines=120 END +moesel call=24 dev=0 T=5 E=512 k=10 sel=56a20f2fdec4ad10 minulps=2 mintok=0 x=5604.4169 logit=16318.8047 exp=2811.55343 shr=27.7470869 lines=125 END +moesel call=25 dev=0 T=5 E=512 k=10 sel=a5741eb95769f3e9 minulps=0 mintok=0 x=5636.99146 logit=16029.4375 exp=2677.11546 shr=28.7961929 lines=130 END +moesel call=26 dev=0 T=5 E=512 k=10 sel=35fd18b3f1d9f02a minulps=0 mintok=1 x=5239.42767 logit=16432.3125 exp=2448.74909 shr=22.5297654 lines=135 END +moesel call=27 dev=0 T=5 E=512 k=10 sel=b21b36340b39c152 minulps=0 mintok=1 x=5248.75765 logit=15959.7188 exp=2343.03231 shr=19.0916266 lines=140 END +moesel call=28 dev=0 T=5 E=512 k=10 sel=c0a026f799fbbd91 minulps=0 mintok=1 x=5273.6478 logit=16085.125 exp=2355.23044 shr=23.3756012 lines=145 END +moesel call=29 dev=0 T=5 E=512 k=10 sel=86319e59463c6c8c minulps=0 mintok=2 x=5503.20621 logit=16543.1953 exp=2749.27634 shr=24.8984516 lines=150 END +moesel call=30 dev=0 T=5 E=512 k=10 sel=da88b0bcd291a413 minulps=0 mintok=2 x=6399.91875 logit=16652.0312 exp=3776.82338 shr=32.4318897 lines=155 END +moesel call=31 dev=0 T=5 E=512 k=10 sel=475532a2b7f1fa69 minulps=0 mintok=1 x=8029.09873 logit=16363.8906 exp=4881.21871 shr=30.7941756 lines=160 END +moesel call=32 dev=0 T=5 E=512 k=10 sel=896d19b81f7e140e minulps=0 mintok=2 x=7358.70547 logit=16027.4609 exp=4366.77026 shr=32.0027352 lines=165 END +moesel call=33 dev=0 T=5 E=512 k=10 sel=6123bb6dff041fa0 minulps=0 mintok=0 x=6689.05148 logit=16679.5156 exp=3787.63997 shr=24.8722188 lines=170 END +moesel call=34 dev=0 T=5 E=512 k=10 sel=3188ef46a7207960 minulps=0 mintok=0 x=6143.31237 logit=16650.6406 exp=3836.00321 shr=34.0574557 lines=175 END +moesel call=35 dev=0 T=5 E=512 k=10 sel=b7e7492a70e7a207 minulps=0 mintok=1 x=7101.2012 logit=16468.8672 exp=4593.54665 shr=33.2344885 lines=180 END +moesel call=36 dev=0 T=5 E=512 k=10 sel=a6674c18f57e7d51 minulps=1 mintok=1 x=8783.09175 logit=16366.5 exp=8155.40826 shr=86.6171582 lines=185 END +moesel call=37 dev=0 T=5 E=512 k=10 sel=abf946cfa9894165 minulps=0 mintok=0 x=8288.97113 logit=16359.5 exp=6937.48806 shr=64.1228779 lines=190 END +moesel call=38 dev=0 T=5 E=512 k=10 sel=bd67adafe51584aa minulps=0 mintok=0 x=8446.94615 logit=16809.3906 exp=9107.50015 shr=108.932895 lines=195 END +moesel call=39 dev=0 T=5 E=512 k=10 sel=b9dc93801db102e5 minulps=0 mintok=0 x=8977.31795 logit=15670.2188 exp=8280.09688 shr=91.7971096 lines=200 END +moesel call=40 dev=0 T=5 E=512 k=10 sel=6d998f5738adf056 minulps=0 mintok=0 x=9499.97248 logit=15696.6094 exp=7250.69534 shr=94.329976 lines=205 END +moesel call=41 dev=0 T=5 E=512 k=10 sel=69d1d0a19457831d minulps=0 mintok=1 x=9620.84339 logit=16114.9375 exp=6914.52953 shr=100.657755 lines=210 END +moesel call=42 dev=0 T=5 E=512 k=10 sel=d6026c35ae47baa9 minulps=0 mintok=0 x=9800.45345 logit=16359.5938 exp=9362.44777 shr=129.539982 lines=215 END +moesel call=43 dev=0 T=5 E=512 k=10 sel=2c74865957babfa5 minulps=0 mintok=3 x=8213.96481 logit=15785.7031 exp=5864.67043 shr=89.8102634 lines=220 END +moesel call=44 dev=0 T=5 E=512 k=10 sel=3a0a55d78229bd21 minulps=0 mintok=1 x=9224.02147 logit=15879.1875 exp=7461.2741 shr=105.141125 lines=225 END +moesel call=45 dev=0 T=5 E=512 k=10 sel=cb02ec54ac780386 minulps=0 mintok=0 x=10381.2064 logit=16223.2344 exp=7515.08854 shr=146.931254 lines=230 END +moesel call=46 dev=0 T=5 E=512 k=10 sel=a05f9c4c058d7890 minulps=0 mintok=2 x=10104.3063 logit=16246 exp=8526.87531 shr=158.675702 lines=235 END +moesel call=47 dev=0 T=5 E=512 k=10 sel=73ac3458f62c111f minulps=0 mintok=3 x=9583.96682 logit=16356.6875 exp=9063.56577 shr=825.793385 lines=240 END +moesel call=48 dev=0 T=1 E=512 k=10 sel=aabd9229c965eb93 minulps=2 mintok=0 x=711.202027 logit=3398.40625 exp=412.329928 shr=43.5615003 lines=241 END +moesel call=49 dev=0 T=1 E=512 k=10 sel=7ef1f3cf7d4f2804 minulps=1 mintok=0 x=792.780496 logit=3110 exp=443.90434 shr=8.94934824 lines=242 END +moesel call=50 dev=0 T=1 E=512 k=10 sel=6c6615cd7c923744 minulps=2 mintok=0 x=796.914679 logit=3283.23438 exp=325.523548 shr=17.2280005 lines=243 END +moesel call=51 dev=0 T=1 E=512 k=10 sel=d2d4fb3fb5f1410d minulps=1 mintok=0 x=1045.87296 logit=3109.07812 exp=593.517361 shr=8.9673944 lines=244 END +moesel call=52 dev=0 T=1 E=512 k=10 sel=86fa045227ed7bd5 minulps=1 mintok=0 x=1060.38456 logit=3132.9375 exp=599.172895 shr=11.508544 lines=245 END +moesel call=53 dev=0 T=1 E=512 k=10 sel=a0b3b88800b67d76 minulps=2 mintok=0 x=761.397766 logit=3059.70312 exp=302.961598 shr=5.06307073 lines=246 END +moesel call=54 dev=0 T=1 E=512 k=10 sel=334effe309200143 minulps=4 mintok=0 x=1079.98165 logit=3158.14062 exp=643.891552 shr=8.80490369 lines=247 END +moesel call=55 dev=0 T=1 E=512 k=10 sel=84252597ab352ed7 minulps=0 mintok=0 x=1068.59662 logit=3243.07812 exp=614.197879 shr=12.7280557 lines=248 END +moesel call=56 dev=0 T=1 E=512 k=10 sel=c1d7aa426974b328 minulps=7 mintok=0 x=1058.49208 logit=3085.25 exp=571.210994 shr=6.95671796 lines=249 END +moesel call=57 dev=0 T=1 E=512 k=10 sel=36b57d61ee92457d minulps=1 mintok=0 x=1048.09664 logit=3101.10938 exp=487.261373 shr=4.99845667 lines=250 END +moesel call=58 dev=0 T=1 E=512 k=10 sel=5a66fc9d4b643f28 minulps=3 mintok=0 x=992.641977 logit=3070.71875 exp=495.778773 shr=5.26890968 lines=251 END +moesel call=59 dev=0 T=1 E=512 k=10 sel=325cd7cf26f72b80 minulps=0 mintok=0 x=1016.83878 logit=3050.8125 exp=488.083284 shr=4.35687687 lines=252 END +moesel call=60 dev=0 T=1 E=512 k=10 sel=8b2e8b711474c593 minulps=3 mintok=0 x=1027.50875 logit=3144.85938 exp=463.934765 shr=5.94254702 lines=253 END +moesel call=61 dev=0 T=1 E=512 k=10 sel=afc0ffcf0383f64d minulps=0 mintok=0 x=942.910339 logit=3106.48438 exp=439.91314 shr=5.42763743 lines=254 END +moesel call=62 dev=0 T=1 E=512 k=10 sel=8efcec71a42408c1 minulps=4 mintok=0 x=1083.31629 logit=3125.53125 exp=560.578572 shr=4.22694099 lines=255 END +moesel call=63 dev=0 T=1 E=512 k=10 sel=4c88cd86fff12233 minulps=2 mintok=0 x=1561.24098 logit=3174.85938 exp=974.06361 shr=7.47097771 lines=256 END +moesel call=64 dev=0 T=1 E=512 k=10 sel=e337f23a62631c60 minulps=1 mintok=0 x=1327.90812 logit=3092.71875 exp=655.407484 shr=7.19295359 lines=257 END +moesel call=65 dev=0 T=1 E=512 k=10 sel=c2a57becde66269e minulps=2 mintok=0 x=1127.2688 logit=3065.51562 exp=518.199874 shr=6.03081573 lines=258 END +moesel call=66 dev=0 T=1 E=512 k=10 sel=34eab51d3a21e181 minulps=2 mintok=0 x=1084.93185 logit=3244.125 exp=571.740371 shr=6.15451421 lines=259 END +moesel call=67 dev=0 T=1 E=512 k=10 sel=8bae9070b56a03f1 minulps=0 mintok=0 x=1197.07237 logit=3153.26562 exp=557.419123 shr=5.07023441 lines=260 END +moesel call=68 dev=0 T=1 E=512 k=10 sel=d0797c4e5cc68d4d minulps=1 mintok=0 x=1449.0214 logit=3137.42188 exp=828.382667 shr=8.49615016 lines=261 END +moesel call=69 dev=0 T=1 E=512 k=10 sel=aae6e24f36be3df4 minulps=3 mintok=0 x=1294.88361 logit=3147.82812 exp=717.240796 shr=5.74877287 lines=262 END +moesel call=70 dev=0 T=1 E=512 k=10 sel=b4ae2ec987af98e8 minulps=1 mintok=0 x=1249.63897 logit=3281.98438 exp=638.724468 shr=7.53334141 lines=263 END +moesel call=71 dev=0 T=1 E=512 k=10 sel=6b54745e8564670b minulps=1 mintok=0 x=1630.56089 logit=3178.07812 exp=954.66822 shr=12.8149843 lines=264 END +moesel call=72 dev=0 T=1 E=512 k=10 sel=d578ce2c70dda8a2 minulps=1 mintok=0 x=1121.67649 logit=3185.1875 exp=533.201703 shr=7.53353781 lines=265 END +moesel call=73 dev=0 T=1 E=512 k=10 sel=b967a85589fd4045 minulps=4 mintok=0 x=1134.39417 logit=3123.57812 exp=520.904717 shr=7.10389788 lines=266 END +moesel call=74 dev=0 T=1 E=512 k=10 sel=29fcdce5f80bc6fb minulps=2 mintok=0 x=1009.4706 logit=3177.17188 exp=460.661269 shr=4.92624117 lines=267 END +moesel call=75 dev=0 T=1 E=512 k=10 sel=8d915ffd5d146bc2 minulps=1 mintok=0 x=1093.21958 logit=3118.75 exp=519.55021 shr=4.85974144 lines=268 END +moesel call=76 dev=0 T=1 E=512 k=10 sel=e3fb5328b5bcbf88 minulps=3 mintok=0 x=1069.93763 logit=3186.10938 exp=476.545881 shr=7.33836451 lines=269 END +moesel call=77 dev=0 T=1 E=512 k=10 sel=18f42552e2caa5c6 minulps=0 mintok=0 x=1124.13273 logit=3191.39062 exp=574.651633 shr=7.35481319 lines=270 END +moesel call=78 dev=0 T=1 E=512 k=10 sel=870bf2b9e0039a8f minulps=5 mintok=0 x=1300.92135 logit=3295.96875 exp=673.63485 shr=8.77113053 lines=271 END +moesel call=79 dev=0 T=1 E=512 k=10 sel=104c9bab786b8034 minulps=0 mintok=0 x=1615.60873 logit=3189.5625 exp=1040.70751 shr=9.76467319 lines=272 END +moesel call=80 dev=0 T=1 E=512 k=10 sel=f3c5730de2391c7a minulps=3 mintok=0 x=1540.40412 logit=3144.60938 exp=858.872363 shr=11.5954378 lines=273 END +moesel call=81 dev=0 T=1 E=512 k=10 sel=e1acf7c505479e63 minulps=5 mintok=0 x=1351.78171 logit=3187.21875 exp=796.809841 shr=8.571166 lines=274 END +moesel call=82 dev=0 T=1 E=512 k=10 sel=69e138de258ce2b4 minulps=0 mintok=0 x=1238.17522 logit=3358.51562 exp=864.889995 shr=7.35813867 lines=275 END +moesel call=83 dev=0 T=1 E=512 k=10 sel=dc8dae3dcd3b07fe minulps=0 mintok=0 x=1430.11096 logit=3220.23438 exp=965.800933 shr=10.1994688 lines=276 END +moesel call=84 dev=0 T=1 E=512 k=10 sel=7e4bdcc479f2fdc8 minulps=2 mintok=0 x=1781.60821 logit=3198 exp=1843.95605 shr=22.3955349 lines=277 END +moesel call=85 dev=0 T=1 E=512 k=10 sel=a0335a14c82fc482 minulps=5 mintok=0 x=1654.70413 logit=3197.04688 exp=1433.99453 shr=14.6948673 lines=278 END +moesel call=86 dev=0 T=1 E=512 k=10 sel=e49bb2f60e4e5bcd minulps=2 mintok=0 x=1721.5401 logit=3309.60938 exp=1631.11502 shr=19.8436993 lines=279 END +moesel call=87 dev=0 T=1 E=512 k=10 sel=1cb0178ca0c9d16b minulps=0 mintok=0 x=1897.42815 logit=3090.01562 exp=2055.54628 shr=26.1610985 lines=280 END +moesel call=88 dev=0 T=1 E=512 k=10 sel=7fe288e3bf1ddb91 minulps=0 mintok=0 x=1875.99385 logit=3063.79688 exp=1450.68229 shr=20.1768826 lines=281 END +moesel call=89 dev=0 T=1 E=512 k=10 sel=8815269d7fe8cfd1 minulps=5 mintok=0 x=2016.26 logit=3151.4375 exp=1488.89249 shr=18.9155179 lines=282 END +moesel call=90 dev=0 T=1 E=512 k=10 sel=c6e629caff9850fa minulps=0 mintok=0 x=1955.149 logit=3205.03125 exp=2227.8235 shr=25.1312252 lines=283 END +moesel call=91 dev=0 T=1 E=512 k=10 sel=c64704829f793e80 minulps=1 mintok=0 x=1922.13663 logit=3174.67188 exp=1526.55625 shr=19.1960543 lines=284 END +moesel call=92 dev=0 T=1 E=512 k=10 sel=387df9e5ab2ea36e minulps=3 mintok=0 x=2144.15098 logit=3208.09375 exp=1932.42218 shr=20.5665565 lines=285 END +moesel call=93 dev=0 T=1 E=512 k=10 sel=2a1c2563aa83e949 minulps=1 mintok=0 x=2240.82428 logit=3280.60938 exp=1734.29974 shr=24.7781918 lines=286 END +moesel call=94 dev=0 T=1 E=512 k=10 sel=6d131570aad3e56c minulps=4 mintok=0 x=2286.95382 logit=3284.53125 exp=1653.91675 shr=34.6715817 lines=287 END +moesel call=95 dev=0 T=1 E=512 k=10 sel=f9e6ace4226e4cf9 minulps=0 mintok=0 x=2058.42086 logit=3239.64062 exp=1604.6322 shr=169.649549 lines=288 END +moesel call=96 dev=0 T=1 E=512 k=10 sel=45d0e90aa559cadb minulps=4 mintok=0 x=721.959273 logit=3429.64062 exp=673.545117 shr=44.7621707 lines=289 END +moesel call=97 dev=0 T=1 E=512 k=10 sel=17cfb2d884234187 minulps=5 mintok=0 x=744.934237 logit=3130.04688 exp=481.194355 shr=6.59894305 lines=290 END +moesel call=98 dev=0 T=1 E=512 k=10 sel=0f243d9291985c24 minulps=1 mintok=0 x=814.22921 logit=3159.96875 exp=474.694537 shr=7.52003516 lines=291 END +moesel call=99 dev=0 T=1 E=512 k=10 sel=c9b415586c05f8b1 minulps=1 mintok=0 x=941.458203 logit=2922.34375 exp=492.467496 shr=4.55752426 lines=292 END +moesel call=100 dev=0 T=1 E=512 k=10 sel=19a4d45b3eb22e7f minulps=0 mintok=0 x=1023.1575 logit=2987.90625 exp=541.392979 shr=5.34364326 lines=293 END +moesel call=101 dev=0 T=1 E=512 k=10 sel=37da03305c750708 minulps=0 mintok=0 x=787.274973 logit=3032.64062 exp=387.346789 shr=4.08089032 lines=294 END +moesel call=102 dev=0 T=1 E=512 k=10 sel=7900205502e3194e minulps=3 mintok=0 x=998.247449 logit=3107.10938 exp=609.805782 shr=7.90607299 lines=295 END +moesel call=103 dev=0 T=1 E=512 k=10 sel=e70364c224214791 minulps=4 mintok=0 x=1068.05092 logit=3148.67188 exp=619.919969 shr=5.91998706 lines=296 END +moesel call=104 dev=0 T=1 E=512 k=10 sel=f733c95e60ef14fa minulps=0 mintok=0 x=1086.83845 logit=3106.29688 exp=620.237022 shr=5.41683059 lines=297 END +moesel call=105 dev=0 T=1 E=512 k=10 sel=de16fc878cf47c0a minulps=1 mintok=0 x=1088.87786 logit=3028.48438 exp=614.294931 shr=6.21226995 lines=298 END +moesel call=106 dev=0 T=1 E=512 k=10 sel=a882e4f32ccbc35b minulps=1 mintok=0 x=1031.23153 logit=3043.23438 exp=538.238265 shr=4.56079428 lines=299 END +moesel call=107 dev=0 T=1 E=512 k=10 sel=40c41ef8a695e12f minulps=4 mintok=0 x=1047.01792 logit=3047.71875 exp=506.939993 shr=3.99706445 lines=300 END +moesel call=108 dev=0 T=1 E=512 k=10 sel=ea91ccc52f46d0bd minulps=1 mintok=0 x=1087.70763 logit=3026.21875 exp=574.908465 shr=6.1161481 lines=301 END +moesel call=109 dev=0 T=1 E=512 k=10 sel=927b13896742804e minulps=1 mintok=0 x=1028.27619 logit=3039.89062 exp=485.082588 shr=5.54919302 lines=302 END +moesel call=110 dev=0 T=1 E=512 k=10 sel=1f8bd4e68bbe9712 minulps=1 mintok=0 x=1136.35674 logit=3027.96875 exp=557.59843 shr=5.56775392 lines=303 END +moesel call=111 dev=0 T=1 E=512 k=10 sel=acbf1db9c37e7c7e minulps=0 mintok=0 x=1530.08761 logit=3091.75 exp=872.510131 shr=6.87254228 lines=304 END +moesel call=112 dev=0 T=1 E=512 k=10 sel=9e461019ce68f288 minulps=2 mintok=0 x=1387.76517 logit=3103.46875 exp=698.423701 shr=4.94248048 lines=305 END +moesel call=113 dev=0 T=1 E=512 k=10 sel=c6cf25721cad5178 minulps=7 mintok=0 x=1176.06792 logit=3113.38281 exp=585.272764 shr=5.42286371 lines=306 END +moesel call=114 dev=0 T=1 E=512 k=10 sel=7af4174660dc2245 minulps=2 mintok=0 x=1132.73144 logit=3230.92188 exp=598.634267 shr=5.56820858 lines=307 END +moesel call=115 dev=0 T=1 E=512 k=10 sel=2d5aac9c0c37e5e6 minulps=0 mintok=0 x=1243.11194 logit=3106.14062 exp=625.335118 shr=6.19604698 lines=308 END +moesel call=116 dev=0 T=1 E=512 k=10 sel=b9578a923a9de6e4 minulps=1 mintok=0 x=1448.87344 logit=3119.29688 exp=946.334552 shr=9.86572686 lines=309 END +moesel call=117 dev=0 T=1 E=512 k=10 sel=0400ba7759444aa7 minulps=1 mintok=0 x=1422.42938 logit=3240.64062 exp=806.576878 shr=5.32338837 lines=310 END +moesel call=118 dev=0 T=1 E=512 k=10 sel=40f05b496f92669a minulps=2 mintok=0 x=1283.71828 logit=3383.8125 exp=682.072263 shr=5.8046688 lines=311 END +moesel call=119 dev=0 T=1 E=512 k=10 sel=c6ee49a3ef5f223a minulps=0 mintok=0 x=1585.95319 logit=3118.64062 exp=936.153902 shr=19.8668005 lines=312 END +moesel call=120 dev=0 T=1 E=512 k=10 sel=3f2beb660bafa2a8 minulps=1 mintok=0 x=1130.52855 logit=3220.75 exp=550.584942 shr=4.38061896 lines=313 END +moesel call=121 dev=0 T=1 E=512 k=10 sel=c5530d98ef4cad0d minulps=0 mintok=0 x=1148.46329 logit=3063.5 exp=540.042083 shr=5.89698556 lines=314 END +moesel call=122 dev=0 T=1 E=512 k=10 sel=369a394e422779dc minulps=10 mintok=0 x=1113.17404 logit=3146.79688 exp=587.988584 shr=3.85845712 lines=315 END +moesel call=123 dev=0 T=1 E=512 k=10 sel=43f36f076a312e3e minulps=3 mintok=0 x=1167.5815 logit=3111.65625 exp=593.37795 shr=5.10464762 lines=316 END +moesel call=124 dev=0 T=1 E=512 k=10 sel=7b57ad1904886660 minulps=4 mintok=0 x=1190.10042 logit=3186.73438 exp=589.303752 shr=5.49600425 lines=317 END +moesel call=125 dev=0 T=1 E=512 k=10 sel=0e748bdab1d7a80c minulps=1 mintok=0 x=1240.59509 logit=3252.04688 exp=672.454912 shr=5.33884706 lines=318 END +moesel call=126 dev=0 T=1 E=512 k=10 sel=fdce871467d97c16 minulps=2 mintok=0 x=1424.39816 logit=3211.09375 exp=885.052741 shr=6.50303227 lines=319 END +moesel call=127 dev=0 T=1 E=512 k=10 sel=b83c04319a08a51d minulps=1 mintok=0 x=1740.47599 logit=3239.39062 exp=1090.04158 shr=7.10091856 lines=320 END +moesel call=128 dev=0 T=1 E=512 k=10 sel=1bf8f022b2eec312 minulps=3 mintok=0 x=1647.05609 logit=3115.20312 exp=1011.3927 shr=6.60869288 lines=321 END +moesel call=129 dev=0 T=1 E=512 k=10 sel=ed35d736a07f76b2 minulps=2 mintok=0 x=1536.28169 logit=3285.55469 exp=1028.69928 shr=5.20611351 lines=322 END +moesel call=130 dev=0 T=1 E=512 k=10 sel=604f7f5982f7ec3a minulps=1 mintok=0 x=1492.01486 logit=3444.32812 exp=916.953086 shr=5.96958305 lines=323 END +moesel call=131 dev=0 T=1 E=512 k=10 sel=3f92392a733cf89b minulps=1 mintok=0 x=1694.0605 logit=3263.1875 exp=1261.00149 shr=7.00404574 lines=324 END +moesel call=132 dev=0 T=1 E=512 k=10 sel=e47c8261c4d73d43 minulps=1 mintok=0 x=2043.89637 logit=3133.75 exp=2080.5263 shr=23.460304 lines=325 END +moesel call=133 dev=0 T=1 E=512 k=10 sel=32b34a4857d3034a minulps=0 mintok=0 x=1972.11004 logit=3223.15625 exp=1378.16079 shr=10.0461127 lines=326 END +moesel call=134 dev=0 T=1 E=512 k=10 sel=ecfcaac740e8eb3e minulps=0 mintok=0 x=1891.92782 logit=3404.23438 exp=1795.02391 shr=14.3305215 lines=327 END +moesel call=135 dev=0 T=1 E=512 k=10 sel=3defa2c2ad1a19c9 minulps=0 mintok=0 x=2030.99916 logit=3141.89062 exp=1735.56049 shr=17.3080959 lines=328 END +moesel call=136 dev=0 T=1 E=512 k=10 sel=9d4c4d575530aaa1 minulps=2 mintok=0 x=2015.94075 logit=3064.125 exp=1498.87653 shr=14.0980233 lines=329 END +moesel call=137 dev=0 T=1 E=512 k=10 sel=a2ca04f2c32af24c minulps=5 mintok=0 x=2164.04088 logit=3173.60938 exp=1476.40351 shr=13.9556701 lines=330 END +moesel call=138 dev=0 T=1 E=512 k=10 sel=2c1052d79fe8a01c minulps=3 mintok=0 x=2116.30283 logit=3201.82812 exp=1908.28173 shr=19.9749909 lines=331 END +moesel call=139 dev=0 T=1 E=512 k=10 sel=63dfce62a1968997 minulps=8 mintok=0 x=1825.40135 logit=3097.98438 exp=1359.85714 shr=15.7230028 lines=332 END +moesel call=140 dev=0 T=1 E=512 k=10 sel=f3c6269367da1868 minulps=1 mintok=0 x=2168.14714 logit=3176.79688 exp=1596.21876 shr=23.0226604 lines=333 END +moesel call=141 dev=0 T=1 E=512 k=10 sel=acb23a0e28d2ff50 minulps=4 mintok=0 x=2369.13889 logit=3233.3125 exp=1501.22406 shr=26.2855051 lines=334 END +moesel call=142 dev=0 T=1 E=512 k=10 sel=7921d1ce05dc1780 minulps=2 mintok=0 x=2269.42081 logit=3240.53125 exp=1456.38467 shr=38.3288927 lines=335 END +moesel call=143 dev=0 T=1 E=512 k=10 sel=cb5ae746a6e98d0f minulps=2 mintok=0 x=1996.28304 logit=3240.89062 exp=1482.50299 shr=133.604757 lines=336 END +moesel call=144 dev=0 T=1 E=512 k=10 sel=64e4241874f1df00 minulps=2 mintok=0 x=704.012409 logit=3391.01562 exp=554.235773 shr=42.2263086 lines=337 END +moesel call=145 dev=0 T=1 E=512 k=10 sel=6e61ac8029b0eb57 minulps=4 mintok=0 x=757.554578 logit=3137.48438 exp=337.795793 shr=8.5666878 lines=338 END +moesel call=146 dev=0 T=1 E=512 k=10 sel=b08fb8a7f3061f39 minulps=1 mintok=0 x=790.976817 logit=3146.625 exp=386.331875 shr=8.11794358 lines=339 END +moesel call=147 dev=0 T=1 E=512 k=10 sel=cc2de51b46bf37b7 minulps=0 mintok=0 x=1030.9344 logit=3043.8125 exp=505.468475 shr=4.88330396 lines=340 END +moesel call=148 dev=0 T=1 E=512 k=10 sel=bb9732cb39f87a45 minulps=1 mintok=0 x=1026.28395 logit=3019.15625 exp=578.007647 shr=5.46301039 lines=341 END +moesel call=149 dev=0 T=1 E=512 k=10 sel=792156da8816e499 minulps=2 mintok=0 x=709.238798 logit=3039.48438 exp=305.731701 shr=4.0652933 lines=342 END +moesel call=150 dev=0 T=1 E=512 k=10 sel=adfa331c6b9ef442 minulps=2 mintok=0 x=1114.27743 logit=3156.125 exp=638.692997 shr=7.41232707 lines=343 END +moesel call=151 dev=0 T=1 E=512 k=10 sel=6ef63e64b3549c52 minulps=2 mintok=0 x=1072.8511 logit=3133.23438 exp=556.26751 shr=6.92626507 lines=344 END +moesel call=152 dev=0 T=1 E=512 k=10 sel=ea6f7ba943a4bce2 minulps=3 mintok=0 x=1066.75546 logit=3010.625 exp=561.820343 shr=5.50392725 lines=345 END +moesel call=153 dev=0 T=1 E=512 k=10 sel=cc820632974d0d21 minulps=1 mintok=0 x=1078.96722 logit=3054.46875 exp=567.575908 shr=5.9065972 lines=346 END +moesel call=154 dev=0 T=1 E=512 k=10 sel=c3ac69b7b9be9b4d minulps=4 mintok=0 x=1023.90607 logit=3072.53125 exp=473.886277 shr=4.85309371 lines=347 END +moesel call=155 dev=0 T=1 E=512 k=10 sel=2c4587f6d923075b minulps=1 mintok=0 x=993.635571 logit=3007.59375 exp=412.653947 shr=4.33550698 lines=348 END +moesel call=156 dev=0 T=1 E=512 k=10 sel=7368df348dda0d2a minulps=0 mintok=0 x=1067.73772 logit=3005.45312 exp=530.45088 shr=6.58520544 lines=349 END +moesel call=157 dev=0 T=1 E=512 k=10 sel=13f5f4bed552fb78 minulps=1 mintok=0 x=980.486447 logit=3051.84375 exp=474.45902 shr=4.74441737 lines=350 END +moesel call=158 dev=0 T=1 E=512 k=10 sel=415c63615f164733 minulps=0 mintok=0 x=1025.23302 logit=2934.09375 exp=550.762955 shr=5.26887361 lines=351 END +moesel call=159 dev=0 T=1 E=512 k=10 sel=6e558f11c4013434 minulps=2 mintok=0 x=1471.486 logit=3099.71875 exp=864.215507 shr=5.8337274 lines=352 END +moesel call=160 dev=0 T=1 E=512 k=10 sel=64ec13a3c0245f29 minulps=2 mintok=0 x=1317.34324 logit=3044.51562 exp=692.437795 shr=5.36932105 lines=353 END +moesel call=161 dev=0 T=1 E=512 k=10 sel=5f4f3c990d22bb31 minulps=2 mintok=0 x=1098.57551 logit=2963.70312 exp=524.686988 shr=5.79104155 lines=354 END +moesel call=162 dev=0 T=1 E=512 k=10 sel=868a69c6fe12189a minulps=1 mintok=0 x=1154.29364 logit=3161.8125 exp=625.385319 shr=7.06885046 lines=355 END +moesel call=163 dev=0 T=1 E=512 k=10 sel=bc12ac54ecf1f835 minulps=7 mintok=0 x=1273.37386 logit=3102.66406 exp=655.353976 shr=5.84881203 lines=356 END +moesel call=164 dev=0 T=1 E=512 k=10 sel=188ebb0c0705055f minulps=1 mintok=0 x=1614.69366 logit=3177.42188 exp=1057.78862 shr=14.9799776 lines=357 END +moesel call=165 dev=0 T=1 E=512 k=10 sel=80088ed263e76837 minulps=6 mintok=0 x=1467.51619 logit=3145.3125 exp=836.37563 shr=7.55925656 lines=358 END +moesel call=166 dev=0 T=1 E=512 k=10 sel=894f039c5a8b4ba4 minulps=2 mintok=0 x=1403.07147 logit=3254.46875 exp=800.448175 shr=7.98383042 lines=359 END +moesel call=167 dev=0 T=1 E=512 k=10 sel=eb8d8255967fecf4 minulps=4 mintok=0 x=1737.49875 logit=3127.09375 exp=1053.51103 shr=18.5937283 lines=360 END +moesel call=168 dev=0 T=1 E=512 k=10 sel=735f79b421393331 minulps=1 mintok=0 x=1177.85914 logit=3191.78125 exp=571.974418 shr=6.57680889 lines=361 END +moesel call=169 dev=0 T=1 E=512 k=10 sel=e6e2e22937550012 minulps=9 mintok=0 x=1153.27574 logit=3059.17188 exp=541.296287 shr=6.18206786 lines=362 END +moesel call=170 dev=0 T=1 E=512 k=10 sel=4077e415e23aed88 minulps=6 mintok=0 x=1091.12849 logit=3117.5625 exp=492.27847 shr=4.54085315 lines=363 END +moesel call=171 dev=0 T=1 E=512 k=10 sel=52d591d1166fcd66 minulps=11 mintok=0 x=1128.09003 logit=3078.46875 exp=490.031421 shr=4.4073793 lines=364 END +moesel call=172 dev=0 T=1 E=512 k=10 sel=d5b7d59b693addc7 minulps=1 mintok=0 x=1246.31363 logit=3108.90625 exp=628.32774 shr=6.3387332 lines=365 END +moesel call=173 dev=0 T=1 E=512 k=10 sel=a76f28caa9389c5c minulps=4 mintok=0 x=1237.95625 logit=3147.45312 exp=591.657497 shr=6.3675811 lines=366 END +moesel call=174 dev=0 T=1 E=512 k=10 sel=2a650cf963392c95 minulps=1 mintok=0 x=1344.51108 logit=3119.4375 exp=783.550203 shr=8.529039 lines=367 END +moesel call=175 dev=0 T=1 E=512 k=10 sel=67b69a4ebc1103c6 minulps=2 mintok=0 x=1763.59213 logit=3157.35938 exp=1111.8995 shr=7.47240212 lines=368 END +moesel call=176 dev=0 T=1 E=512 k=10 sel=b389ccc0f73e7d0a minulps=5 mintok=0 x=1747.07535 logit=3114.29688 exp=1104.01993 shr=9.85640764 lines=369 END +moesel call=177 dev=0 T=1 E=512 k=10 sel=d653c3f146c745db minulps=2 mintok=0 x=1576.18927 logit=3189.375 exp=1020.53586 shr=6.87306631 lines=370 END +moesel call=178 dev=0 T=1 E=512 k=10 sel=b33be1e81fca8628 minulps=0 mintok=0 x=1482.88848 logit=3291.57812 exp=902.234411 shr=7.58216667 lines=371 END +moesel call=179 dev=0 T=1 E=512 k=10 sel=126affb7f16f1935 minulps=1 mintok=0 x=1768.0564 logit=3204.6875 exp=1384.13685 shr=7.81168029 lines=372 END +moesel call=180 dev=0 T=1 E=512 k=10 sel=a581d13c9085d7bd minulps=1 mintok=0 x=2256.299 logit=3255.92188 exp=2356.2561 shr=22.8342528 lines=373 END +moesel call=181 dev=0 T=1 E=512 k=10 sel=1fc2a0f7cc1d336a minulps=0 mintok=0 x=2140.51995 logit=3245.95312 exp=1706.35723 shr=18.3786964 lines=374 END +moesel call=182 dev=0 T=1 E=512 k=10 sel=ff4fd74bb3dc6518 minulps=0 mintok=0 x=2056.02953 logit=3287.51562 exp=2182.11039 shr=22.1919606 lines=375 END +moesel call=183 dev=0 T=1 E=512 k=10 sel=1378d58df87832ca minulps=6 mintok=0 x=2265.28398 logit=3070.65625 exp=1908.95659 shr=10.7998445 lines=376 END +moesel call=184 dev=0 T=1 E=512 k=10 sel=eaa3ad093aa15a51 minulps=1 mintok=0 x=2321.61115 logit=3115.9375 exp=1922.93207 shr=14.6389322 lines=377 END +moesel call=185 dev=0 T=1 E=512 k=10 sel=e640c6edc8997ed6 minulps=5 mintok=0 x=2479.74604 logit=3166.71875 exp=2057.65835 shr=14.9026581 lines=378 END +moesel call=186 dev=0 T=1 E=512 k=10 sel=5c54b1dbef71949b minulps=1 mintok=0 x=2383.45681 logit=3251.1875 exp=2245.25949 shr=21.1885175 lines=379 END +moesel call=187 dev=0 T=1 E=512 k=10 sel=f949b3520df67388 minulps=0 mintok=0 x=1985.20757 logit=3131.78125 exp=1416.58352 shr=13.3448344 lines=380 END +moesel call=188 dev=0 T=1 E=512 k=10 sel=8ffcb0ae131665b6 minulps=2 mintok=0 x=2426.90604 logit=3164.84375 exp=1845.1169 shr=21.1736459 lines=381 END +moesel call=189 dev=0 T=1 E=512 k=10 sel=4b3c8e6dc667dd7f minulps=2 mintok=0 x=2495.70905 logit=3197.51562 exp=1765.98206 shr=20.5617286 lines=382 END +moesel call=190 dev=0 T=1 E=512 k=10 sel=b0a8ea541f8bd42e minulps=1 mintok=0 x=2445.78047 logit=3187.85938 exp=1627.32222 shr=33.7317171 lines=383 END +moesel call=191 dev=0 T=1 E=512 k=10 sel=b72a0bf0cb2ae7f4 minulps=7 mintok=0 x=2395.1438 logit=3300.82812 exp=1680.61398 shr=89.5329192 lines=384 END +moesel call=192 dev=0 T=1 E=512 k=10 sel=4a58e59eedd05614 minulps=6 mintok=0 x=713.292795 logit=3547.25 exp=495.988081 shr=46.8861551 lines=385 END +moesel call=193 dev=0 T=1 E=512 k=10 sel=3c10167bba9572f3 minulps=5 mintok=0 x=809.619069 logit=3187.96875 exp=437.949026 shr=6.4195975 lines=386 END +moesel call=194 dev=0 T=1 E=512 k=10 sel=f3629bf962962ed4 minulps=1 mintok=0 x=842.832673 logit=3179.79688 exp=376.089623 shr=8.64113953 lines=387 END +moesel call=195 dev=0 T=1 E=512 k=10 sel=d55bf8389a54c9fb minulps=1 mintok=0 x=1078.46139 logit=3033.20312 exp=590.782449 shr=5.92302778 lines=388 END +moesel call=196 dev=0 T=1 E=512 k=10 sel=36dc03036fc7c978 minulps=0 mintok=0 x=1060.07721 logit=3011.92188 exp=582.046325 shr=5.94607417 lines=389 END +moesel call=197 dev=0 T=1 E=512 k=10 sel=a88e6a7d9a5a07dd minulps=1 mintok=0 x=949.684791 logit=3075.35938 exp=394.444535 shr=5.0281306 lines=390 END +moesel call=198 dev=0 T=1 E=512 k=10 sel=1bde0ab8790b3a9f minulps=5 mintok=0 x=1144.69436 logit=3118.64062 exp=691.466931 shr=6.24584097 lines=391 END +moesel call=199 dev=0 T=1 E=512 k=10 sel=a5901fcade5aeb5a minulps=6 mintok=0 x=1084.59545 logit=3181.70312 exp=514.253633 shr=4.58468512 lines=392 END +moesel call=200 dev=0 T=1 E=512 k=10 sel=651be32f40ba2189 minulps=2 mintok=0 x=1094.96036 logit=3051.125 exp=592.097759 shr=5.58841326 lines=393 END +moesel call=201 dev=0 T=1 E=512 k=10 sel=4417b1db3d817d3e minulps=0 mintok=0 x=1118.3233 logit=3082.14062 exp=602.553823 shr=5.69393443 lines=394 END +moesel call=202 dev=0 T=1 E=512 k=10 sel=58ed711438b36896 minulps=0 mintok=0 x=1041.59428 logit=3158.1875 exp=505.892227 shr=4.74559257 lines=395 END +moesel call=203 dev=0 T=1 E=512 k=10 sel=2798c511b0ca9ac7 minulps=1 mintok=0 x=1051.18178 logit=3081.125 exp=525.968272 shr=4.96797041 lines=396 END +moesel call=204 dev=0 T=1 E=512 k=10 sel=d6dbe34036838bfe minulps=6 mintok=0 x=1101.62494 logit=3145.875 exp=496.542189 shr=5.36030903 lines=397 END +moesel call=205 dev=0 T=1 E=512 k=10 sel=267cf3e13ec62502 minulps=9 mintok=0 x=1023.10753 logit=3136.07812 exp=503.564436 shr=4.55933059 lines=398 END +moesel call=206 dev=0 T=1 E=512 k=10 sel=4d0bd04cb548f074 minulps=3 mintok=0 x=1128.66985 logit=3069.54688 exp=595.28454 shr=6.81580678 lines=399 END +moesel call=207 dev=0 T=1 E=512 k=10 sel=0018511c7cea8f6f minulps=1 mintok=0 x=1461.56197 logit=3111.71875 exp=804.730988 shr=7.45506278 lines=400 END +moesel call=208 dev=0 T=1 E=512 k=10 sel=038c91e55b73dfd6 minulps=6 mintok=0 x=1295.11488 logit=3057.35938 exp=653.777343 shr=5.7832199 lines=401 END +moesel call=209 dev=0 T=1 E=512 k=10 sel=20af3977d7677193 minulps=1 mintok=0 x=1080.79119 logit=2994.48438 exp=479.642704 shr=5.36518809 lines=402 END +moesel call=210 dev=0 T=1 E=512 k=10 sel=ee831f5d0cf832ce minulps=2 mintok=0 x=1135.78924 logit=3196.26562 exp=548.740983 shr=5.72230443 lines=403 END +moesel call=211 dev=0 T=1 E=512 k=10 sel=e2ce040888d1bb11 minulps=1 mintok=0 x=1256.81571 logit=3183.99219 exp=654.013566 shr=5.91730656 lines=404 END +moesel call=212 dev=0 T=1 E=512 k=10 sel=447e7ca666c6e126 minulps=0 mintok=0 x=1552.87338 logit=3278.48438 exp=1149.22229 shr=12.3917163 lines=405 END +moesel call=213 dev=0 T=1 E=512 k=10 sel=9d5fa6a60280a40a minulps=1 mintok=0 x=1460.57157 logit=3178.59375 exp=837.561507 shr=6.90216361 lines=406 END +moesel call=214 dev=0 T=1 E=512 k=10 sel=f0dbba8e2b188530 minulps=11 mintok=0 x=1435.35947 logit=3347.32812 exp=807.203559 shr=6.9085494 lines=407 END +moesel call=215 dev=0 T=1 E=512 k=10 sel=66c0f1f0479ffadb minulps=1 mintok=0 x=1606.52447 logit=3155.17188 exp=1033.16949 shr=15.3148386 lines=408 END +moesel call=216 dev=0 T=1 E=512 k=10 sel=cc510b9c49ed2d00 minulps=1 mintok=0 x=1161.21321 logit=3186.01562 exp=583.139586 shr=5.90356094 lines=409 END +moesel call=217 dev=0 T=1 E=512 k=10 sel=ac32ad67d4bd751a minulps=3 mintok=0 x=1120.68004 logit=3077.76562 exp=534.875565 shr=5.12656626 lines=410 END +moesel call=218 dev=0 T=1 E=512 k=10 sel=959ffa2010515604 minulps=9 mintok=0 x=1126.36991 logit=3185.39062 exp=536.596135 shr=4.70694034 lines=411 END +moesel call=219 dev=0 T=1 E=512 k=10 sel=6b67e7dc07b53f5b minulps=3 mintok=0 x=1124.05293 logit=3063.07812 exp=487.673697 shr=4.76643467 lines=412 END +moesel call=220 dev=0 T=1 E=512 k=10 sel=3ac2ba29dde59d39 minulps=3 mintok=0 x=1191.57999 logit=3152.03125 exp=561.666117 shr=4.72839332 lines=413 END +moesel call=221 dev=0 T=1 E=512 k=10 sel=ce99874228f254b0 minulps=5 mintok=0 x=1197.95257 logit=3152.48438 exp=596.825586 shr=5.33076208 lines=414 END +moesel call=222 dev=0 T=1 E=512 k=10 sel=3af0aa8ba3838d70 minulps=0 mintok=0 x=1273.0381 logit=3064.42188 exp=721.419823 shr=9.51686171 lines=415 END +moesel call=223 dev=0 T=1 E=512 k=10 sel=ad77c8d45c7ce9c0 minulps=1 mintok=0 x=1523.41024 logit=3128.51562 exp=887.931278 shr=10.2387804 lines=416 END +moesel call=224 dev=0 T=1 E=512 k=10 sel=252c7c0de8c512b2 minulps=4 mintok=0 x=1497.84688 logit=3076.26562 exp=823.353848 shr=11.8247422 lines=417 END +moesel call=225 dev=0 T=1 E=512 k=10 sel=c320c05ede9fc421 minulps=1 mintok=0 x=1461.94109 logit=3160.96875 exp=918.791436 shr=7.01439513 lines=418 END +moesel call=226 dev=0 T=1 E=512 k=10 sel=89b01295228b1b96 minulps=2 mintok=0 x=1368.63893 logit=3252.625 exp=827.248699 shr=8.87961981 lines=419 END +moesel call=227 dev=0 T=1 E=512 k=10 sel=c198c77e26eea249 minulps=1 mintok=0 x=1678.00058 logit=3194.26562 exp=1209.90576 shr=11.5028279 lines=420 END +moesel call=228 dev=0 T=1 E=512 k=10 sel=e18856bce8825f99 minulps=1 mintok=0 x=2098.54796 logit=3199.03125 exp=2244.56434 shr=22.519245 lines=421 END +moesel call=229 dev=0 T=1 E=512 k=10 sel=869de8075d997083 minulps=0 mintok=0 x=1962.59503 logit=3198.96875 exp=1661.12753 shr=15.560072 lines=422 END +moesel call=230 dev=0 T=1 E=512 k=10 sel=733d8575121d937d minulps=3 mintok=0 x=1927.04253 logit=3310.73438 exp=2137.46422 shr=19.1797356 lines=423 END +moesel call=231 dev=0 T=1 E=512 k=10 sel=8eae380edd67ae35 minulps=4 mintok=0 x=1951.92702 logit=3038.76562 exp=1906.98826 shr=20.5522859 lines=424 END +moesel call=232 dev=0 T=1 E=512 k=10 sel=5cf08d1ad6f1fb7a minulps=0 mintok=0 x=1954.75556 logit=3016.32812 exp=1524.3282 shr=22.0171233 lines=425 END +moesel call=233 dev=0 T=1 E=512 k=10 sel=57b5aaeccae2daf3 minulps=0 mintok=0 x=2030.80098 logit=3131.5625 exp=1501.26161 shr=17.2168976 lines=426 END +moesel call=234 dev=0 T=1 E=512 k=10 sel=2a899308e2b70ab3 minulps=1 mintok=0 x=2148.25961 logit=3215.17188 exp=2059.23845 shr=22.0976569 lines=427 END +moesel call=235 dev=0 T=1 E=512 k=10 sel=fe1acb1931b52830 minulps=0 mintok=0 x=1667.36501 logit=3090.6875 exp=1054.48358 shr=18.6527985 lines=428 END +moesel call=236 dev=0 T=1 E=512 k=10 sel=38220bd5b93d987a minulps=0 mintok=0 x=2100.92886 logit=3219.29688 exp=1510.38157 shr=18.487669 lines=429 END +moesel call=237 dev=0 T=1 E=512 k=10 sel=16b9ebe672a7440b minulps=0 mintok=0 x=2252.26691 logit=3289.04688 exp=1527.21831 shr=20.6977258 lines=430 END +moesel call=238 dev=0 T=1 E=512 k=10 sel=4585ce64901c1120 minulps=1 mintok=0 x=2225.3584 logit=3155.09375 exp=1539.90151 shr=28.5997086 lines=431 END +moesel call=239 dev=0 T=1 E=512 k=10 sel=aeab7eb7aaad1d90 minulps=3 mintok=0 x=2125.31551 logit=3224.57812 exp=1246.8143 shr=108.693364 lines=432 END +moesel call=240 dev=0 T=1 E=512 k=10 sel=313e8aa1f8b2371d minulps=0 mintok=0 x=679.889969 logit=3463.76562 exp=567.10042 shr=37.5760253 lines=433 END +moesel call=241 dev=0 T=1 E=512 k=10 sel=1a1dea2cb93dc617 minulps=3 mintok=0 x=860.278532 logit=3185.25 exp=463.900769 shr=5.37393649 lines=434 END +moesel call=242 dev=0 T=1 E=512 k=10 sel=80d46976e8832350 minulps=6 mintok=0 x=874.798262 logit=3168.625 exp=399.728898 shr=7.68230708 lines=435 END +moesel call=243 dev=0 T=1 E=512 k=10 sel=d65b645151f3cadd minulps=3 mintok=0 x=1092.09847 logit=3000.07812 exp=660.582582 shr=8.39706464 lines=436 END +moesel call=244 dev=0 T=1 E=512 k=10 sel=ade897a8ca71a377 minulps=2 mintok=0 x=1134.79967 logit=3062.0625 exp=625.690093 shr=10.9471523 lines=437 END +moesel call=245 dev=0 T=1 E=512 k=10 sel=3337f7727d302517 minulps=3 mintok=0 x=1007.48689 logit=3076.28125 exp=437.289432 shr=9.92289427 lines=438 END +moesel call=246 dev=0 T=1 E=512 k=10 sel=6ced9e0fbb9734bc minulps=4 mintok=0 x=1147.92157 logit=3179.40625 exp=681.492524 shr=6.15881637 lines=439 END +moesel call=247 dev=0 T=1 E=512 k=10 sel=8c3ef3b0d0ae7913 minulps=2 mintok=0 x=1107.90611 logit=3146.95312 exp=569.927384 shr=6.37451947 lines=440 END +moesel call=248 dev=0 T=1 E=512 k=10 sel=8cdfdfb77e408218 minulps=0 mintok=0 x=1115.51341 logit=3023.89062 exp=583.387254 shr=6.30009348 lines=441 END +moesel call=249 dev=0 T=1 E=512 k=10 sel=1a5ebbf8bccb68ee minulps=2 mintok=0 x=1162.72114 logit=3037.25 exp=655.106069 shr=5.92989672 lines=442 END +moesel call=250 dev=0 T=1 E=512 k=10 sel=5c7702307f31fdc8 minulps=1 mintok=0 x=1063.1795 logit=3063.39062 exp=565.082634 shr=4.69016513 lines=443 END +moesel call=251 dev=0 T=1 E=512 k=10 sel=7d3020228600a698 minulps=0 mintok=0 x=1074.89305 logit=3028.03125 exp=531.726752 shr=4.95197909 lines=444 END +moesel call=252 dev=0 T=1 E=512 k=10 sel=33d71b1a2e5ef978 minulps=0 mintok=0 x=1116.43097 logit=3098.85938 exp=513.639663 shr=6.05464247 lines=445 END +moesel call=253 dev=0 T=1 E=512 k=10 sel=5dc669be1db0220c minulps=2 mintok=0 x=1001.71599 logit=3008.79688 exp=462.334784 shr=5.82557359 lines=446 END +moesel call=254 dev=0 T=1 E=512 k=10 sel=d8cce58214282a56 minulps=1 mintok=0 x=1126.47991 logit=3002.5625 exp=560.081899 shr=7.61708035 lines=447 END +moesel call=255 dev=0 T=1 E=512 k=10 sel=d606b23cad97fc89 minulps=5 mintok=0 x=1676.88349 logit=3134.4375 exp=1156.8899 shr=7.20779438 lines=448 END +moesel call=256 dev=0 T=1 E=512 k=10 sel=55d784c5cd482218 minulps=8 mintok=0 x=1369.21154 logit=3017.04688 exp=753.026495 shr=6.38711438 lines=449 END +moesel call=257 dev=0 T=1 E=512 k=10 sel=2647c20ba371219d minulps=0 mintok=0 x=1191.20086 logit=3023.98438 exp=562.921191 shr=5.45886013 lines=450 END +moesel call=258 dev=0 T=1 E=512 k=10 sel=db85e3b84703f782 minulps=2 mintok=0 x=1192.62422 logit=3201.9375 exp=617.203346 shr=6.92112707 lines=451 END +moesel call=259 dev=0 T=1 E=512 k=10 sel=d554b88e910dd8f9 minulps=1 mintok=0 x=1313.76158 logit=3109.86719 exp=654.178819 shr=6.85460453 lines=452 END +moesel call=260 dev=0 T=1 E=512 k=10 sel=3cebd5841feff49c minulps=1 mintok=0 x=1748.36034 logit=3199.75 exp=1200.58362 shr=14.2438044 lines=453 END +moesel call=261 dev=0 T=1 E=512 k=10 sel=ff271e465447dc87 minulps=7 mintok=0 x=1475.9109 logit=3105.01562 exp=882.76117 shr=11.476743 lines=454 END +moesel call=262 dev=0 T=1 E=512 k=10 sel=2e554904dd76e57e minulps=3 mintok=0 x=1405.51104 logit=3312.8125 exp=765.863996 shr=9.73312861 lines=455 END +moesel call=263 dev=0 T=1 E=512 k=10 sel=585837905fa10257 minulps=7 mintok=0 x=1760.61483 logit=3180.46875 exp=1084.86851 shr=11.8500076 lines=456 END +moesel call=264 dev=0 T=1 E=512 k=10 sel=b63cd6085d2a092e minulps=0 mintok=0 x=1230.74654 logit=3169.21875 exp=635.532898 shr=7.00597061 lines=457 END +moesel call=265 dev=0 T=1 E=512 k=10 sel=223462b18d423d6e minulps=9 mintok=0 x=1138.43392 logit=2991.09375 exp=511.542922 shr=5.27611259 lines=458 END +moesel call=266 dev=0 T=1 E=512 k=10 sel=69fa182ee332548d minulps=9 mintok=0 x=1172.15502 logit=3157.23438 exp=546.535292 shr=5.93225051 lines=459 END +moesel call=267 dev=0 T=1 E=512 k=10 sel=8e36dc807fb6d776 minulps=2 mintok=0 x=1198.90388 logit=3112 exp=550.252909 shr=5.44168846 lines=460 END +moesel call=268 dev=0 T=1 E=512 k=10 sel=c908d12bb6ded22f minulps=11 mintok=0 x=1248.57796 logit=3207.45312 exp=584.063421 shr=5.4245027 lines=461 END +moesel call=269 dev=0 T=1 E=512 k=10 sel=951bf44f7b6351d4 minulps=8 mintok=0 x=1254.04182 logit=3142.57031 exp=609.852381 shr=7.00630525 lines=462 END +moesel call=270 dev=0 T=1 E=512 k=10 sel=4a6c62a92417fb07 minulps=0 mintok=0 x=1443.55998 logit=3153.78125 exp=831.783232 shr=8.63710169 lines=463 END +moesel call=271 dev=0 T=1 E=512 k=10 sel=464daf12dd534cb9 minulps=0 mintok=0 x=1728.59673 logit=3136.89062 exp=1032.99495 shr=8.16373086 lines=464 END +moesel call=272 dev=0 T=1 E=512 k=10 sel=1c55e95e26d65235 minulps=2 mintok=0 x=1672.97059 logit=3087.75 exp=966.479639 shr=10.3026717 lines=465 END +moesel call=273 dev=0 T=1 E=512 k=10 sel=5d6e0fbdc5007cb5 minulps=2 mintok=0 x=1541.22878 logit=3202.98438 exp=991.586527 shr=7.00148208 lines=466 END +moesel call=274 dev=0 T=1 E=512 k=10 sel=40b206eb41aaee1b minulps=1 mintok=0 x=1450.42547 logit=3270.01562 exp=967.231181 shr=9.64170421 lines=467 END +moesel call=275 dev=0 T=1 E=512 k=10 sel=74fac48c1e2c31f6 minulps=1 mintok=0 x=1719.49511 logit=3271.76562 exp=1267.3607 shr=9.73379068 lines=468 END +moesel call=276 dev=0 T=1 E=512 k=10 sel=67aeeae4008e2417 minulps=1 mintok=0 x=2059.33632 logit=3192.67188 exp=1810.40256 shr=20.807757 lines=469 END +moesel call=277 dev=0 T=1 E=512 k=10 sel=2e20000d94e9b920 minulps=1 mintok=0 x=1968.24071 logit=3161.92188 exp=1820.24125 shr=17.4798741 lines=470 END +moesel call=278 dev=0 T=1 E=512 k=10 sel=fafa110131150eab minulps=3 mintok=0 x=1889.36966 logit=3200.65625 exp=1755.15098 shr=23.6713966 lines=471 END +moesel call=279 dev=0 T=1 E=512 k=10 sel=756c12fd46a83e14 minulps=0 mintok=0 x=2077.86097 logit=3063.20312 exp=1900.61443 shr=19.1389007 lines=472 END +moesel call=280 dev=0 T=1 E=512 k=10 sel=1d6e1ee1d12b1dbb minulps=0 mintok=0 x=2124.35651 logit=3064.42188 exp=1825.3641 shr=19.3226681 lines=473 END +moesel call=281 dev=0 T=1 E=512 k=10 sel=3d86cc3ed6bc1119 minulps=2 mintok=0 x=2252.72728 logit=3196.1875 exp=1895.10492 shr=20.5761556 lines=474 END +moesel call=282 dev=0 T=1 E=512 k=10 sel=7c852893544cd14d minulps=0 mintok=0 x=1977.10798 logit=3096.79688 exp=2173.13381 shr=26.9104897 lines=475 END +moesel call=283 dev=0 T=1 E=512 k=10 sel=02b087e5c0a5b528 minulps=2 mintok=0 x=1771.88104 logit=3096.0625 exp=1201.84482 shr=15.8606838 lines=476 END +moesel call=284 dev=0 T=1 E=512 k=10 sel=9b240facc3be5e58 minulps=1 mintok=0 x=2235.23505 logit=3167.04688 exp=1786.89934 shr=22.6928716 lines=477 END +moesel call=285 dev=0 T=1 E=512 k=10 sel=8560a07d382f356b minulps=0 mintok=0 x=2439.82397 logit=3225.67188 exp=1591.14833 shr=23.9335559 lines=478 END +moesel call=286 dev=0 T=1 E=512 k=10 sel=dc5a384394b3345e minulps=0 mintok=0 x=2250.02311 logit=3232.1875 exp=1454.84173 shr=33.1123497 lines=479 END +moesel call=287 dev=0 T=1 E=512 k=10 sel=25f74f64ea240606 minulps=0 mintok=0 x=1986.24271 logit=3244.15625 exp=1190.10885 shr=110.485694 lines=480 END +moesel call=288 dev=0 T=1 E=512 k=10 sel=65c4a1664e07f708 minulps=1 mintok=0 x=725.714377 logit=3479.07812 exp=574.053736 shr=54.5889122 lines=481 END +moesel call=289 dev=0 T=1 E=512 k=10 sel=d9553908682b329c minulps=2 mintok=0 x=698.395152 logit=3100.6875 exp=395.024348 shr=4.60022465 lines=482 END +moesel call=290 dev=0 T=1 E=512 k=10 sel=283ad63a8ed7b442 minulps=0 mintok=0 x=781.103161 logit=3124.78125 exp=355.528568 shr=5.40893444 lines=483 END +moesel call=291 dev=0 T=1 E=512 k=10 sel=9b9c4f4a845881ed minulps=6 mintok=0 x=1026.58546 logit=3054.29688 exp=624.412025 shr=5.17255085 lines=484 END +moesel call=292 dev=0 T=1 E=512 k=10 sel=7a1ea1e6ba54a448 minulps=8 mintok=0 x=1050.63661 logit=2992.46875 exp=508.878341 shr=5.27879809 lines=485 END +moesel call=293 dev=0 T=1 E=512 k=10 sel=16b57f7a8b628201 minulps=4 mintok=0 x=904.385968 logit=3091.84375 exp=405.241234 shr=3.92045931 lines=486 END +moesel call=294 dev=0 T=1 E=512 k=10 sel=ac8bbfbea67faeff minulps=3 mintok=0 x=1070.35198 logit=3126.20312 exp=586.737313 shr=6.37898166 lines=487 END +moesel call=295 dev=0 T=1 E=512 k=10 sel=d9343c9619a6bfec minulps=4 mintok=0 x=1042.55654 logit=3152.40625 exp=524.855719 shr=4.48317985 lines=488 END +moesel call=296 dev=0 T=1 E=512 k=10 sel=8f70a9093f81cd98 minulps=1 mintok=0 x=1043.43202 logit=3019.78125 exp=537.966236 shr=5.05144785 lines=489 END +moesel call=297 dev=0 T=1 E=512 k=10 sel=d2035c730d97d8bb minulps=3 mintok=0 x=1075.2371 logit=3033.375 exp=596.580955 shr=6.06830943 lines=490 END +moesel call=298 dev=0 T=1 E=512 k=10 sel=8152ea1786d24b85 minulps=5 mintok=0 x=1040.69753 logit=3057.45312 exp=501.279944 shr=4.24556739 lines=491 END +moesel call=299 dev=0 T=1 E=512 k=10 sel=17af1b8fdc35b64e minulps=0 mintok=0 x=1073.69147 logit=3067.95312 exp=525.250522 shr=4.67117937 lines=492 END +moesel call=300 dev=0 T=1 E=512 k=10 sel=9a5dcd8f9cd8779c minulps=4 mintok=0 x=1100.9368 logit=3071.82812 exp=544.323194 shr=5.98198618 lines=493 END +moesel call=301 dev=0 T=1 E=512 k=10 sel=2969ce41bf41fb50 minulps=4 mintok=0 x=1057.58238 logit=3039.5625 exp=531.945031 shr=5.90877565 lines=494 END +moesel call=302 dev=0 T=1 E=512 k=10 sel=db7200e9d5b99318 minulps=3 mintok=0 x=1104.32088 logit=3009.09375 exp=558.763368 shr=7.39087255 lines=495 END +moesel call=303 dev=0 T=1 E=512 k=10 sel=1d93d36f3c11d1c8 minulps=1 mintok=0 x=1502.52884 logit=3103.98438 exp=1063.21384 shr=7.18970741 lines=496 END +moesel call=304 dev=0 T=1 E=512 k=10 sel=87a86dfce31576b9 minulps=3 mintok=0 x=1347.74365 logit=3053.71875 exp=753.566094 shr=7.00670701 lines=497 END +moesel call=305 dev=0 T=1 E=512 k=10 sel=41cc0b6a1718c880 minulps=4 mintok=0 x=1164.14633 logit=2986.84375 exp=547.457562 shr=5.73651992 lines=498 END +moesel call=306 dev=0 T=1 E=512 k=10 sel=de1b62ef14aa78de minulps=2 mintok=0 x=1134.59466 logit=3167.23438 exp=574.346282 shr=6.03064606 lines=499 END +moesel call=307 dev=0 T=1 E=512 k=10 sel=1b3784779dc0af8f minulps=0 mintok=0 x=1316.92364 logit=3119.42188 exp=694.134603 shr=7.14131635 lines=500 END +moesel call=308 dev=0 T=1 E=512 k=10 sel=2a23e5d32125c71f minulps=0 mintok=0 x=1702.7851 logit=3166.625 exp=1230.6785 shr=16.4940931 lines=501 END +moesel call=309 dev=0 T=1 E=512 k=10 sel=0e96d181fbe0f662 minulps=0 mintok=0 x=1454.35589 logit=3214.42188 exp=819.497698 shr=9.12751558 lines=502 END +moesel call=310 dev=0 T=1 E=512 k=10 sel=20d48bff19c4622b minulps=1 mintok=0 x=1365.67586 logit=3303.09375 exp=706.827515 shr=9.78021464 lines=503 END +moesel call=311 dev=0 T=1 E=512 k=10 sel=41be0d7ea6bfae2a minulps=1 mintok=0 x=1705.86607 logit=3131.96875 exp=1070.42255 shr=17.436403 lines=504 END +moesel call=312 dev=0 T=1 E=512 k=10 sel=f07b08f0b824624f minulps=1 mintok=0 x=1175.55225 logit=3134.71875 exp=543.637867 shr=6.6405838 lines=505 END +moesel call=313 dev=0 T=1 E=512 k=10 sel=306067407f578c6c minulps=4 mintok=0 x=1151.1226 logit=3062.95312 exp=544.949956 shr=4.31586302 lines=506 END +moesel call=314 dev=0 T=1 E=512 k=10 sel=af889cc77de28889 minulps=3 mintok=0 x=1118.60958 logit=3112.25 exp=532.277276 shr=4.5413089 lines=507 END +moesel call=315 dev=0 T=1 E=512 k=10 sel=fbb4003d2cf903e5 minulps=1 mintok=0 x=1146.01284 logit=3064.59375 exp=510.087216 shr=4.25406353 lines=508 END +moesel call=316 dev=0 T=1 E=512 k=10 sel=7b5eb38026a67fa0 minulps=5 mintok=0 x=1202.11007 logit=3088.65625 exp=591.727532 shr=4.87267454 lines=509 END +moesel call=317 dev=0 T=1 E=512 k=10 sel=1438151a99fdf1fc minulps=7 mintok=0 x=1206.8433 logit=3162.17969 exp=591.162721 shr=5.94734176 lines=510 END +moesel call=318 dev=0 T=1 E=512 k=10 sel=afa719036db2c0f5 minulps=1 mintok=0 x=1372.0859 logit=3136.40625 exp=718.917334 shr=6.30778893 lines=511 END +moesel call=319 dev=0 T=1 E=512 k=10 sel=a981fd53e22fef1e minulps=5 mintok=0 x=1769.45648 logit=3175.03125 exp=1089.9384 shr=6.01519078 lines=512 END +moesel call=320 dev=0 T=1 E=512 k=10 sel=f512022075ab967a minulps=4 mintok=0 x=1677.24164 logit=3127.76562 exp=959.01701 shr=6.77459425 lines=513 END +moesel call=321 dev=0 T=1 E=512 k=10 sel=e3e83d079774bac0 minulps=1 mintok=0 x=1543.26454 logit=3190.46094 exp=942.540585 shr=5.17241367 lines=514 END +moesel call=322 dev=0 T=1 E=512 k=10 sel=d83b85ddd5d2e1f0 minulps=2 mintok=0 x=1387.29871 logit=3252.98438 exp=794.058034 shr=7.04065768 lines=515 END +moesel call=323 dev=0 T=1 E=512 k=10 sel=22861aec8ff4b678 minulps=3 mintok=0 x=1706.48158 logit=3243.46875 exp=1256.44632 shr=8.59600396 lines=516 END +moesel call=324 dev=0 T=1 E=512 k=10 sel=e9cdf6e7585ee84c minulps=2 mintok=0 x=2122.00943 logit=3170.73438 exp=2229.17721 shr=18.9982709 lines=517 END +moesel call=325 dev=0 T=1 E=512 k=10 sel=cb523256829ff045 minulps=0 mintok=0 x=1952.80993 logit=3240.79688 exp=1593.96826 shr=13.937657 lines=518 END +moesel call=326 dev=0 T=1 E=512 k=10 sel=d5906ad53d3a6484 minulps=1 mintok=0 x=1826.51982 logit=3330.98438 exp=1620.15082 shr=20.6728923 lines=519 END +moesel call=327 dev=0 T=1 E=512 k=10 sel=98f926914da9d6ae minulps=3 mintok=0 x=2110.17197 logit=3134.96875 exp=1890.36801 shr=15.6439174 lines=520 END +moesel call=328 dev=0 T=1 E=512 k=10 sel=7ae646d409f66388 minulps=1 mintok=0 x=2126.12093 logit=3055.54688 exp=1829.18977 shr=21.8441796 lines=521 END +moesel call=329 dev=0 T=1 E=512 k=10 sel=754474e0ad3b3d7a minulps=1 mintok=0 x=2351.24663 logit=3231.59375 exp=2071.29422 shr=14.9193765 lines=522 END +moesel call=330 dev=0 T=1 E=512 k=10 sel=24c522327b640a75 minulps=0 mintok=0 x=1999.79314 logit=3185.20312 exp=1894.82306 shr=24.8555946 lines=523 END +moesel call=331 dev=0 T=1 E=512 k=10 sel=8e5f507371f0d7b0 minulps=3 mintok=0 x=1933.59995 logit=3132.17188 exp=1420.90853 shr=16.1825735 lines=524 END +moesel call=332 dev=0 T=1 E=512 k=10 sel=270eebc98e4bead2 minulps=3 mintok=0 x=2184.76522 logit=3141.25 exp=1842.80225 shr=20.4444085 lines=525 END +moesel call=333 dev=0 T=1 E=512 k=10 sel=5354b8cea9dba18b minulps=3 mintok=0 x=2459.68673 logit=3237.875 exp=1626.42225 shr=27.8580039 lines=526 END +moesel call=334 dev=0 T=1 E=512 k=10 sel=1142e3534969a49e minulps=0 mintok=0 x=2406.31829 logit=3322.17188 exp=1799.39596 shr=40.4937158 lines=527 END +moesel call=335 dev=0 T=1 E=512 k=10 sel=c94a22fd88631213 minulps=3 mintok=0 x=2081.18664 logit=3374.34375 exp=1217.60359 shr=122.361991 lines=528 END +moesel call=336 dev=0 T=1 E=512 k=10 sel=b291daf010588d28 minulps=1 mintok=0 x=659.597758 logit=3447.39062 exp=469.159108 shr=45.7341356 lines=529 END +moesel call=337 dev=0 T=1 E=512 k=10 sel=ee32bc9736b6cf00 minulps=0 mintok=0 x=744.214747 logit=3119.85938 exp=405.268057 shr=6.88231257 lines=530 END +moesel call=338 dev=0 T=1 E=512 k=10 sel=50f50dedf5022755 minulps=6 mintok=0 x=820.894992 logit=3177.5 exp=397.777142 shr=5.8978748 lines=531 END +moesel call=339 dev=0 T=1 E=512 k=10 sel=c4132c5ffb9e3f9c minulps=1 mintok=0 x=976.810976 logit=3053.20312 exp=555.519444 shr=4.75735583 lines=532 END +moesel call=340 dev=0 T=1 E=512 k=10 sel=9c9cbc587c4e17e2 minulps=1 mintok=0 x=1021.63636 logit=3012.84375 exp=512.161756 shr=4.92166897 lines=533 END +moesel call=341 dev=0 T=1 E=512 k=10 sel=ecdbd26eba1207f3 minulps=1 mintok=0 x=895.423302 logit=3059.6875 exp=386.687547 shr=5.24265565 lines=534 END +moesel call=342 dev=0 T=1 E=512 k=10 sel=c2aec880ac22d520 minulps=6 mintok=0 x=1103.41817 logit=3091.20312 exp=562.54641 shr=6.19814041 lines=535 END +moesel call=343 dev=0 T=1 E=512 k=10 sel=2548dd9c77c6e7e4 minulps=0 mintok=0 x=1043.25394 logit=3116.15625 exp=487.197401 shr=6.63845846 lines=536 END +moesel call=344 dev=0 T=1 E=512 k=10 sel=4ad94ca019845820 minulps=7 mintok=0 x=1072.25548 logit=3064.90625 exp=503.086349 shr=6.67753621 lines=537 END +moesel call=345 dev=0 T=1 E=512 k=10 sel=8c3e238035b2e00c minulps=2 mintok=0 x=1084.56166 logit=3044.3125 exp=580.709931 shr=5.82407161 lines=538 END +moesel call=346 dev=0 T=1 E=512 k=10 sel=52893f4b249024aa minulps=1 mintok=0 x=1022.18437 logit=3088.23438 exp=455.926706 shr=5.36088835 lines=539 END +moesel call=347 dev=0 T=1 E=512 k=10 sel=d21e98016e134761 minulps=1 mintok=0 x=1029.04039 logit=3036.70312 exp=442.093493 shr=4.50964108 lines=540 END +moesel call=348 dev=0 T=1 E=512 k=10 sel=1b76e7d9385e7722 minulps=0 mintok=0 x=1087.00528 logit=3137.04688 exp=486.450697 shr=5.53218515 lines=541 END +moesel call=349 dev=0 T=1 E=512 k=10 sel=30013c162d01f2fe minulps=1 mintok=0 x=1003.66151 logit=3078.34375 exp=400.118767 shr=5.05321492 lines=542 END +moesel call=350 dev=0 T=1 E=512 k=10 sel=f11772ae6c3a563c minulps=2 mintok=0 x=1091.84171 logit=3083 exp=568.548157 shr=5.41433711 lines=543 END +moesel call=351 dev=0 T=1 E=512 k=10 sel=9f05bf74205c0721 minulps=7 mintok=0 x=1519.62847 logit=3129.46875 exp=906.670658 shr=6.11613493 lines=544 END +moesel call=352 dev=0 T=1 E=512 k=10 sel=eb54668ec73370b0 minulps=2 mintok=0 x=1376.96477 logit=3132.51562 exp=693.985085 shr=5.12042207 lines=545 END +moesel call=353 dev=0 T=1 E=512 k=10 sel=b6f816b2af49cdba minulps=3 mintok=0 x=1178.19841 logit=3098.24219 exp=564.683437 shr=3.89768215 lines=546 END +moesel call=354 dev=0 T=1 E=512 k=10 sel=26b79864e98625c0 minulps=0 mintok=0 x=1191.77339 logit=3240.42188 exp=572.89218 shr=5.64262325 lines=547 END +moesel call=355 dev=0 T=1 E=512 k=10 sel=f1d72110a36eaffc minulps=1 mintok=0 x=1327.32575 logit=3190.39844 exp=676.942388 shr=5.84557788 lines=548 END +moesel call=356 dev=0 T=1 E=512 k=10 sel=c2c0fc9410fbbe29 minulps=2 mintok=0 x=1675.41935 logit=3261.26562 exp=1132.40968 shr=12.6039939 lines=549 END +moesel call=357 dev=0 T=1 E=512 k=10 sel=1ce725bd05554cae minulps=5 mintok=0 x=1560.32608 logit=3228.60938 exp=818.527346 shr=8.10074924 lines=550 END +moesel call=358 dev=0 T=1 E=512 k=10 sel=63488d7e9d2c4504 minulps=4 mintok=0 x=1479.46151 logit=3321.84375 exp=878.031076 shr=9.61285484 lines=551 END +moesel call=359 dev=0 T=1 E=512 k=10 sel=0eee9341edc422b7 minulps=1 mintok=0 x=1755.06528 logit=3166.26562 exp=1101.81624 shr=17.2246516 lines=552 END +moesel call=360 dev=0 T=1 E=512 k=10 sel=a6b071237faf4939 minulps=0 mintok=0 x=1208.2373 logit=3162.96875 exp=533.579158 shr=7.38802949 lines=553 END +moesel call=361 dev=0 T=1 E=512 k=10 sel=1ca8d72d91047dfa minulps=1 mintok=0 x=1230.13184 logit=3076.29688 exp=558.367758 shr=5.81657105 lines=554 END +moesel call=362 dev=0 T=1 E=512 k=10 sel=933cd14c3bd4b52e minulps=2 mintok=0 x=1140.70204 logit=3155.53125 exp=533.99655 shr=4.07003582 lines=555 END +moesel call=363 dev=0 T=1 E=512 k=10 sel=2dbdc55966c7dd8b minulps=7 mintok=0 x=1153.27789 logit=3073.625 exp=557.406727 shr=5.22911939 lines=556 END +moesel call=364 dev=0 T=1 E=512 k=10 sel=8b7f2d2d2c099b60 minulps=17 mintok=0 x=1207.63631 logit=3124.90625 exp=582.194724 shr=5.69001283 lines=557 END +moesel call=365 dev=0 T=1 E=512 k=10 sel=6643974331867f0f minulps=1 mintok=0 x=1233.70158 logit=3159.85156 exp=598.597766 shr=6.52653512 lines=558 END +moesel call=366 dev=0 T=1 E=512 k=10 sel=05fd9dc99c8030b0 minulps=0 mintok=0 x=1427.28672 logit=3160.21875 exp=878.17876 shr=9.61677593 lines=559 END +moesel call=367 dev=0 T=1 E=512 k=10 sel=4dd5c3066fd85d47 minulps=2 mintok=0 x=1735.77043 logit=3152.92188 exp=1124.61654 shr=8.02624789 lines=560 END +moesel call=368 dev=0 T=1 E=512 k=10 sel=0f3004f32e150574 minulps=2 mintok=0 x=1686.73968 logit=3083.09375 exp=1044.54516 shr=8.6367342 lines=561 END +moesel call=369 dev=0 T=1 E=512 k=10 sel=7f9ba315ca0d0e17 minulps=2 mintok=0 x=1526.28354 logit=3173.75 exp=962.364124 shr=5.67905901 lines=562 END +moesel call=370 dev=0 T=1 E=512 k=10 sel=a06dce1696f255a7 minulps=2 mintok=0 x=1456.22031 logit=3270.875 exp=1028.26673 shr=9.75792856 lines=563 END +moesel call=371 dev=0 T=1 E=512 k=10 sel=d8acbe0688ac969c minulps=0 mintok=0 x=1684.10358 logit=3224.45312 exp=1357.37747 shr=11.3340499 lines=564 END +moesel call=372 dev=0 T=1 E=512 k=10 sel=0bc097545ee84525 minulps=1 mintok=0 x=2024.9783 logit=3170.73438 exp=2188.19417 shr=24.8378046 lines=565 END +moesel call=373 dev=0 T=1 E=512 k=10 sel=d2176357f953daec minulps=1 mintok=0 x=1936.66635 logit=3245.34375 exp=1617.78563 shr=16.9567539 lines=566 END +moesel call=374 dev=0 T=1 E=512 k=10 sel=bafbe7a4b3084f1c minulps=1 mintok=0 x=2046.25077 logit=3364.85938 exp=2342.15202 shr=24.7374004 lines=567 END +moesel call=375 dev=0 T=1 E=512 k=10 sel=e683a36f83bc94f1 minulps=0 mintok=0 x=2122.94685 logit=3053.25 exp=2084.32494 shr=24.3336982 lines=568 END +moesel call=376 dev=0 T=1 E=512 k=10 sel=b157d56d3aafcca5 minulps=1 mintok=0 x=2061.46975 logit=3035.375 exp=1779.68694 shr=27.7318195 lines=569 END +moesel call=377 dev=0 T=1 E=512 k=10 sel=1e1dbb003a35f100 minulps=1 mintok=0 x=2239.2497 logit=3123.10938 exp=2039.06469 shr=19.410901 lines=570 END +moesel call=378 dev=0 T=1 E=512 k=10 sel=93a4e9383ea41440 minulps=1 mintok=0 x=2189.8285 logit=3234.20312 exp=2259.36728 shr=26.0850141 lines=571 END +moesel call=379 dev=0 T=1 E=512 k=10 sel=bb5aa0e989464ade minulps=4 mintok=0 x=1933.07185 logit=3144.82812 exp=1486.27811 shr=23.1175638 lines=572 END +moesel call=380 dev=0 T=1 E=512 k=10 sel=45b34bd15ceb4c30 minulps=0 mintok=0 x=2333.60364 logit=3283.95312 exp=2022.50347 shr=22.2763712 lines=573 END +moesel call=381 dev=0 T=1 E=512 k=10 sel=f4e46c128544fade minulps=1 mintok=0 x=2434.59585 logit=3230.5 exp=1760.3718 shr=28.0803697 lines=574 END +moesel call=382 dev=0 T=1 E=512 k=10 sel=8fbba3271d996794 minulps=1 mintok=0 x=2435.83194 logit=3288.67188 exp=1834.84836 shr=33.4673995 lines=575 END +moesel call=383 dev=0 T=1 E=512 k=10 sel=f50c7fe117d083f5 minulps=2 mintok=0 x=2213.45872 logit=3272.98438 exp=1350.7858 shr=128.38923 lines=576 END diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt new file mode 100644 index 000000000..3d1214d1c --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-GDNSEQ.txt @@ -0,0 +1,384 @@ +moesel call=0 dev=1 T=5 E=512 k=10 sel=f06a56dd5ffbf75d minulps=0 mintok=2 x=3613.74301 logit=17079.1641 exp=2540.45471 shr=250.794444 lines=5 END +moesel call=1 dev=1 T=5 E=512 k=10 sel=fb2a5272f1e4f8e0 minulps=0 mintok=2 x=3777.83058 logit=15567.3438 exp=2017.76373 shr=34.0138945 lines=10 END +moesel call=2 dev=1 T=5 E=512 k=10 sel=6ea2aef387496645 minulps=0 mintok=3 x=3842.47361 logit=15793.9375 exp=1800.37012 shr=51.3618547 lines=15 END +moesel call=3 dev=1 T=5 E=512 k=10 sel=eed17fad6513ba9d minulps=0 mintok=1 x=4873.4291 logit=15074.5156 exp=2582.47021 shr=37.7886879 lines=20 END +moesel call=4 dev=1 T=5 E=512 k=10 sel=4537bad749058f33 minulps=1 mintok=0 x=5355.60817 logit=15440.0156 exp=3064.05299 shr=47.2066198 lines=25 END +moesel call=5 dev=1 T=5 E=512 k=10 sel=ae1f95fee2b91838 minulps=0 mintok=4 x=4433.97312 logit=15627.6094 exp=2007.19558 shr=33.6438845 lines=30 END +moesel call=6 dev=1 T=5 E=512 k=10 sel=2ba8099d8f428dab minulps=0 mintok=0 x=5469.48399 logit=15911.25 exp=3096.6044 shr=46.8479666 lines=35 END +moesel call=7 dev=1 T=5 E=512 k=10 sel=4d910d271b635a3c minulps=0 mintok=4 x=5332.68826 logit=15908.0312 exp=2688.59794 shr=43.7803475 lines=40 END +moesel call=8 dev=1 T=5 E=512 k=10 sel=c0b7afc86cb49d16 minulps=0 mintok=2 x=5381.94492 logit=15746.7188 exp=3038.7866 shr=28.1943145 lines=45 END +moesel call=9 dev=1 T=5 E=512 k=10 sel=d27ac843303df465 minulps=0 mintok=0 x=5525.55351 logit=15657.6875 exp=2980.15014 shr=34.7109443 lines=50 END +moesel call=10 dev=1 T=5 E=512 k=10 sel=b14a7db9158b4e90 minulps=0 mintok=0 x=4963.28279 logit=15763.2812 exp=2419.79498 shr=26.9856972 lines=55 END +moesel call=11 dev=1 T=5 E=512 k=10 sel=a9f7fa4736c5c260 minulps=0 mintok=0 x=4866.5477 logit=15586.4688 exp=2152.86156 shr=18.6761832 lines=60 END +moesel call=12 dev=1 T=5 E=512 k=10 sel=98f4025615644ee6 minulps=0 mintok=0 x=4969.61794 logit=15637.7969 exp=2263.46059 shr=26.5210111 lines=65 END +moesel call=13 dev=1 T=5 E=512 k=10 sel=5ed47518079967d8 minulps=0 mintok=0 x=4648.40087 logit=15946.6172 exp=2030.92948 shr=27.4775673 lines=70 END +moesel call=14 dev=1 T=5 E=512 k=10 sel=031ae0561aec26c7 minulps=1 mintok=2 x=5256.95827 logit=15837.1875 exp=2909.15434 shr=22.7684577 lines=75 END +moesel call=15 dev=1 T=5 E=512 k=10 sel=88cb94c69381a618 minulps=0 mintok=0 x=6739.13267 logit=15811.4531 exp=3943.28047 shr=29.9124121 lines=80 END +moesel call=16 dev=1 T=5 E=512 k=10 sel=a8002b1c0688c414 minulps=0 mintok=1 x=6158.84255 logit=15758.3984 exp=3197.36537 shr=28.148731 lines=85 END +moesel call=17 dev=1 T=5 E=512 k=10 sel=cb0c0f7107d1b9f1 minulps=0 mintok=0 x=5371.43593 logit=15997.0938 exp=2598.86246 shr=23.7154928 lines=90 END +moesel call=18 dev=1 T=5 E=512 k=10 sel=f105feb5386cb177 minulps=1 mintok=2 x=5329.31178 logit=16324.9844 exp=2863.26153 shr=25.0691803 lines=95 END +moesel call=19 dev=1 T=5 E=512 k=10 sel=f099e75c3c364d86 minulps=0 mintok=0 x=5867.62627 logit=16197.375 exp=2835.76687 shr=28.2019645 lines=100 END +moesel call=20 dev=1 T=5 E=512 k=10 sel=7e2589d8f592e931 minulps=0 mintok=1 x=7148.21312 logit=16143 exp=4349.18548 shr=42.8663893 lines=105 END +moesel call=21 dev=1 T=5 E=512 k=10 sel=7155e9a4f0d1e731 minulps=0 mintok=0 x=6732.45336 logit=16193.1875 exp=3776.13145 shr=28.3413144 lines=110 END +moesel call=22 dev=1 T=5 E=512 k=10 sel=b3c62ea616442afa minulps=1 mintok=0 x=6492.17328 logit=16639.5312 exp=3611.11839 shr=34.8373965 lines=115 END +moesel call=23 dev=1 T=5 E=512 k=10 sel=a6e5fe051244ddf9 minulps=0 mintok=0 x=7579.43175 logit=16095.375 exp=4441.04099 shr=64.5485376 lines=120 END +moesel call=24 dev=1 T=5 E=512 k=10 sel=9540c28f286b90c6 minulps=0 mintok=1 x=5623.64631 logit=16329.8281 exp=2826.56102 shr=27.9391904 lines=125 END +moesel call=25 dev=1 T=5 E=512 k=10 sel=5817b743f19b664b minulps=0 mintok=0 x=5644.2011 logit=16026.5625 exp=2680.53693 shr=28.6913512 lines=130 END +moesel call=26 dev=1 T=5 E=512 k=10 sel=42c8ba089092b1b8 minulps=0 mintok=3 x=5242.63172 logit=16462.6016 exp=2432.41667 shr=22.653267 lines=135 END +moesel call=27 dev=1 T=5 E=512 k=10 sel=cb459862b970b77b minulps=0 mintok=1 x=5250.96584 logit=15928.7188 exp=2364.57499 shr=18.957759 lines=140 END +moesel call=28 dev=1 T=5 E=512 k=10 sel=3a31d0795bc6f47c minulps=1 mintok=3 x=5288.7323 logit=16112.25 exp=2343.11091 shr=23.6415624 lines=145 END +moesel call=29 dev=1 T=5 E=512 k=10 sel=6efd42d528ac6d8a minulps=0 mintok=2 x=5506.69599 logit=16535.8672 exp=2795.36289 shr=25.0596944 lines=150 END +moesel call=30 dev=1 T=5 E=512 k=10 sel=fb598e1355134959 minulps=0 mintok=4 x=6410.06056 logit=16661.2188 exp=3735.56547 shr=32.8470495 lines=155 END +moesel call=31 dev=1 T=5 E=512 k=10 sel=0050705d4ac202f3 minulps=0 mintok=1 x=8011.73438 logit=16345.3906 exp=4824.42603 shr=31.2061786 lines=160 END +moesel call=32 dev=1 T=5 E=512 k=10 sel=b7dded05065392b3 minulps=0 mintok=0 x=7319.41444 logit=16006.7109 exp=4399.70613 shr=32.7077647 lines=165 END +moesel call=33 dev=1 T=5 E=512 k=10 sel=4724818dacbff9f5 minulps=0 mintok=0 x=6685.99892 logit=16719.4062 exp=3823.74303 shr=25.313163 lines=170 END +moesel call=34 dev=1 T=5 E=512 k=10 sel=3188ef46a7207960 minulps=0 mintok=0 x=6111.76998 logit=16670.3125 exp=3806.79113 shr=34.2649005 lines=175 END +moesel call=35 dev=1 T=5 E=512 k=10 sel=a2b69c167a483a9f minulps=1 mintok=1 x=7041.42279 logit=16437.7656 exp=4584.31584 shr=34.4142891 lines=180 END +moesel call=36 dev=1 T=5 E=512 k=10 sel=f103fcb3508aa75b minulps=0 mintok=2 x=8744.37358 logit=16389.3984 exp=8149.37753 shr=88.5638159 lines=185 END +moesel call=37 dev=1 T=5 E=512 k=10 sel=8cffc9396f331d74 minulps=0 mintok=3 x=8264.8447 logit=16347.4062 exp=7002.85247 shr=65.2569962 lines=190 END +moesel call=38 dev=1 T=5 E=512 k=10 sel=bb350da9dcf52f46 minulps=0 mintok=3 x=8415.83688 logit=16815.75 exp=9246.79079 shr=109.602601 lines=195 END +moesel call=39 dev=1 T=5 E=512 k=10 sel=24d44eb6c44320f8 minulps=0 mintok=0 x=9002.68804 logit=15687.1641 exp=8439.7024 shr=96.6948645 lines=200 END +moesel call=40 dev=1 T=5 E=512 k=10 sel=ec50e4833d15dd6a minulps=0 mintok=0 x=9469.13997 logit=15723.0781 exp=7270.17214 shr=95.6658297 lines=205 END +moesel call=41 dev=1 T=5 E=512 k=10 sel=ee151309f1ded60c minulps=1 mintok=1 x=9563.14343 logit=16093.5781 exp=6882.67483 shr=100.706101 lines=210 END +moesel call=42 dev=1 T=5 E=512 k=10 sel=1a57c6c6a3bb19fb minulps=1 mintok=0 x=9784.81565 logit=16399.5625 exp=9223.56793 shr=131.682382 lines=215 END +moesel call=43 dev=1 T=5 E=512 k=10 sel=1a5af66d4fce33e6 minulps=0 mintok=1 x=8164.36896 logit=15766.6875 exp=5799.10439 shr=91.4694543 lines=220 END +moesel call=44 dev=1 T=5 E=512 k=10 sel=fbc17851e34f09a8 minulps=0 mintok=2 x=9133.89888 logit=15832.5312 exp=7374.70481 shr=105.2929 lines=225 END +moesel call=45 dev=1 T=5 E=512 k=10 sel=9fa59df33f617085 minulps=0 mintok=0 x=10331.2517 logit=16207.375 exp=7222.62607 shr=148.028945 lines=230 END +moesel call=46 dev=1 T=5 E=512 k=10 sel=1b55df0881bfc3af minulps=0 mintok=3 x=10065.2716 logit=16250.6719 exp=8490.58183 shr=158.074511 lines=235 END +moesel call=47 dev=1 T=5 E=512 k=10 sel=73ac3458f62c111f minulps=1 mintok=1 x=9562.57522 logit=16366.3281 exp=9053.50797 shr=825.618974 lines=240 END +moesel call=48 dev=1 T=1 E=512 k=10 sel=8fce961a87c895de minulps=7 mintok=0 x=723.470177 logit=3429.89062 exp=563.539422 shr=31.5357613 lines=241 END +moesel call=49 dev=1 T=1 E=512 k=10 sel=da67409db4906d8e minulps=2 mintok=0 x=816.576027 logit=3055.9375 exp=412.922141 shr=6.05394083 lines=242 END +moesel call=50 dev=1 T=1 E=512 k=10 sel=dc6a93971be9cf6b minulps=0 mintok=0 x=851.727616 logit=3039.20312 exp=411.391483 shr=9.24520291 lines=243 END +moesel call=51 dev=1 T=1 E=512 k=10 sel=da92804a7ed93391 minulps=2 mintok=0 x=1061.18465 logit=2983.54688 exp=515.079629 shr=5.82929541 lines=244 END +moesel call=52 dev=1 T=1 E=512 k=10 sel=79e503d24cec8ea1 minulps=10 mintok=0 x=1090.49174 logit=2982.1875 exp=544.015517 shr=5.63867059 lines=245 END +moesel call=53 dev=1 T=1 E=512 k=10 sel=8423f417266f13a9 minulps=1 mintok=0 x=941.708185 logit=3028.28125 exp=427.827993 shr=4.90885232 lines=246 END +moesel call=54 dev=1 T=1 E=512 k=10 sel=38d2a77ae773af17 minulps=2 mintok=0 x=1090.95048 logit=2902.09375 exp=691.67611 shr=6.25859495 lines=247 END +moesel call=55 dev=1 T=1 E=512 k=10 sel=e2b2fcd5dc379493 minulps=1 mintok=0 x=1110.22383 logit=2969.21875 exp=571.045208 shr=5.27463305 lines=248 END +moesel call=56 dev=1 T=1 E=512 k=10 sel=23e3af4450f40f94 minulps=5 mintok=0 x=1131.85893 logit=2973.25 exp=630.538729 shr=5.053668 lines=249 END +moesel call=57 dev=1 T=1 E=512 k=10 sel=c02b53ae5ee547bd minulps=2 mintok=0 x=1167.77365 logit=3020.34375 exp=599.142368 shr=7.05602749 lines=250 END +moesel call=58 dev=1 T=1 E=512 k=10 sel=b80e559c042768ff minulps=0 mintok=0 x=1063.17118 logit=2990.57812 exp=517.027451 shr=4.58549351 lines=251 END +moesel call=59 dev=1 T=1 E=512 k=10 sel=b978d5771b953316 minulps=2 mintok=0 x=1070.02329 logit=3021.53125 exp=477.88352 shr=3.21880716 lines=252 END +moesel call=60 dev=1 T=1 E=512 k=10 sel=5f147bce39d3d018 minulps=1 mintok=0 x=1108.20363 logit=3036.79688 exp=500.637898 shr=5.64360538 lines=253 END +moesel call=61 dev=1 T=1 E=512 k=10 sel=84b8a4ffe8160de6 minulps=0 mintok=0 x=998.216994 logit=3043.34375 exp=421.396003 shr=4.870466 lines=254 END +moesel call=62 dev=1 T=1 E=512 k=10 sel=9f55aa23ddc601de minulps=2 mintok=0 x=1127.07912 logit=3031.84375 exp=513.611865 shr=5.490476 lines=255 END +moesel call=63 dev=1 T=1 E=512 k=10 sel=e6abc54b43ce81a5 minulps=1 mintok=0 x=1455.34824 logit=3050.59375 exp=791.663157 shr=6.71446584 lines=256 END +moesel call=64 dev=1 T=1 E=512 k=10 sel=1255bc20aafb1161 minulps=1 mintok=0 x=1290.26098 logit=3052.90625 exp=622.36894 shr=5.60229019 lines=257 END +moesel call=65 dev=1 T=1 E=512 k=10 sel=c5ba50675c9f9d6d minulps=2 mintok=0 x=1140.36913 logit=3127.625 exp=593.306181 shr=4.68912232 lines=258 END +moesel call=66 dev=1 T=1 E=512 k=10 sel=cf9b2d4e92ee8295 minulps=2 mintok=0 x=1114.25081 logit=3188.0625 exp=577.227118 shr=4.53733855 lines=259 END +moesel call=67 dev=1 T=1 E=512 k=10 sel=7b0562becd0aba31 minulps=2 mintok=0 x=1281.4322 logit=3164.85938 exp=667.557706 shr=5.21218349 lines=260 END +moesel call=68 dev=1 T=1 E=512 k=10 sel=c8c8bfef3ac9cf11 minulps=1 mintok=0 x=1534.10805 logit=3156.46875 exp=945.605924 shr=9.54545289 lines=261 END +moesel call=69 dev=1 T=1 E=512 k=10 sel=fcd94a844c502abc minulps=0 mintok=0 x=1496.12201 logit=3201.54688 exp=848.053703 shr=7.07275957 lines=262 END +moesel call=70 dev=1 T=1 E=512 k=10 sel=ba1241b9927c42f3 minulps=5 mintok=0 x=1391.05154 logit=3276.57812 exp=828.893357 shr=4.11207919 lines=263 END +moesel call=71 dev=1 T=1 E=512 k=10 sel=5d8589e5e9b25070 minulps=5 mintok=0 x=1710.48188 logit=3154.39062 exp=1068.14446 shr=15.594517 lines=264 END +moesel call=72 dev=1 T=1 E=512 k=10 sel=91612b3fa6473fb0 minulps=1 mintok=0 x=1175.36515 logit=3137.07812 exp=584.083677 shr=5.80112762 lines=265 END +moesel call=73 dev=1 T=1 E=512 k=10 sel=b2ea85fa9a669cf8 minulps=11 mintok=0 x=1248.51615 logit=3123.1875 exp=596.386972 shr=5.07536156 lines=266 END +moesel call=74 dev=1 T=1 E=512 k=10 sel=9d9437086dced8c5 minulps=7 mintok=0 x=1149.56437 logit=3154.14062 exp=568.216715 shr=4.88735269 lines=267 END +moesel call=75 dev=1 T=1 E=512 k=10 sel=b943fedc5d203b58 minulps=10 mintok=0 x=1202.45795 logit=3088.84375 exp=639.258215 shr=5.79201026 lines=268 END +moesel call=76 dev=1 T=1 E=512 k=10 sel=000c6654b4417e5a minulps=4 mintok=0 x=1238.31239 logit=3142.3125 exp=650.707318 shr=5.53921388 lines=269 END +moesel call=77 dev=1 T=1 E=512 k=10 sel=ee80fbd9c62d5569 minulps=4 mintok=0 x=1271.90447 logit=3151.98438 exp=707.469367 shr=7.42523699 lines=270 END +moesel call=78 dev=1 T=1 E=512 k=10 sel=bd58136d2912631a minulps=1 mintok=0 x=1461.16225 logit=3129.1875 exp=884.908316 shr=7.92177817 lines=271 END +moesel call=79 dev=1 T=1 E=512 k=10 sel=eb29872faed407da minulps=4 mintok=0 x=1834.92 logit=3084.75 exp=1238.039 shr=9.50212419 lines=272 END +moesel call=80 dev=1 T=1 E=512 k=10 sel=292074bcffc58544 minulps=2 mintok=0 x=1759.32481 logit=3040.1875 exp=1064.99146 shr=9.66410056 lines=273 END +moesel call=81 dev=1 T=1 E=512 k=10 sel=4a574cc2fc7fb9fb minulps=4 mintok=0 x=1596.62545 logit=3309.0625 exp=1104.52725 shr=6.23227578 lines=274 END +moesel call=82 dev=1 T=1 E=512 k=10 sel=b3eaa9c6c6eb3130 minulps=5 mintok=0 x=1430.16452 logit=3348 exp=993.164888 shr=5.87929016 lines=275 END +moesel call=83 dev=1 T=1 E=512 k=10 sel=2bb0197f9528dae3 minulps=1 mintok=0 x=1680.11528 logit=3312.57812 exp=1237.64796 shr=6.98264772 lines=276 END +moesel call=84 dev=1 T=1 E=512 k=10 sel=cd4c4051f92b4d23 minulps=2 mintok=0 x=2020.4353 logit=3222.48438 exp=1912.06777 shr=17.9209241 lines=277 END +moesel call=85 dev=1 T=1 E=512 k=10 sel=76936aeb5eaaab51 minulps=6 mintok=0 x=1931.11175 logit=3290.15625 exp=1570.44779 shr=12.4351218 lines=278 END +moesel call=86 dev=1 T=1 E=512 k=10 sel=efc0b3e6db95dc18 minulps=0 mintok=0 x=1897.23602 logit=3249.34375 exp=2210.29989 shr=18.043677 lines=279 END +moesel call=87 dev=1 T=1 E=512 k=10 sel=2b804122fbd84b29 minulps=0 mintok=0 x=2039.54198 logit=3036.3125 exp=1692.76127 shr=26.3856548 lines=280 END +moesel call=88 dev=1 T=1 E=512 k=10 sel=c91c0963d866cf8c minulps=1 mintok=0 x=2063.36669 logit=3078.40625 exp=1645.13399 shr=23.3459053 lines=281 END +moesel call=89 dev=1 T=1 E=512 k=10 sel=6c1d2b803ded5a66 minulps=1 mintok=0 x=2106.95315 logit=3222.42188 exp=1856.79287 shr=14.4382533 lines=282 END +moesel call=90 dev=1 T=1 E=512 k=10 sel=130cc2b2b901b5c5 minulps=1 mintok=0 x=2103.03736 logit=3184 exp=2505.59213 shr=23.2879008 lines=283 END +moesel call=91 dev=1 T=1 E=512 k=10 sel=d72d542d6b6ef0ec minulps=0 mintok=0 x=1824.80401 logit=3087.8125 exp=1415.92651 shr=21.9134084 lines=284 END +moesel call=92 dev=1 T=1 E=512 k=10 sel=dc0e191167472a9a minulps=0 mintok=0 x=2065.04691 logit=3159.75 exp=1687.67223 shr=18.6924855 lines=285 END +moesel call=93 dev=1 T=1 E=512 k=10 sel=0770365869cd05cc minulps=7 mintok=0 x=2202.8219 logit=3269.01562 exp=1561.783 shr=23.8309534 lines=286 END +moesel call=94 dev=1 T=1 E=512 k=10 sel=3cac9ccd67a99bfc minulps=1 mintok=0 x=2221.9064 logit=3263.76562 exp=1527.75915 shr=32.8177135 lines=287 END +moesel call=95 dev=1 T=1 E=512 k=10 sel=2873dc2ce72e08bb minulps=2 mintok=0 x=2000.9953 logit=3182.67188 exp=1438.48454 shr=137.679196 lines=288 END +moesel call=96 dev=1 T=1 E=512 k=10 sel=0148dfd4911f327c minulps=0 mintok=0 x=711.156308 logit=3388.53125 exp=609.459642 shr=38.3518281 lines=289 END +moesel call=97 dev=1 T=1 E=512 k=10 sel=2a8a39b413aa2418 minulps=2 mintok=0 x=812.718665 logit=3055.0625 exp=457.39149 shr=5.11571234 lines=290 END +moesel call=98 dev=1 T=1 E=512 k=10 sel=7091a270949c8cf6 minulps=0 mintok=0 x=793.294253 logit=3057.59375 exp=304.46167 shr=7.27015826 lines=291 END +moesel call=99 dev=1 T=1 E=512 k=10 sel=835f7d56dda2a4d0 minulps=2 mintok=0 x=1148.8844 logit=3032.5625 exp=562.579764 shr=3.96235645 lines=292 END +moesel call=100 dev=1 T=1 E=512 k=10 sel=ca64ce98d54c003b minulps=2 mintok=0 x=1151.14993 logit=2937.5 exp=528.301102 shr=5.20830437 lines=293 END +moesel call=101 dev=1 T=1 E=512 k=10 sel=82c9513497c8dab5 minulps=2 mintok=0 x=1052.03611 logit=3041.64062 exp=469.374281 shr=8.19047566 lines=294 END +moesel call=102 dev=1 T=1 E=512 k=10 sel=fd2e2488b48a2177 minulps=2 mintok=0 x=1202.21154 logit=3076.23438 exp=646.929276 shr=6.38832703 lines=295 END +moesel call=103 dev=1 T=1 E=512 k=10 sel=b40e07a81fdf978f minulps=0 mintok=0 x=1098.26801 logit=3023.5625 exp=524.833726 shr=5.2388922 lines=296 END +moesel call=104 dev=1 T=1 E=512 k=10 sel=24e17d809db8c11a minulps=2 mintok=0 x=1118.6887 logit=3037.42188 exp=601.489528 shr=6.11610687 lines=297 END +moesel call=105 dev=1 T=1 E=512 k=10 sel=67c8568c46a49a4a minulps=0 mintok=0 x=1157.52346 logit=3072.96875 exp=540.229177 shr=5.98809193 lines=298 END +moesel call=106 dev=1 T=1 E=512 k=10 sel=a815754084dc4839 minulps=1 mintok=0 x=1068.68475 logit=3096.5625 exp=490.029281 shr=3.74977484 lines=299 END +moesel call=107 dev=1 T=1 E=512 k=10 sel=e31b42713f519e1a minulps=3 mintok=0 x=1055.69063 logit=3066.85938 exp=461.200665 shr=3.00659831 lines=300 END +moesel call=108 dev=1 T=1 E=512 k=10 sel=83b2d2fe7b7d991c minulps=2 mintok=0 x=1068.40184 logit=3072.42188 exp=498.168385 shr=6.41813868 lines=301 END +moesel call=109 dev=1 T=1 E=512 k=10 sel=7f5cc3e8d5161609 minulps=2 mintok=0 x=959.089637 logit=3026.51562 exp=419.919528 shr=6.11775106 lines=302 END +moesel call=110 dev=1 T=1 E=512 k=10 sel=98305dfd7917367d minulps=2 mintok=0 x=1192.49512 logit=3149.20312 exp=638.70187 shr=5.31779092 lines=303 END +moesel call=111 dev=1 T=1 E=512 k=10 sel=38ddb3334bf52011 minulps=12 mintok=0 x=1578.1546 logit=3117.59375 exp=927.968217 shr=7.89197639 lines=304 END +moesel call=112 dev=1 T=1 E=512 k=10 sel=d14c90fc283815e4 minulps=1 mintok=0 x=1321.545 logit=3179.85938 exp=670.672803 shr=4.59513244 lines=305 END +moesel call=113 dev=1 T=1 E=512 k=10 sel=5bf394db409c91de minulps=8 mintok=0 x=1084.32861 logit=3069.04688 exp=470.568587 shr=4.149268 lines=306 END +moesel call=114 dev=1 T=1 E=512 k=10 sel=db2f2b90ff1ca8c4 minulps=4 mintok=0 x=1062.79936 logit=3204.85938 exp=503.707939 shr=3.60143873 lines=307 END +moesel call=115 dev=1 T=1 E=512 k=10 sel=bf85a0dd154e12b1 minulps=1 mintok=0 x=1186.4054 logit=3132.625 exp=571.425364 shr=5.03959958 lines=308 END +moesel call=116 dev=1 T=1 E=512 k=10 sel=bd629352fffafcc6 minulps=0 mintok=0 x=1474.49221 logit=3201.65625 exp=814.89747 shr=7.91446456 lines=309 END +moesel call=117 dev=1 T=1 E=512 k=10 sel=4625653a1b19e96c minulps=2 mintok=0 x=1436.522 logit=3280.26562 exp=788.017591 shr=4.93058806 lines=310 END +moesel call=118 dev=1 T=1 E=512 k=10 sel=38c8a9ac128de4be minulps=2 mintok=0 x=1387.48718 logit=3292.9375 exp=765.92464 shr=3.74038844 lines=311 END +moesel call=119 dev=1 T=1 E=512 k=10 sel=9e60c6524f345583 minulps=0 mintok=0 x=1690.051 logit=3135.51562 exp=939.131987 shr=10.7073113 lines=312 END +moesel call=120 dev=1 T=1 E=512 k=10 sel=5fad91ec51de038f minulps=2 mintok=0 x=1188.62005 logit=3185.76562 exp=560.85911 shr=4.86120316 lines=313 END +moesel call=121 dev=1 T=1 E=512 k=10 sel=a033bbe69a732965 minulps=3 mintok=0 x=1223.35445 logit=3199.75 exp=606.992688 shr=4.56049468 lines=314 END +moesel call=122 dev=1 T=1 E=512 k=10 sel=0c39cc88655373dd minulps=2 mintok=0 x=1107.90326 logit=3252.17188 exp=558.658191 shr=4.66765702 lines=315 END +moesel call=123 dev=1 T=1 E=512 k=10 sel=1b10475baf5406c8 minulps=7 mintok=0 x=1193.38707 logit=3228.53125 exp=610.211403 shr=5.04859781 lines=316 END +moesel call=124 dev=1 T=1 E=512 k=10 sel=6bcf0c3c28679919 minulps=1 mintok=0 x=1201.07731 logit=3259.875 exp=585.112188 shr=5.20723253 lines=317 END +moesel call=125 dev=1 T=1 E=512 k=10 sel=1be218557f73f10e minulps=0 mintok=0 x=1197.28618 logit=3180.78125 exp=717.723588 shr=6.28841968 lines=318 END +moesel call=126 dev=1 T=1 E=512 k=10 sel=ab608f161fba4564 minulps=2 mintok=0 x=1454.99889 logit=3286.875 exp=989.59422 shr=9.29866271 lines=319 END +moesel call=127 dev=1 T=1 E=512 k=10 sel=2b9e8eeb37f55c79 minulps=1 mintok=0 x=1942.50397 logit=3202.98438 exp=1258.63799 shr=11.0705444 lines=320 END +moesel call=128 dev=1 T=1 E=512 k=10 sel=749568181cf50c4f minulps=6 mintok=0 x=1783.84903 logit=3163.57812 exp=1107.94875 shr=9.83864037 lines=321 END +moesel call=129 dev=1 T=1 E=512 k=10 sel=45d1e46900535c92 minulps=13 mintok=0 x=1534.101 logit=3155.03125 exp=1024.5957 shr=8.38383763 lines=322 END +moesel call=130 dev=1 T=1 E=512 k=10 sel=7ecefc81d491db2f minulps=1 mintok=0 x=1341.92368 logit=3231.20312 exp=791.356913 shr=5.80059241 lines=323 END +moesel call=131 dev=1 T=1 E=512 k=10 sel=5deb7233735a17d6 minulps=2 mintok=0 x=1643.58147 logit=3183.79688 exp=1020.21192 shr=8.19191471 lines=324 END +moesel call=132 dev=1 T=1 E=512 k=10 sel=bccd379b9a29dc4b minulps=2 mintok=0 x=1930.93375 logit=3198.125 exp=1744.26319 shr=22.8635413 lines=325 END +moesel call=133 dev=1 T=1 E=512 k=10 sel=0307bb7dc110b2b1 minulps=0 mintok=0 x=1780.71473 logit=3185.17188 exp=1323.80137 shr=11.6337567 lines=326 END +moesel call=134 dev=1 T=1 E=512 k=10 sel=823351c2efeb4459 minulps=1 mintok=0 x=1841.61302 logit=3173.40625 exp=1866.47121 shr=15.1784789 lines=327 END +moesel call=135 dev=1 T=1 E=512 k=10 sel=fe47f81894317833 minulps=2 mintok=0 x=2051.44921 logit=3084.29688 exp=1859.42941 shr=19.9916793 lines=328 END +moesel call=136 dev=1 T=1 E=512 k=10 sel=4caad7e0c0a547a3 minulps=0 mintok=0 x=2086.01662 logit=3108.67188 exp=1556.17088 shr=14.7403516 lines=329 END +moesel call=137 dev=1 T=1 E=512 k=10 sel=c8237987eca74795 minulps=2 mintok=0 x=2162.5299 logit=3213.9375 exp=1971.4318 shr=10.2028748 lines=330 END +moesel call=138 dev=1 T=1 E=512 k=10 sel=051106c8056901f3 minulps=0 mintok=0 x=2246.8277 logit=3174.71875 exp=1949.91024 shr=19.8890601 lines=331 END +moesel call=139 dev=1 T=1 E=512 k=10 sel=97a75e8e00d0b342 minulps=2 mintok=0 x=1882.35801 logit=3092.54688 exp=1340.40161 shr=18.3256835 lines=332 END +moesel call=140 dev=1 T=1 E=512 k=10 sel=713b8ce875e2c311 minulps=1 mintok=0 x=2155.21587 logit=3210.75 exp=1659.56717 shr=21.6259448 lines=333 END +moesel call=141 dev=1 T=1 E=512 k=10 sel=c367f92089653128 minulps=0 mintok=0 x=2231.77249 logit=3202.84375 exp=1724.72208 shr=29.3759355 lines=334 END +moesel call=142 dev=1 T=1 E=512 k=10 sel=ac6441082a2c0801 minulps=2 mintok=0 x=2130.89023 logit=3145.03125 exp=1735.93224 shr=34.0439154 lines=335 END +moesel call=143 dev=1 T=1 E=512 k=10 sel=ef919b7e56630278 minulps=1 mintok=0 x=1974.69898 logit=3181.9375 exp=1722.13843 shr=144.74768 lines=336 END +moesel call=144 dev=1 T=1 E=512 k=10 sel=09c0aa1fa6c8c877 minulps=0 mintok=0 x=709.483277 logit=3420.57812 exp=589.078646 shr=44.0248033 lines=337 END +moesel call=145 dev=1 T=1 E=512 k=10 sel=a916136dafb38ba5 minulps=2 mintok=0 x=805.992065 logit=3073.1875 exp=521.093776 shr=5.51094551 lines=338 END +moesel call=146 dev=1 T=1 E=512 k=10 sel=c3ae50cfa21dff8e minulps=0 mintok=0 x=834.956304 logit=3038.53125 exp=384.249757 shr=8.0188053 lines=339 END +moesel call=147 dev=1 T=1 E=512 k=10 sel=e9f1a22d10c9b363 minulps=1 mintok=0 x=1167.29771 logit=3167.39062 exp=533.255623 shr=3.74983719 lines=340 END +moesel call=148 dev=1 T=1 E=512 k=10 sel=a07ce5b4d7e4c0f4 minulps=0 mintok=0 x=1151.53777 logit=3034.25 exp=610.441006 shr=6.48309856 lines=341 END +moesel call=149 dev=1 T=1 E=512 k=10 sel=b19205a95b25a5a3 minulps=0 mintok=0 x=1082.42335 logit=3101.91406 exp=457.349422 shr=5.85124576 lines=342 END +moesel call=150 dev=1 T=1 E=512 k=10 sel=d705f8437be03c27 minulps=0 mintok=0 x=1201.75989 logit=3197 exp=640.053712 shr=7.00659662 lines=343 END +moesel call=151 dev=1 T=1 E=512 k=10 sel=55930248058bd417 minulps=1 mintok=0 x=1124.08582 logit=3105.375 exp=551.808847 shr=4.42178992 lines=344 END +moesel call=152 dev=1 T=1 E=512 k=10 sel=c8b64ec26f3ea291 minulps=1 mintok=0 x=1129.19646 logit=3045.6875 exp=561.122629 shr=4.68783407 lines=345 END +moesel call=153 dev=1 T=1 E=512 k=10 sel=ec36b67594288457 minulps=2 mintok=0 x=1172.88517 logit=3091.9375 exp=535.528177 shr=5.83415864 lines=346 END +moesel call=154 dev=1 T=1 E=512 k=10 sel=6ceb14ce65e1679f minulps=0 mintok=0 x=1051.11735 logit=3161.3125 exp=505.286261 shr=3.63972754 lines=347 END +moesel call=155 dev=1 T=1 E=512 k=10 sel=3b73bf134ebe2733 minulps=2 mintok=0 x=1057.84783 logit=3087.5 exp=455.834315 shr=3.05580962 lines=348 END +moesel call=156 dev=1 T=1 E=512 k=10 sel=d66b184040443e9e minulps=2 mintok=0 x=1101.54184 logit=3094.53125 exp=536.417945 shr=6.63558327 lines=349 END +moesel call=157 dev=1 T=1 E=512 k=10 sel=33c5d13ae10a34a4 minulps=1 mintok=0 x=979.399919 logit=3017.42188 exp=420.453636 shr=5.07466524 lines=350 END +moesel call=158 dev=1 T=1 E=512 k=10 sel=3901af792cd4064e minulps=2 mintok=0 x=1192.45647 logit=3166.73438 exp=586.611596 shr=4.74686272 lines=351 END +moesel call=159 dev=1 T=1 E=512 k=10 sel=ecea62446c550089 minulps=1 mintok=0 x=1616.97093 logit=3195.76562 exp=930.605066 shr=5.80639515 lines=352 END +moesel call=160 dev=1 T=1 E=512 k=10 sel=6dea3b8b688439cf minulps=3 mintok=0 x=1382.02513 logit=3142.1875 exp=656.488554 shr=4.61721072 lines=353 END +moesel call=161 dev=1 T=1 E=512 k=10 sel=1ab7e1e78d5fd1d8 minulps=13 mintok=0 x=1167.59768 logit=3055.20312 exp=537.666164 shr=5.2065028 lines=354 END +moesel call=162 dev=1 T=1 E=512 k=10 sel=2dcd573eb4fc01b4 minulps=1 mintok=0 x=1053.68328 logit=3182.125 exp=485.981195 shr=3.71340064 lines=355 END +moesel call=163 dev=1 T=1 E=512 k=10 sel=6b0da33229709cca minulps=0 mintok=0 x=1260.96437 logit=3168.07812 exp=612.015255 shr=4.95779561 lines=356 END +moesel call=164 dev=1 T=1 E=512 k=10 sel=c30e84a97933daad minulps=0 mintok=0 x=1504.6571 logit=3194.65625 exp=838.207693 shr=8.71455801 lines=357 END +moesel call=165 dev=1 T=1 E=512 k=10 sel=7bb39829cf126047 minulps=1 mintok=0 x=1409.34995 logit=3223.95312 exp=721.739153 shr=5.49940975 lines=358 END +moesel call=166 dev=1 T=1 E=512 k=10 sel=024eef52bb54717a minulps=3 mintok=0 x=1341.19608 logit=3351.73438 exp=705.514118 shr=4.06778384 lines=359 END +moesel call=167 dev=1 T=1 E=512 k=10 sel=eb52d195dc1e4d9d minulps=1 mintok=0 x=1621.8557 logit=3176.01562 exp=850.487498 shr=9.49747722 lines=360 END +moesel call=168 dev=1 T=1 E=512 k=10 sel=3b2fbe3418eff725 minulps=1 mintok=0 x=1190.36975 logit=3203.04688 exp=602.699007 shr=5.59183539 lines=361 END +moesel call=169 dev=1 T=1 E=512 k=10 sel=7ce39d832fb4a4c3 minulps=4 mintok=0 x=1280.78194 logit=3237.04688 exp=669.382044 shr=5.95203981 lines=362 END +moesel call=170 dev=1 T=1 E=512 k=10 sel=f8c3febace9a2e04 minulps=1 mintok=0 x=1134.26234 logit=3256.14844 exp=563.907842 shr=4.31081375 lines=363 END +moesel call=171 dev=1 T=1 E=512 k=10 sel=f1bd61005bc74eed minulps=1 mintok=0 x=1158.93305 logit=3194.48438 exp=552.877189 shr=4.51102593 lines=364 END +moesel call=172 dev=1 T=1 E=512 k=10 sel=208c78b6b20505f9 minulps=5 mintok=0 x=1209.99551 logit=3279.59375 exp=609.162565 shr=6.4372271 lines=365 END +moesel call=173 dev=1 T=1 E=512 k=10 sel=cdfc5eb8b1cdc02c minulps=10 mintok=0 x=1168.66896 logit=3203.35938 exp=700.878574 shr=6.8646412 lines=366 END +moesel call=174 dev=1 T=1 E=512 k=10 sel=97b8993e354862fe minulps=3 mintok=0 x=1437.9202 logit=3321.5625 exp=883.806022 shr=8.39914337 lines=367 END +moesel call=175 dev=1 T=1 E=512 k=10 sel=ad191e1c27426e2b minulps=1 mintok=0 x=1935.0825 logit=3242.09375 exp=1369.8418 shr=10.7598722 lines=368 END +moesel call=176 dev=1 T=1 E=512 k=10 sel=ddec8052fbe701ac minulps=1 mintok=0 x=1790.82479 logit=3217.04688 exp=1149.80673 shr=10.6727672 lines=369 END +moesel call=177 dev=1 T=1 E=512 k=10 sel=460ed40fbb28ce6e minulps=2 mintok=0 x=1555.90134 logit=3167.17188 exp=1039.83153 shr=8.08700437 lines=370 END +moesel call=178 dev=1 T=1 E=512 k=10 sel=01aec4439a260211 minulps=4 mintok=0 x=1429.73537 logit=3299.67188 exp=908.815103 shr=6.94476729 lines=371 END +moesel call=179 dev=1 T=1 E=512 k=10 sel=50154f5b367db687 minulps=2 mintok=0 x=1618.79316 logit=3195.625 exp=1038.68821 shr=9.20681597 lines=372 END +moesel call=180 dev=1 T=1 E=512 k=10 sel=dd994ce5c1673149 minulps=2 mintok=0 x=1874.35114 logit=3199.82812 exp=1654.29785 shr=23.3827562 lines=373 END +moesel call=181 dev=1 T=1 E=512 k=10 sel=3972ac1752b2eb41 minulps=0 mintok=0 x=1876.30117 logit=3315.125 exp=1407.24905 shr=11.709172 lines=374 END +moesel call=182 dev=1 T=1 E=512 k=10 sel=3f2d4a43d000d03c minulps=1 mintok=0 x=1911.34686 logit=3218.28125 exp=1845.18509 shr=14.4705107 lines=375 END +moesel call=183 dev=1 T=1 E=512 k=10 sel=8404ef6f85cd3288 minulps=0 mintok=0 x=1974.98183 logit=3112.85938 exp=1713.42116 shr=27.1394798 lines=376 END +moesel call=184 dev=1 T=1 E=512 k=10 sel=390caacb7024806b minulps=0 mintok=0 x=1954.29539 logit=3059.90625 exp=1365.72387 shr=16.5918299 lines=377 END +moesel call=185 dev=1 T=1 E=512 k=10 sel=46bbdd5729df570f minulps=1 mintok=0 x=2190.51382 logit=3202.73438 exp=1703.29756 shr=12.784988 lines=378 END +moesel call=186 dev=1 T=1 E=512 k=10 sel=fdd86d7f02a516c9 minulps=1 mintok=0 x=2351.97609 logit=3212.40625 exp=2130.96326 shr=24.0804143 lines=379 END +moesel call=187 dev=1 T=1 E=512 k=10 sel=1266f27edf621c1b minulps=0 mintok=0 x=1896.9653 logit=3094.79688 exp=1334.10299 shr=21.6554436 lines=380 END +moesel call=188 dev=1 T=1 E=512 k=10 sel=57f1e9475aff0171 minulps=0 mintok=0 x=2163.76679 logit=3197.3125 exp=1899.18018 shr=25.5749763 lines=381 END +moesel call=189 dev=1 T=1 E=512 k=10 sel=39a7a8e89337e51a minulps=4 mintok=0 x=2359.99121 logit=3273.03125 exp=1838.0856 shr=22.521865 lines=382 END +moesel call=190 dev=1 T=1 E=512 k=10 sel=861266fd7f73af51 minulps=1 mintok=0 x=2226.46057 logit=3210.28125 exp=1993.76646 shr=48.9299308 lines=383 END +moesel call=191 dev=1 T=1 E=512 k=10 sel=926e58e367c5382d minulps=5 mintok=0 x=2000.70777 logit=3205.5 exp=1690.58827 shr=161.124053 lines=384 END +moesel call=192 dev=1 T=1 E=512 k=10 sel=779637335a2e27ec minulps=0 mintok=0 x=716.147677 logit=3468.875 exp=487.75294 shr=51.390801 lines=385 END +moesel call=193 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=765.411524 logit=3042.59375 exp=447.925371 shr=5.44892272 lines=386 END +moesel call=194 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=0 mintok=0 x=790.436476 logit=3073.125 exp=368.452928 shr=6.00434699 lines=387 END +moesel call=195 dev=1 T=1 E=512 k=10 sel=715855688ae0f4c5 minulps=1 mintok=0 x=1146.69417 logit=3151.15625 exp=585.124857 shr=3.69654149 lines=388 END +moesel call=196 dev=1 T=1 E=512 k=10 sel=e75f20bfc53651de minulps=4 mintok=0 x=1099.03376 logit=2972.45312 exp=545.371254 shr=6.2964395 lines=389 END +moesel call=197 dev=1 T=1 E=512 k=10 sel=e199dea9a8bf81ec minulps=1 mintok=0 x=1101.94353 logit=3080.90625 exp=518.980939 shr=5.99029895 lines=390 END +moesel call=198 dev=1 T=1 E=512 k=10 sel=d07e4ef7dedbb018 minulps=0 mintok=0 x=1236.59581 logit=3145.20312 exp=672.002836 shr=7.11468908 lines=391 END +moesel call=199 dev=1 T=1 E=512 k=10 sel=b24da9a0b8ab6f1c minulps=0 mintok=0 x=1121.32949 logit=3087.5 exp=562.042839 shr=6.24267779 lines=392 END +moesel call=200 dev=1 T=1 E=512 k=10 sel=bf0a220efb56ee89 minulps=0 mintok=0 x=1105.57162 logit=3081.57812 exp=557.250946 shr=5.51314575 lines=393 END +moesel call=201 dev=1 T=1 E=512 k=10 sel=325e66aef4c1aa8a minulps=2 mintok=0 x=1177.02908 logit=3086.60938 exp=541.972698 shr=6.67578374 lines=394 END +moesel call=202 dev=1 T=1 E=512 k=10 sel=d2f4cd6d11682004 minulps=1 mintok=0 x=1060.25186 logit=3121.77344 exp=501.771266 shr=3.73486739 lines=395 END +moesel call=203 dev=1 T=1 E=512 k=10 sel=89bf9a12d44a0482 minulps=4 mintok=0 x=1063.31313 logit=3036.1875 exp=428.680316 shr=3.30325386 lines=396 END +moesel call=204 dev=1 T=1 E=512 k=10 sel=f3b171608a86ada7 minulps=3 mintok=0 x=1110.2053 logit=3080.90625 exp=539.927993 shr=6.87042566 lines=397 END +moesel call=205 dev=1 T=1 E=512 k=10 sel=5a340e8c3bc82a78 minulps=0 mintok=0 x=975.890717 logit=3075.875 exp=431.624742 shr=5.38765036 lines=398 END +moesel call=206 dev=1 T=1 E=512 k=10 sel=cb068b058d469c85 minulps=6 mintok=0 x=1176.68664 logit=3175.10938 exp=616.773291 shr=5.05151851 lines=399 END +moesel call=207 dev=1 T=1 E=512 k=10 sel=314184de6bfb7963 minulps=1 mintok=0 x=1546.15161 logit=3148.78125 exp=927.606893 shr=5.13259371 lines=400 END +moesel call=208 dev=1 T=1 E=512 k=10 sel=8b82a962f008f00e minulps=1 mintok=0 x=1366.02425 logit=3048.25 exp=682.192104 shr=3.56387435 lines=401 END +moesel call=209 dev=1 T=1 E=512 k=10 sel=120081ac18996dac minulps=0 mintok=0 x=1167.5068 logit=3007.96875 exp=524.051682 shr=4.2190643 lines=402 END +moesel call=210 dev=1 T=1 E=512 k=10 sel=25a8a4bc71d12bfa minulps=3 mintok=0 x=1100.30862 logit=3195.85938 exp=502.684609 shr=3.75043126 lines=403 END +moesel call=211 dev=1 T=1 E=512 k=10 sel=001509a3e0b4ce7e minulps=1 mintok=0 x=1287.11083 logit=3201.375 exp=613.999208 shr=4.0550633 lines=404 END +moesel call=212 dev=1 T=1 E=512 k=10 sel=41744aa8dbe9161f minulps=1 mintok=0 x=1570.91101 logit=3210.96875 exp=933.894274 shr=10.2405203 lines=405 END +moesel call=213 dev=1 T=1 E=512 k=10 sel=4287716436e44132 minulps=2 mintok=0 x=1437.59795 logit=3186.28125 exp=735.577279 shr=5.56163217 lines=406 END +moesel call=214 dev=1 T=1 E=512 k=10 sel=5dea9924948c0e3c minulps=0 mintok=0 x=1385.44258 logit=3246.20312 exp=708.992863 shr=3.78521363 lines=407 END +moesel call=215 dev=1 T=1 E=512 k=10 sel=9458619ca2d1b7a1 minulps=1 mintok=0 x=1692.75305 logit=3189.625 exp=944.580063 shr=10.9508233 lines=408 END +moesel call=216 dev=1 T=1 E=512 k=10 sel=24fdfa6aa86a5f52 minulps=5 mintok=0 x=1224.17387 logit=3162.39062 exp=613.257584 shr=5.39118691 lines=409 END +moesel call=217 dev=1 T=1 E=512 k=10 sel=f16f12b2e4688234 minulps=2 mintok=0 x=1306.69958 logit=3199.79688 exp=639.83127 shr=6.10713226 lines=410 END +moesel call=218 dev=1 T=1 E=512 k=10 sel=ac4f5ffd3e72d566 minulps=5 mintok=0 x=1143.23705 logit=3216.25781 exp=561.018667 shr=4.16465363 lines=411 END +moesel call=219 dev=1 T=1 E=512 k=10 sel=38dafeab32fd4282 minulps=1 mintok=0 x=1198.31271 logit=3162.76562 exp=592.81153 shr=4.26553735 lines=412 END +moesel call=220 dev=1 T=1 E=512 k=10 sel=d4e2b8a9288e46db minulps=2 mintok=0 x=1257.38946 logit=3255.40625 exp=696.537776 shr=5.32452663 lines=413 END +moesel call=221 dev=1 T=1 E=512 k=10 sel=2f65244a0856f622 minulps=1 mintok=0 x=1229.23446 logit=3239.89062 exp=690.785637 shr=7.22456931 lines=414 END +moesel call=222 dev=1 T=1 E=512 k=10 sel=fa0039720647820e minulps=2 mintok=0 x=1489.03941 logit=3324.375 exp=885.436165 shr=7.73208681 lines=415 END +moesel call=223 dev=1 T=1 E=512 k=10 sel=cf39959fe1a82ffc minulps=8 mintok=0 x=1920.93703 logit=3248.79688 exp=1267.45386 shr=9.17116281 lines=416 END +moesel call=224 dev=1 T=1 E=512 k=10 sel=a8f3ea369e9e86d6 minulps=5 mintok=0 x=1808.22214 logit=3195.85938 exp=1095.51868 shr=8.29888845 lines=417 END +moesel call=225 dev=1 T=1 E=512 k=10 sel=95cabe388a3fea46 minulps=1 mintok=0 x=1533.78564 logit=3201.78125 exp=944.96525 shr=7.15416351 lines=418 END +moesel call=226 dev=1 T=1 E=512 k=10 sel=aa45077b077b6962 minulps=2 mintok=0 x=1425.75299 logit=3382.39062 exp=853.198978 shr=4.64491395 lines=419 END +moesel call=227 dev=1 T=1 E=512 k=10 sel=5f6726bda4071304 minulps=1 mintok=0 x=1709.32 logit=3314.23438 exp=1157.89745 shr=8.22336807 lines=420 END +moesel call=228 dev=1 T=1 E=512 k=10 sel=30cb53ea52bfef17 minulps=1 mintok=0 x=2139.26325 logit=3304.17188 exp=1901.37701 shr=20.5340446 lines=421 END +moesel call=229 dev=1 T=1 E=512 k=10 sel=3da639862025f4d6 minulps=2 mintok=0 x=1999.99795 logit=3264.89062 exp=1386.41496 shr=11.2634649 lines=422 END +moesel call=230 dev=1 T=1 E=512 k=10 sel=cb19c87acd501fc0 minulps=0 mintok=0 x=1938.46453 logit=3120.98438 exp=1874.35078 shr=15.390727 lines=423 END +moesel call=231 dev=1 T=1 E=512 k=10 sel=b04de2f650660194 minulps=0 mintok=0 x=2085.96049 logit=3063.39062 exp=1603.72511 shr=21.0686349 lines=424 END +moesel call=232 dev=1 T=1 E=512 k=10 sel=e6d6f8a353fc5f0b minulps=1 mintok=0 x=2178.48807 logit=3108.0625 exp=1554.84994 shr=15.8733611 lines=425 END +moesel call=233 dev=1 T=1 E=512 k=10 sel=027cb00b19eaecc9 minulps=2 mintok=0 x=2198.12783 logit=3227.64062 exp=1867.98647 shr=11.2329605 lines=426 END +moesel call=234 dev=1 T=1 E=512 k=10 sel=89dfd7a13488d3e8 minulps=2 mintok=0 x=2255.41025 logit=3256.25 exp=1868.25884 shr=19.0055339 lines=427 END +moesel call=235 dev=1 T=1 E=512 k=10 sel=f8a7a5bd176b326e minulps=4 mintok=0 x=1909.94999 logit=3164.17188 exp=1401.34578 shr=18.5656592 lines=428 END +moesel call=236 dev=1 T=1 E=512 k=10 sel=b7744dee46c05f1a minulps=1 mintok=0 x=2226.24713 logit=3248.5625 exp=1988.42235 shr=23.1235812 lines=429 END +moesel call=237 dev=1 T=1 E=512 k=10 sel=762dcd493bb39ada minulps=1 mintok=0 x=2310.78488 logit=3242.46875 exp=1783.5975 shr=28.7035796 lines=430 END +moesel call=238 dev=1 T=1 E=512 k=10 sel=0dde70617e68873f minulps=5 mintok=0 x=2169.89524 logit=3210.82812 exp=1672.0583 shr=38.2316203 lines=431 END +moesel call=239 dev=1 T=1 E=512 k=10 sel=281cca5db19787bc minulps=0 mintok=0 x=2013.20563 logit=3224.53125 exp=1636.04632 shr=149.592793 lines=432 END +moesel call=240 dev=1 T=1 E=512 k=10 sel=61b49097fc368a4f minulps=1 mintok=0 x=720.28059 logit=3485.40625 exp=483.675413 shr=54.8119662 lines=433 END +moesel call=241 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=771.935771 logit=3070.79688 exp=417.276456 shr=5.94142606 lines=434 END +moesel call=242 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=3 mintok=0 x=787.540872 logit=3067.625 exp=402.074727 shr=7.02242934 lines=435 END +moesel call=243 dev=1 T=1 E=512 k=10 sel=fcadc8fa2617fb34 minulps=2 mintok=0 x=1124.66685 logit=3166.34375 exp=580.25582 shr=3.28912977 lines=436 END +moesel call=244 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1082.60521 logit=2976.375 exp=527.938656 shr=5.48257715 lines=437 END +moesel call=245 dev=1 T=1 E=512 k=10 sel=6c3ea50f741c1689 minulps=3 mintok=0 x=1125.42378 logit=3122.9375 exp=511.366771 shr=4.98313789 lines=438 END +moesel call=246 dev=1 T=1 E=512 k=10 sel=0aa7734f47636857 minulps=1 mintok=0 x=1323.94484 logit=3165.26562 exp=671.581911 shr=8.14461842 lines=439 END +moesel call=247 dev=1 T=1 E=512 k=10 sel=8e15ae330207bff0 minulps=1 mintok=0 x=1173.52005 logit=3075.85938 exp=571.254446 shr=6.22336972 lines=440 END +moesel call=248 dev=1 T=1 E=512 k=10 sel=0c140a8e49a781fd minulps=3 mintok=0 x=1183.84978 logit=3052.59375 exp=604.445688 shr=6.84836599 lines=441 END +moesel call=249 dev=1 T=1 E=512 k=10 sel=db717a9b30383fe4 minulps=3 mintok=0 x=1211.28881 logit=3059.46875 exp=580.742396 shr=5.14751594 lines=442 END +moesel call=250 dev=1 T=1 E=512 k=10 sel=59619e2e9965e888 minulps=1 mintok=0 x=1083.64493 logit=3153.60938 exp=503.252649 shr=3.60288797 lines=443 END +moesel call=251 dev=1 T=1 E=512 k=10 sel=74d1d7272204b827 minulps=5 mintok=0 x=1078.65814 logit=3063.03125 exp=474.361346 shr=2.34464275 lines=444 END +moesel call=252 dev=1 T=1 E=512 k=10 sel=72c14e01f28ffe9a minulps=0 mintok=0 x=1123.4959 logit=3052.09375 exp=527.66608 shr=5.18539597 lines=445 END +moesel call=253 dev=1 T=1 E=512 k=10 sel=d386a2016da631fd minulps=3 mintok=0 x=996.801906 logit=3047.92188 exp=417.467618 shr=4.93362171 lines=446 END +moesel call=254 dev=1 T=1 E=512 k=10 sel=e73cb09b45b079b4 minulps=6 mintok=0 x=1224.72177 logit=3078.6875 exp=587.41722 shr=4.98894918 lines=447 END +moesel call=255 dev=1 T=1 E=512 k=10 sel=3c461f614b5b675e minulps=1 mintok=0 x=1634.41036 logit=3118.46875 exp=900.135087 shr=8.71689782 lines=448 END +moesel call=256 dev=1 T=1 E=512 k=10 sel=f162918a3e59d49e minulps=1 mintok=0 x=1441.67168 logit=3060.29688 exp=718.891064 shr=7.18912839 lines=449 END +moesel call=257 dev=1 T=1 E=512 k=10 sel=5beb06c103212760 minulps=2 mintok=0 x=1211.66689 logit=3142.42188 exp=613.215454 shr=5.64889552 lines=450 END +moesel call=258 dev=1 T=1 E=512 k=10 sel=40423a978ab31a36 minulps=1 mintok=0 x=1087.29929 logit=3215.6875 exp=473.434067 shr=3.84199879 lines=451 END +moesel call=259 dev=1 T=1 E=512 k=10 sel=de4339573503bcd4 minulps=1 mintok=0 x=1264.18889 logit=3241.79688 exp=593.864389 shr=4.92873746 lines=452 END +moesel call=260 dev=1 T=1 E=512 k=10 sel=f790f933190f981e minulps=2 mintok=0 x=1497.17991 logit=3274.29688 exp=820.056975 shr=8.91404494 lines=453 END +moesel call=261 dev=1 T=1 E=512 k=10 sel=a6b70eb4e1969847 minulps=3 mintok=0 x=1415.73308 logit=3250.04688 exp=680.218217 shr=4.77076492 lines=454 END +moesel call=262 dev=1 T=1 E=512 k=10 sel=7b7a3168b2ff06a2 minulps=7 mintok=0 x=1350.50629 logit=3259.57812 exp=654.255479 shr=3.91195978 lines=455 END +moesel call=263 dev=1 T=1 E=512 k=10 sel=f5590947f667141f minulps=1 mintok=0 x=1614.07453 logit=3184.35938 exp=860.567514 shr=7.31600365 lines=456 END +moesel call=264 dev=1 T=1 E=512 k=10 sel=ef6c35ec164a8a49 minulps=0 mintok=0 x=1205.80305 logit=3202.09375 exp=586.778015 shr=5.23748323 lines=457 END +moesel call=265 dev=1 T=1 E=512 k=10 sel=fb96ce369453d56b minulps=2 mintok=0 x=1305.60037 logit=3266.23438 exp=606.43976 shr=5.04525727 lines=458 END +moesel call=266 dev=1 T=1 E=512 k=10 sel=933d7afaff8cc437 minulps=10 mintok=0 x=1109.77298 logit=3258.35156 exp=548.58202 shr=3.68036258 lines=459 END +moesel call=267 dev=1 T=1 E=512 k=10 sel=7cbbfaa10ca363ad minulps=2 mintok=0 x=1186.14458 logit=3144.14062 exp=535.223339 shr=5.36935256 lines=460 END +moesel call=268 dev=1 T=1 E=512 k=10 sel=100a651971679b44 minulps=2 mintok=0 x=1243.57518 logit=3286.28125 exp=637.414101 shr=6.02404639 lines=461 END +moesel call=269 dev=1 T=1 E=512 k=10 sel=a4fdb20db9cc56a8 minulps=0 mintok=0 x=1234.17061 logit=3215.42188 exp=652.155181 shr=9.21329071 lines=462 END +moesel call=270 dev=1 T=1 E=512 k=10 sel=d551b1ca5da9dbb5 minulps=1 mintok=0 x=1431.30979 logit=3378.96875 exp=738.425532 shr=7.25745194 lines=463 END +moesel call=271 dev=1 T=1 E=512 k=10 sel=dcef15817b363caa minulps=2 mintok=0 x=1662.58431 logit=3247.28125 exp=987.379433 shr=11.751852 lines=464 END +moesel call=272 dev=1 T=1 E=512 k=10 sel=1860028c3edaf876 minulps=1 mintok=0 x=1540.5087 logit=3175.40625 exp=837.511548 shr=10.8390061 lines=465 END +moesel call=273 dev=1 T=1 E=512 k=10 sel=1186c47d631d5f82 minulps=0 mintok=0 x=1431.05815 logit=3268.01562 exp=800.615634 shr=6.03382078 lines=466 END +moesel call=274 dev=1 T=1 E=512 k=10 sel=26c585a5a9a96de1 minulps=4 mintok=0 x=1368.66555 logit=3365.40625 exp=784.150686 shr=5.20184667 lines=467 END +moesel call=275 dev=1 T=1 E=512 k=10 sel=5447e5522c9978ba minulps=0 mintok=0 x=1576.16108 logit=3258.95312 exp=980.637357 shr=7.11631892 lines=468 END +moesel call=276 dev=1 T=1 E=512 k=10 sel=52face1eca90650c minulps=2 mintok=0 x=1852.7599 logit=3289.90625 exp=1242.96738 shr=17.6701261 lines=469 END +moesel call=277 dev=1 T=1 E=512 k=10 sel=c37a90b706905352 minulps=0 mintok=0 x=1826.76319 logit=3325.07812 exp=1238.72894 shr=11.1069868 lines=470 END +moesel call=278 dev=1 T=1 E=512 k=10 sel=72cfbe3454d4b6de minulps=1 mintok=0 x=1881.6622 logit=3186.0625 exp=1661.28122 shr=13.5915908 lines=471 END +moesel call=279 dev=1 T=1 E=512 k=10 sel=bfd9f1aaa5d844d7 minulps=1 mintok=0 x=1906.10385 logit=3046.67188 exp=1310.62947 shr=21.4008919 lines=472 END +moesel call=280 dev=1 T=1 E=512 k=10 sel=4f1e914b26764374 minulps=0 mintok=0 x=2037.13792 logit=3109.51562 exp=1402.99566 shr=14.5560947 lines=473 END +moesel call=281 dev=1 T=1 E=512 k=10 sel=5fba2b1dbb53abc0 minulps=1 mintok=0 x=1989.86975 logit=3266 exp=1452.78641 shr=9.4343206 lines=474 END +moesel call=282 dev=1 T=1 E=512 k=10 sel=18b6530e503683e3 minulps=4 mintok=0 x=2156.46802 logit=3193.28125 exp=1845.50453 shr=22.3346339 lines=475 END +moesel call=283 dev=1 T=1 E=512 k=10 sel=89b0aadca1054264 minulps=0 mintok=0 x=1886.75411 logit=3152.70312 exp=1358.00446 shr=18.8158455 lines=476 END +moesel call=284 dev=1 T=1 E=512 k=10 sel=b9719722eba427d0 minulps=1 mintok=0 x=2217.40372 logit=3220.92188 exp=1993.46938 shr=21.7836 lines=477 END +moesel call=285 dev=1 T=1 E=512 k=10 sel=71bbaf05cdb5130d minulps=0 mintok=0 x=2319.32773 logit=3235.875 exp=1773.67931 shr=25.0276712 lines=478 END +moesel call=286 dev=1 T=1 E=512 k=10 sel=0b57979055d8fdca minulps=1 mintok=0 x=2185.67904 logit=3188.79688 exp=1739.53171 shr=39.9090071 lines=479 END +moesel call=287 dev=1 T=1 E=512 k=10 sel=70924589f171ffe4 minulps=1 mintok=0 x=2032.05483 logit=3193.03125 exp=1692.53224 shr=153.420908 lines=480 END +moesel call=288 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=1 mintok=0 x=720.856046 logit=3492.75 exp=470.546575 shr=56.0767549 lines=481 END +moesel call=289 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=0 mintok=0 x=770.703693 logit=3051.84375 exp=392.605082 shr=6.61083775 lines=482 END +moesel call=290 dev=1 T=1 E=512 k=10 sel=19a4aa22fd55ceb9 minulps=0 mintok=0 x=797.399337 logit=3060.17188 exp=449.968985 shr=7.74540238 lines=483 END +moesel call=291 dev=1 T=1 E=512 k=10 sel=2a01e5c40d6cc0a3 minulps=1 mintok=0 x=1130.79295 logit=3148.03125 exp=630.561784 shr=3.20717122 lines=484 END +moesel call=292 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1107.32255 logit=3057.42188 exp=612.944122 shr=4.60172829 lines=485 END +moesel call=293 dev=1 T=1 E=512 k=10 sel=5bfbcc85bf27e03a minulps=0 mintok=0 x=1195.62512 logit=3226.40625 exp=568.449331 shr=5.80444346 lines=486 END +moesel call=294 dev=1 T=1 E=512 k=10 sel=872e07b269673330 minulps=0 mintok=0 x=1356.65116 logit=3237.53125 exp=760.423626 shr=8.0394378 lines=487 END +moesel call=295 dev=1 T=1 E=512 k=10 sel=5ca98857c1285dfe minulps=1 mintok=0 x=1237.19989 logit=3187.35938 exp=636.745745 shr=6.45760946 lines=488 END +moesel call=296 dev=1 T=1 E=512 k=10 sel=2808381ab44ad57f minulps=0 mintok=0 x=1263.79819 logit=3167.4375 exp=663.189906 shr=8.30687244 lines=489 END +moesel call=297 dev=1 T=1 E=512 k=10 sel=1e6ef2e2bd5bbe6e minulps=1 mintok=0 x=1308.03402 logit=3105.6875 exp=655.636429 shr=6.0368379 lines=490 END +moesel call=298 dev=1 T=1 E=512 k=10 sel=6d9a04e77a545573 minulps=0 mintok=0 x=1146.06395 logit=3228.01562 exp=597.569373 shr=4.32204409 lines=491 END +moesel call=299 dev=1 T=1 E=512 k=10 sel=5386a17f0a85355d minulps=0 mintok=0 x=1177.25779 logit=3092.0625 exp=561.144001 shr=3.26688719 lines=492 END +moesel call=300 dev=1 T=1 E=512 k=10 sel=6df890014a37fbfa minulps=2 mintok=0 x=1217.40623 logit=3067.8125 exp=577.586853 shr=4.4797349 lines=493 END +moesel call=301 dev=1 T=1 E=512 k=10 sel=a7d9e1f34c1f1e03 minulps=5 mintok=0 x=1056.97793 logit=3131.34375 exp=458.905468 shr=5.41991125 lines=494 END +moesel call=302 dev=1 T=1 E=512 k=10 sel=cbcf5808f868f2ee minulps=2 mintok=0 x=1295.18274 logit=3124.60938 exp=776.900305 shr=6.12398742 lines=495 END +moesel call=303 dev=1 T=1 E=512 k=10 sel=0b1314471336dbd0 minulps=1 mintok=0 x=1615.90277 logit=3082.39062 exp=928.556103 shr=10.2991807 lines=496 END +moesel call=304 dev=1 T=1 E=512 k=10 sel=81f99954a1527b57 minulps=1 mintok=0 x=1461.03125 logit=3065.98438 exp=717.857508 shr=6.48855598 lines=497 END +moesel call=305 dev=1 T=1 E=512 k=10 sel=e3d6b1ce22c8075e minulps=6 mintok=0 x=1269.97015 logit=3140.25 exp=619.990238 shr=4.45730279 lines=498 END +moesel call=306 dev=1 T=1 E=512 k=10 sel=d157df379a8be237 minulps=2 mintok=0 x=1184.60336 logit=3220.07812 exp=606.101411 shr=3.86488661 lines=499 END +moesel call=307 dev=1 T=1 E=512 k=10 sel=2a2ee430acd400cb minulps=5 mintok=0 x=1291.61908 logit=3141.15625 exp=661.987925 shr=4.67138139 lines=500 END +moesel call=308 dev=1 T=1 E=512 k=10 sel=c05e3ebbece289a1 minulps=1 mintok=0 x=1514.79625 logit=3150.875 exp=900.476488 shr=9.86281496 lines=501 END +moesel call=309 dev=1 T=1 E=512 k=10 sel=ef5beefc010fbc10 minulps=2 mintok=0 x=1490.35327 logit=3213.45312 exp=797.527022 shr=4.82737862 lines=502 END +moesel call=310 dev=1 T=1 E=512 k=10 sel=eda1b89e68411e03 minulps=6 mintok=0 x=1397.02388 logit=3237.34375 exp=733.108261 shr=4.77679757 lines=503 END +moesel call=311 dev=1 T=1 E=512 k=10 sel=075f24b2bb439374 minulps=1 mintok=0 x=1562.3066 logit=3135.10938 exp=756.076709 shr=12.6417682 lines=504 END +moesel call=312 dev=1 T=1 E=512 k=10 sel=afa97bd21171c051 minulps=0 mintok=0 x=1236.9966 logit=3235.8125 exp=623.439627 shr=5.93616664 lines=505 END +moesel call=313 dev=1 T=1 E=512 k=10 sel=cf66e769dbb64fc5 minulps=1 mintok=0 x=1318.87473 logit=3333.09375 exp=612.478401 shr=5.13968842 lines=506 END +moesel call=314 dev=1 T=1 E=512 k=10 sel=5689cecff7b5d5fa minulps=3 mintok=0 x=1129.20906 logit=3255.76562 exp=557.015241 shr=4.12521553 lines=507 END +moesel call=315 dev=1 T=1 E=512 k=10 sel=e4e002100e756a6a minulps=1 mintok=0 x=1203.42205 logit=3248.04688 exp=514.741366 shr=4.05207102 lines=508 END +moesel call=316 dev=1 T=1 E=512 k=10 sel=6f7356e1764238df minulps=0 mintok=0 x=1261.03246 logit=3370.92188 exp=619.130554 shr=4.29648276 lines=509 END +moesel call=317 dev=1 T=1 E=512 k=10 sel=ff0e8dba386525fc minulps=1 mintok=0 x=1235.21792 logit=3276.48438 exp=639.089688 shr=6.69108815 lines=510 END +moesel call=318 dev=1 T=1 E=512 k=10 sel=c1b1d02eae132273 minulps=0 mintok=0 x=1509.74702 logit=3320.64062 exp=797.675336 shr=4.56329502 lines=511 END +moesel call=319 dev=1 T=1 E=512 k=10 sel=afa51735733f0336 minulps=2 mintok=0 x=1753.00647 logit=3180.0625 exp=1001.70105 shr=7.61484663 lines=512 END +moesel call=320 dev=1 T=1 E=512 k=10 sel=0e0a4109a6700048 minulps=3 mintok=0 x=1699.79817 logit=3207.42188 exp=911.617586 shr=6.69794951 lines=513 END +moesel call=321 dev=1 T=1 E=512 k=10 sel=a6826b4479452d20 minulps=1 mintok=0 x=1546.95507 logit=3408.875 exp=895.016724 shr=4.04376189 lines=514 END +moesel call=322 dev=1 T=1 E=512 k=10 sel=8eda5f9a5229ddbb minulps=3 mintok=0 x=1436.64717 logit=3439.375 exp=840.710471 shr=4.31456018 lines=515 END +moesel call=323 dev=1 T=1 E=512 k=10 sel=d5ad71babb6fc5c7 minulps=0 mintok=0 x=1660.51322 logit=3350.73438 exp=977.062221 shr=5.02763242 lines=516 END +moesel call=324 dev=1 T=1 E=512 k=10 sel=dd12cf205d559b5f minulps=0 mintok=0 x=2002.32102 logit=3385.6875 exp=1631.89902 shr=15.2682286 lines=517 END +moesel call=325 dev=1 T=1 E=512 k=10 sel=703f6b74f64cf6b8 minulps=2 mintok=0 x=1954.07742 logit=3413.625 exp=1309.03405 shr=7.16108526 lines=518 END +moesel call=326 dev=1 T=1 E=512 k=10 sel=992f4482448a2e26 minulps=1 mintok=0 x=1946.47071 logit=3227.64062 exp=1703.92125 shr=16.0447938 lines=519 END +moesel call=327 dev=1 T=1 E=512 k=10 sel=afad6d7161729fbf minulps=0 mintok=0 x=1820.88432 logit=3014.78125 exp=1211.98775 shr=22.5960127 lines=520 END +moesel call=328 dev=1 T=1 E=512 k=10 sel=4b4b9ea682bbc8dd minulps=2 mintok=0 x=2039.01312 logit=3099.79688 exp=1363.61623 shr=16.1566821 lines=521 END +moesel call=329 dev=1 T=1 E=512 k=10 sel=d22f8948640f21f9 minulps=4 mintok=0 x=2040.4512 logit=3155.51562 exp=1436.01293 shr=11.5047986 lines=522 END +moesel call=330 dev=1 T=1 E=512 k=10 sel=5196aed810d04733 minulps=2 mintok=0 x=2173.83652 logit=3217.875 exp=1889.13136 shr=20.5893423 lines=523 END +moesel call=331 dev=1 T=1 E=512 k=10 sel=328a3aca9d96518b minulps=2 mintok=0 x=1944.52967 logit=3112.14062 exp=1430.88715 shr=21.6439633 lines=524 END +moesel call=332 dev=1 T=1 E=512 k=10 sel=e656fd5283a60b62 minulps=5 mintok=0 x=2199.25831 logit=3197.28125 exp=1847.42863 shr=23.1650873 lines=525 END +moesel call=333 dev=1 T=1 E=512 k=10 sel=987174c07c2caf33 minulps=1 mintok=0 x=2261.67968 logit=3255.39062 exp=1604.87861 shr=24.7171497 lines=526 END +moesel call=334 dev=1 T=1 E=512 k=10 sel=988d1ff90a540b9e minulps=0 mintok=0 x=2125.40543 logit=3217.32812 exp=1541.53301 shr=33.6606202 lines=527 END +moesel call=335 dev=1 T=1 E=512 k=10 sel=86f7247dd3bef976 minulps=4 mintok=0 x=1955.88045 logit=3172.40625 exp=1570.11116 shr=131.855621 lines=528 END +moesel call=336 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=0 mintok=0 x=720.338139 logit=3493.625 exp=464.303988 shr=56.7048903 lines=529 END +moesel call=337 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=1 mintok=0 x=769.10639 logit=3054.76562 exp=382.768493 shr=6.88952735 lines=530 END +moesel call=338 dev=1 T=1 E=512 k=10 sel=da8470c4d884deef minulps=1 mintok=0 x=782.368277 logit=3049.375 exp=451.578546 shr=8.05544319 lines=531 END +moesel call=339 dev=1 T=1 E=512 k=10 sel=d71ea9ac6dcf42f0 minulps=1 mintok=0 x=1124.95057 logit=3140.40625 exp=528.174591 shr=3.15244679 lines=532 END +moesel call=340 dev=1 T=1 E=512 k=10 sel=1d8b22207c3f012b minulps=2 mintok=0 x=1105.89104 logit=3068.79688 exp=613.785158 shr=4.04117057 lines=533 END +moesel call=341 dev=1 T=1 E=512 k=10 sel=dddd50c876a89191 minulps=0 mintok=0 x=1149.2405 logit=3142.51562 exp=509.933435 shr=5.5840464 lines=534 END +moesel call=342 dev=1 T=1 E=512 k=10 sel=b0359bc2c09a1b3b minulps=3 mintok=0 x=1298.50399 logit=3192.84375 exp=713.807125 shr=7.99223403 lines=535 END +moesel call=343 dev=1 T=1 E=512 k=10 sel=6a520a326b8a25b7 minulps=0 mintok=0 x=1194.55711 logit=3172.79688 exp=590.636703 shr=6.75776705 lines=536 END +moesel call=344 dev=1 T=1 E=512 k=10 sel=7d588424fcd98bcc minulps=1 mintok=0 x=1202.00878 logit=3131.35938 exp=603.862679 shr=8.54666037 lines=537 END +moesel call=345 dev=1 T=1 E=512 k=10 sel=31626831636724dd minulps=2 mintok=0 x=1226.89993 logit=3080.14062 exp=615.39382 shr=6.22751211 lines=538 END +moesel call=346 dev=1 T=1 E=512 k=10 sel=43f426a771e6476d minulps=2 mintok=0 x=1110.41097 logit=3227.71875 exp=600.466138 shr=3.9388314 lines=539 END +moesel call=347 dev=1 T=1 E=512 k=10 sel=9255641d811fa14b minulps=1 mintok=0 x=1198.20226 logit=3134.39062 exp=546.312478 shr=3.36125358 lines=540 END +moesel call=348 dev=1 T=1 E=512 k=10 sel=16b6bff91911aacc minulps=1 mintok=0 x=1235.28627 logit=3116.42188 exp=607.790884 shr=4.53681195 lines=541 END +moesel call=349 dev=1 T=1 E=512 k=10 sel=7b00267773828180 minulps=1 mintok=0 x=1093.17051 logit=3140.20312 exp=477.441844 shr=5.05125144 lines=542 END +moesel call=350 dev=1 T=1 E=512 k=10 sel=3c5c42aa762d7ed2 minulps=0 mintok=0 x=1297.56948 logit=3115.23438 exp=800.623956 shr=4.55729351 lines=543 END +moesel call=351 dev=1 T=1 E=512 k=10 sel=a1c98a491fdf8f53 minulps=1 mintok=0 x=1630.07224 logit=3116.92188 exp=847.179979 shr=9.87963495 lines=544 END +moesel call=352 dev=1 T=1 E=512 k=10 sel=36c2b92ae5213cf2 minulps=2 mintok=0 x=1486.85555 logit=3080.14062 exp=744.776459 shr=6.95439906 lines=545 END +moesel call=353 dev=1 T=1 E=512 k=10 sel=da733df3a073e796 minulps=3 mintok=0 x=1310.67543 logit=3194.54688 exp=623.855955 shr=4.44543065 lines=546 END +moesel call=354 dev=1 T=1 E=512 k=10 sel=1a70d14b8759a6a7 minulps=0 mintok=0 x=1191.35604 logit=3210.07812 exp=561.163461 shr=3.8106307 lines=547 END +moesel call=355 dev=1 T=1 E=512 k=10 sel=c7786e018fa0b376 minulps=0 mintok=0 x=1356.12887 logit=3178.14062 exp=635.307415 shr=4.51006264 lines=548 END +moesel call=356 dev=1 T=1 E=512 k=10 sel=bb5711b42a292ab8 minulps=0 mintok=0 x=1590.40884 logit=3208.67188 exp=901.43754 shr=7.99312962 lines=549 END +moesel call=357 dev=1 T=1 E=512 k=10 sel=1b95d55c3f33adbb minulps=1 mintok=0 x=1539.14113 logit=3263.67188 exp=785.560501 shr=4.4835915 lines=550 END +moesel call=358 dev=1 T=1 E=512 k=10 sel=bc6664ad83820499 minulps=2 mintok=0 x=1384.95764 logit=3234.65625 exp=734.015604 shr=4.67692428 lines=551 END +moesel call=359 dev=1 T=1 E=512 k=10 sel=935ebb549d36348a minulps=0 mintok=0 x=1625.68447 logit=3217.82812 exp=851.083263 shr=9.25576943 lines=552 END +moesel call=360 dev=1 T=1 E=512 k=10 sel=c14c2922f69377f4 minulps=3 mintok=0 x=1264.72816 logit=3181.32812 exp=646.316954 shr=5.49239905 lines=553 END +moesel call=361 dev=1 T=1 E=512 k=10 sel=3b98798a42003bd3 minulps=4 mintok=0 x=1367.85755 logit=3292.75 exp=605.989986 shr=4.37376307 lines=554 END +moesel call=362 dev=1 T=1 E=512 k=10 sel=9807414b2758b93b minulps=1 mintok=0 x=1156.36278 logit=3238.89844 exp=602.228519 shr=3.92336114 lines=555 END +moesel call=363 dev=1 T=1 E=512 k=10 sel=e91bee73ea7d500a minulps=2 mintok=0 x=1257.65634 logit=3272.20312 exp=580.206607 shr=3.96203512 lines=556 END +moesel call=364 dev=1 T=1 E=512 k=10 sel=5bf4960145709db9 minulps=2 mintok=0 x=1299.88866 logit=3405.14062 exp=642.298995 shr=3.61211545 lines=557 END +moesel call=365 dev=1 T=1 E=512 k=10 sel=4c383640b19fd6e2 minulps=0 mintok=0 x=1260.04282 logit=3294.4375 exp=663.644972 shr=6.15692358 lines=558 END +moesel call=366 dev=1 T=1 E=512 k=10 sel=016b1b4bfb070eb2 minulps=1 mintok=0 x=1536.37145 logit=3296.45312 exp=843.175764 shr=4.8903624 lines=559 END +moesel call=367 dev=1 T=1 E=512 k=10 sel=e7350d2ead76a3c0 minulps=1 mintok=0 x=1859.7461 logit=3306.5 exp=1200.1414 shr=9.07440132 lines=560 END +moesel call=368 dev=1 T=1 E=512 k=10 sel=6ca0cd00f758e308 minulps=1 mintok=0 x=1722.59768 logit=3267.75 exp=959.635834 shr=7.49882364 lines=561 END +moesel call=369 dev=1 T=1 E=512 k=10 sel=f8f91f0b2d22aeb1 minulps=0 mintok=0 x=1562.56308 logit=3393.4375 exp=889.603898 shr=4.85634382 lines=562 END +moesel call=370 dev=1 T=1 E=512 k=10 sel=882a94af44d71095 minulps=1 mintok=0 x=1454.78902 logit=3452.9375 exp=857.448726 shr=4.25973188 lines=563 END +moesel call=371 dev=1 T=1 E=512 k=10 sel=ff64b0bf2202b4fc minulps=0 mintok=0 x=1692.0218 logit=3291.65625 exp=1019.0725 shr=6.75417233 lines=564 END +moesel call=372 dev=1 T=1 E=512 k=10 sel=9b91a180a5678cdc minulps=1 mintok=0 x=2027.62852 logit=3403.25 exp=1435.13897 shr=14.9619538 lines=565 END +moesel call=373 dev=1 T=1 E=512 k=10 sel=1429782ecd13da9a minulps=1 mintok=0 x=2013.09555 logit=3381.17188 exp=1315.70277 shr=7.95567352 lines=566 END +moesel call=374 dev=1 T=1 E=512 k=10 sel=bf3fc9cb980fbaa3 minulps=2 mintok=0 x=1951.77497 logit=3209.48438 exp=1672.33791 shr=11.5469591 lines=567 END +moesel call=375 dev=1 T=1 E=512 k=10 sel=f33e480a7a6c8c87 minulps=0 mintok=0 x=2032.46027 logit=3127.0625 exp=1221.10053 shr=18.475357 lines=568 END +moesel call=376 dev=1 T=1 E=512 k=10 sel=21d1673aca0f1f26 minulps=3 mintok=0 x=2186.27908 logit=3095.125 exp=1615.30269 shr=15.3497127 lines=569 END +moesel call=377 dev=1 T=1 E=512 k=10 sel=222ff097af306e69 minulps=4 mintok=0 x=2135.07565 logit=3275.53125 exp=1590.75785 shr=8.49492601 lines=570 END +moesel call=378 dev=1 T=1 E=512 k=10 sel=1e01f620cc0dbe4e minulps=0 mintok=0 x=2322.02223 logit=3195.76562 exp=1922.07308 shr=18.8426979 lines=571 END +moesel call=379 dev=1 T=1 E=512 k=10 sel=52bd328d002baa24 minulps=1 mintok=0 x=1975.22079 logit=3130.84375 exp=1444.17408 shr=16.6541057 lines=572 END +moesel call=380 dev=1 T=1 E=512 k=10 sel=714d4354831b9d0c minulps=0 mintok=0 x=2107.97066 logit=3165.39062 exp=1600.94059 shr=17.4046731 lines=573 END +moesel call=381 dev=1 T=1 E=512 k=10 sel=d169c968f5828ca8 minulps=0 mintok=0 x=2147.22314 logit=3235.29688 exp=1470.35388 shr=19.0868192 lines=574 END +moesel call=382 dev=1 T=1 E=512 k=10 sel=2718c25d93354ec4 minulps=0 mintok=0 x=2065.08524 logit=3180.26562 exp=1359.24738 shr=28.9335856 lines=575 END +moesel call=383 dev=1 T=1 E=512 k=10 sel=764408d7e514a33e minulps=3 mintok=0 x=1877.81024 logit=3102.1875 exp=1514.72134 shr=103.995275 lines=576 END diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt new file mode 100644 index 000000000..f6fa37b49 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/digests-CUDA-PROD.txt @@ -0,0 +1,384 @@ +moesel call=0 dev=1 T=5 E=512 k=10 sel=5bbd84f0b21b362b minulps=0 mintok=4 x=3615.62777 logit=17083.2891 exp=2527.65707 shr=250.743384 lines=5 END +moesel call=1 dev=1 T=5 E=512 k=10 sel=993cb7ac2f9ebaf3 minulps=0 mintok=1 x=3778.69921 logit=15553.9531 exp=2031.5105 shr=34.0787401 lines=10 END +moesel call=2 dev=1 T=5 E=512 k=10 sel=6c216b6b2cfe27d8 minulps=0 mintok=2 x=3841.05556 logit=15788.7969 exp=1789.27763 shr=52.0076308 lines=15 END +moesel call=3 dev=1 T=5 E=512 k=10 sel=8f7b3342cf4416a7 minulps=0 mintok=4 x=4867.96077 logit=15090.2969 exp=2596.98063 shr=37.9476791 lines=20 END +moesel call=4 dev=1 T=5 E=512 k=10 sel=4537bad749058f33 minulps=1 mintok=0 x=5355.29174 logit=15451.1719 exp=3073.73295 shr=47.5011171 lines=25 END +moesel call=5 dev=1 T=5 E=512 k=10 sel=9158e7125f290134 minulps=0 mintok=4 x=4456.92829 logit=15641.9844 exp=2040.0845 shr=34.254536 lines=30 END +moesel call=6 dev=1 T=5 E=512 k=10 sel=82e18d30854928c2 minulps=0 mintok=0 x=5469.83739 logit=15914.7188 exp=3136.38253 shr=47.063142 lines=35 END +moesel call=7 dev=1 T=5 E=512 k=10 sel=d94a6478b8761de3 minulps=0 mintok=2 x=5334.0979 logit=15909.9688 exp=2703.35417 shr=44.1492911 lines=40 END +moesel call=8 dev=1 T=5 E=512 k=10 sel=372c48dfeade5c01 minulps=0 mintok=2 x=5380.46048 logit=15765.6094 exp=3055.85262 shr=28.1771415 lines=45 END +moesel call=9 dev=1 T=5 E=512 k=10 sel=496b7f503ecc0638 minulps=0 mintok=0 x=5517.20036 logit=15662.5 exp=2956.27156 shr=34.5520656 lines=50 END +moesel call=10 dev=1 T=5 E=512 k=10 sel=b14a7db9158b4e90 minulps=0 mintok=0 x=4967.56581 logit=15753.2188 exp=2433.28837 shr=27.2410728 lines=55 END +moesel call=11 dev=1 T=5 E=512 k=10 sel=f068ea276978664c minulps=0 mintok=0 x=4870.5168 logit=15590.9219 exp=2156.46224 shr=18.6210697 lines=60 END +moesel call=12 dev=1 T=5 E=512 k=10 sel=d7e678c7724801db minulps=0 mintok=0 x=4971.30472 logit=15636.0938 exp=2244.17018 shr=26.6883839 lines=65 END +moesel call=13 dev=1 T=5 E=512 k=10 sel=7fce6e907d8d40a9 minulps=0 mintok=0 x=4645.09897 logit=15943.6406 exp=2044.70843 shr=27.4566855 lines=70 END +moesel call=14 dev=1 T=5 E=512 k=10 sel=cec11c8b5014de56 minulps=1 mintok=4 x=5259.56363 logit=15817.8125 exp=2911.1454 shr=22.834866 lines=75 END +moesel call=15 dev=1 T=5 E=512 k=10 sel=88cb94c69381a618 minulps=0 mintok=0 x=6750.75098 logit=15817.3438 exp=3967.80807 shr=29.9183135 lines=80 END +moesel call=16 dev=1 T=5 E=512 k=10 sel=3e7f3d84739b8124 minulps=0 mintok=3 x=6160.80079 logit=15761.7188 exp=3181.57952 shr=28.3739225 lines=85 END +moesel call=17 dev=1 T=5 E=512 k=10 sel=cb0c0f7107d1b9f1 minulps=0 mintok=0 x=5372.15441 logit=15998.7812 exp=2587.95085 shr=23.7375047 lines=90 END +moesel call=18 dev=1 T=5 E=512 k=10 sel=f105feb5386cb177 minulps=0 mintok=3 x=5328.18034 logit=16318.9531 exp=2864.13171 shr=25.2442927 lines=95 END +moesel call=19 dev=1 T=5 E=512 k=10 sel=28ca1992b70a0310 minulps=1 mintok=0 x=5867.30701 logit=16202.9375 exp=2841.54083 shr=28.7731155 lines=100 END +moesel call=20 dev=1 T=5 E=512 k=10 sel=f735fa40c10add85 minulps=0 mintok=1 x=7151.09091 logit=16145.7344 exp=4341.68606 shr=43.5266942 lines=105 END +moesel call=21 dev=1 T=5 E=512 k=10 sel=e80a415a8a18321d minulps=0 mintok=0 x=6725.81356 logit=16177.8594 exp=3800.6881 shr=28.3593103 lines=110 END +moesel call=22 dev=1 T=5 E=512 k=10 sel=b3c62ea616442afa minulps=2 mintok=0 x=6497.52442 logit=16640.3281 exp=3627.13058 shr=35.1753135 lines=115 END +moesel call=23 dev=1 T=5 E=512 k=10 sel=74c07c747cc4d2cc minulps=0 mintok=0 x=7567.35111 logit=16092.75 exp=4452.38452 shr=64.4897626 lines=120 END +moesel call=24 dev=1 T=5 E=512 k=10 sel=a999bb8e78af4388 minulps=1 mintok=1 x=5616.96724 logit=16321.6562 exp=2815.24834 shr=27.9610005 lines=125 END +moesel call=25 dev=1 T=5 E=512 k=10 sel=8c1d1c68d8bc3aa6 minulps=0 mintok=0 x=5637.05077 logit=16019.5156 exp=2673.41674 shr=28.9463229 lines=130 END +moesel call=26 dev=1 T=5 E=512 k=10 sel=35fd18b3f1d9f02a minulps=0 mintok=1 x=5242.66694 logit=16425.1406 exp=2446.62897 shr=22.7984998 lines=135 END +moesel call=27 dev=1 T=5 E=512 k=10 sel=d5d4e03947aadf34 minulps=1 mintok=1 x=5245.88568 logit=15944.8594 exp=2370.57481 shr=19.0522638 lines=140 END +moesel call=28 dev=1 T=5 E=512 k=10 sel=827c2ba2af341d0e minulps=0 mintok=3 x=5277.3485 logit=16122.0156 exp=2359.31454 shr=23.4091644 lines=145 END +moesel call=29 dev=1 T=5 E=512 k=10 sel=3595d2909c9a67db minulps=1 mintok=2 x=5501.65057 logit=16555.7891 exp=2816.01307 shr=25.0176327 lines=150 END +moesel call=30 dev=1 T=5 E=512 k=10 sel=04758182a6a02d86 minulps=1 mintok=0 x=6430.20349 logit=16688.5312 exp=3843.51206 shr=32.5737284 lines=155 END +moesel call=31 dev=1 T=5 E=512 k=10 sel=90f3a853d44ec229 minulps=0 mintok=1 x=8011.68789 logit=16377.6406 exp=4867.48047 shr=30.3206993 lines=160 END +moesel call=32 dev=1 T=5 E=512 k=10 sel=bdedff50562962b8 minulps=0 mintok=1 x=7325.23582 logit=15981.2734 exp=4370.99278 shr=32.5776461 lines=165 END +moesel call=33 dev=1 T=5 E=512 k=10 sel=3c833769e5259f6d minulps=0 mintok=0 x=6701.2299 logit=16714.3281 exp=3827.1734 shr=25.0316927 lines=170 END +moesel call=34 dev=1 T=5 E=512 k=10 sel=3188ef46a7207960 minulps=1 mintok=0 x=6143.44704 logit=16665.9531 exp=3841.58992 shr=33.8632984 lines=175 END +moesel call=35 dev=1 T=5 E=512 k=10 sel=11d8430c7058002b minulps=2 mintok=1 x=7099.00982 logit=16456.9531 exp=4640.90574 shr=32.8907891 lines=180 END +moesel call=36 dev=1 T=5 E=512 k=10 sel=ce85453d6b312686 minulps=0 mintok=2 x=8781.90772 logit=16370.8594 exp=8225.27047 shr=88.246615 lines=185 END +moesel call=37 dev=1 T=5 E=512 k=10 sel=8bb5758a7850c86e minulps=0 mintok=1 x=8303.17253 logit=16354.3594 exp=7001.25341 shr=65.9959179 lines=190 END +moesel call=38 dev=1 T=5 E=512 k=10 sel=1088ca83dc40c8cf minulps=0 mintok=0 x=8461.53261 logit=16808.5312 exp=9278.23279 shr=109.359189 lines=195 END +moesel call=39 dev=1 T=5 E=512 k=10 sel=79bf51e408d85e86 minulps=0 mintok=0 x=9068.89016 logit=15714.6328 exp=8503.02438 shr=94.729246 lines=200 END +moesel call=40 dev=1 T=5 E=512 k=10 sel=5547dd9fd5bb6f4b minulps=0 mintok=0 x=9530.33126 logit=15709.2969 exp=7270.07352 shr=95.0038095 lines=205 END +moesel call=41 dev=1 T=5 E=512 k=10 sel=fc783871f0472791 minulps=1 mintok=1 x=9615.44181 logit=16108.7188 exp=6946.6853 shr=102.375323 lines=210 END +moesel call=42 dev=1 T=5 E=512 k=10 sel=d282c5f9fcddd1db minulps=0 mintok=0 x=9808.29724 logit=16385.2812 exp=9273.51059 shr=131.467709 lines=215 END +moesel call=43 dev=1 T=5 E=512 k=10 sel=21efbad6440ee049 minulps=0 mintok=3 x=8195.22645 logit=15776.9609 exp=5803.28836 shr=89.4814494 lines=220 END +moesel call=44 dev=1 T=5 E=512 k=10 sel=09168f1f28093967 minulps=0 mintok=1 x=9189.70934 logit=15894.25 exp=7386.05759 shr=105.226898 lines=225 END +moesel call=45 dev=1 T=5 E=512 k=10 sel=9fa59df33f617085 minulps=0 mintok=0 x=10345.0287 logit=16224.1719 exp=7223.67518 shr=145.161863 lines=230 END +moesel call=46 dev=1 T=5 E=512 k=10 sel=2ebdca3d3e7ecf8e minulps=1 mintok=2 x=10114.0947 logit=16271.3906 exp=8474.7606 shr=155.178569 lines=235 END +moesel call=47 dev=1 T=5 E=512 k=10 sel=b6add1408e56cc74 minulps=1 mintok=3 x=9601.74072 logit=16386.2812 exp=9116.37751 shr=815.846916 lines=240 END +moesel call=48 dev=1 T=1 E=512 k=10 sel=8fce961a87c895de minulps=7 mintok=0 x=723.664909 logit=3430.46875 exp=563.497883 shr=31.5413151 lines=241 END +moesel call=49 dev=1 T=1 E=512 k=10 sel=da67409db4906d8e minulps=2 mintok=0 x=815.778957 logit=3050.65625 exp=411.528591 shr=6.04054234 lines=242 END +moesel call=50 dev=1 T=1 E=512 k=10 sel=f1effeeba82642ba minulps=1 mintok=0 x=850.152067 logit=3036.60938 exp=419.765577 shr=9.20946218 lines=243 END +moesel call=51 dev=1 T=1 E=512 k=10 sel=da92804a7ed93391 minulps=0 mintok=0 x=1061.76275 logit=2973.15625 exp=522.752055 shr=5.76472539 lines=244 END +moesel call=52 dev=1 T=1 E=512 k=10 sel=79e503d24cec8ea1 minulps=4 mintok=0 x=1097.65658 logit=2991.65625 exp=552.884984 shr=5.48180634 lines=245 END +moesel call=53 dev=1 T=1 E=512 k=10 sel=8423f417266f13a9 minulps=0 mintok=0 x=959.375325 logit=3035.70312 exp=439.299158 shr=5.19935743 lines=246 END +moesel call=54 dev=1 T=1 E=512 k=10 sel=ac94027b28df13c6 minulps=0 mintok=0 x=1087.99149 logit=2885.35938 exp=719.38391 shr=6.26721723 lines=247 END +moesel call=55 dev=1 T=1 E=512 k=10 sel=a12e2a23cfcd41a7 minulps=0 mintok=0 x=1105.11678 logit=2956.125 exp=564.357492 shr=5.27389602 lines=248 END +moesel call=56 dev=1 T=1 E=512 k=10 sel=23e3af4450f40f94 minulps=10 mintok=0 x=1132.14298 logit=2970.45312 exp=636.273753 shr=4.96539277 lines=249 END +moesel call=57 dev=1 T=1 E=512 k=10 sel=4f5439affa3f7823 minulps=1 mintok=0 x=1169.21249 logit=3023.01562 exp=604.602861 shr=7.07437545 lines=250 END +moesel call=58 dev=1 T=1 E=512 k=10 sel=fa8a6c4748735536 minulps=1 mintok=0 x=1065.15538 logit=3002.625 exp=508.546191 shr=4.56790698 lines=251 END +moesel call=59 dev=1 T=1 E=512 k=10 sel=610e505750ac340d minulps=1 mintok=0 x=1074.04506 logit=3049.89062 exp=477.510162 shr=3.14345438 lines=252 END +moesel call=60 dev=1 T=1 E=512 k=10 sel=6e98fc1568dabe37 minulps=1 mintok=0 x=1099.14507 logit=3040.84375 exp=506.666758 shr=5.59145327 lines=253 END +moesel call=61 dev=1 T=1 E=512 k=10 sel=e556fe4cf283032d minulps=1 mintok=0 x=1004.17902 logit=3055.09375 exp=429.474333 shr=5.29509216 lines=254 END +moesel call=62 dev=1 T=1 E=512 k=10 sel=9f55aa23ddc601de minulps=1 mintok=0 x=1135.75445 logit=3041.59375 exp=528.632194 shr=5.57348994 lines=255 END +moesel call=63 dev=1 T=1 E=512 k=10 sel=b5fa1a7c51a11adb minulps=3 mintok=0 x=1481.94626 logit=3048.6875 exp=833.688153 shr=7.24646278 lines=256 END +moesel call=64 dev=1 T=1 E=512 k=10 sel=3f5ae586af492907 minulps=3 mintok=0 x=1308.29881 logit=3068.84375 exp=636.236269 shr=5.68035066 lines=257 END +moesel call=65 dev=1 T=1 E=512 k=10 sel=f0b2fb36fcbcebfc minulps=1 mintok=0 x=1135.90056 logit=3116.04688 exp=591.092324 shr=4.57442022 lines=258 END +moesel call=66 dev=1 T=1 E=512 k=10 sel=02878f048ba3b3cc minulps=0 mintok=0 x=1110.02275 logit=3212.3125 exp=565.944166 shr=4.58377909 lines=259 END +moesel call=67 dev=1 T=1 E=512 k=10 sel=44c730c5e4ad62b4 minulps=0 mintok=0 x=1277.21745 logit=3174.32812 exp=662.280042 shr=5.30693895 lines=260 END +moesel call=68 dev=1 T=1 E=512 k=10 sel=c8c8bfef3ac9cf11 minulps=3 mintok=0 x=1526.47765 logit=3156.125 exp=919.959777 shr=9.40919015 lines=261 END +moesel call=69 dev=1 T=1 E=512 k=10 sel=f0f1f9191ede49b5 minulps=1 mintok=0 x=1488.26378 logit=3190.21875 exp=857.027212 shr=8.0348592 lines=262 END +moesel call=70 dev=1 T=1 E=512 k=10 sel=ba1241b9927c42f3 minulps=8 mintok=0 x=1389.98263 logit=3250.26562 exp=826.856896 shr=4.24134622 lines=263 END +moesel call=71 dev=1 T=1 E=512 k=10 sel=a18a19bedd1d013f minulps=2 mintok=0 x=1725.68997 logit=3138.65625 exp=1067.90509 shr=14.7051098 lines=264 END +moesel call=72 dev=1 T=1 E=512 k=10 sel=f792ae08b7ac3a7b minulps=1 mintok=0 x=1169.60753 logit=3133.48438 exp=591.516186 shr=6.21655642 lines=265 END +moesel call=73 dev=1 T=1 E=512 k=10 sel=18c70210b6f7eecc minulps=5 mintok=0 x=1239.81597 logit=3114.59375 exp=591.640211 shr=5.13053793 lines=266 END +moesel call=74 dev=1 T=1 E=512 k=10 sel=ab65a2d9f6c06e5f minulps=0 mintok=0 x=1145.68917 logit=3141.35938 exp=564.744467 shr=4.88275191 lines=267 END +moesel call=75 dev=1 T=1 E=512 k=10 sel=b943fedc5d203b58 minulps=6 mintok=0 x=1200.04994 logit=3088.3125 exp=639.699755 shr=5.89255972 lines=268 END +moesel call=76 dev=1 T=1 E=512 k=10 sel=392b4ca5bc1fa11b minulps=0 mintok=0 x=1232.23133 logit=3132.15625 exp=643.189591 shr=5.68870719 lines=269 END +moesel call=77 dev=1 T=1 E=512 k=10 sel=c598bc5cb5506b7c minulps=3 mintok=0 x=1275.81649 logit=3159.20312 exp=740.54248 shr=7.27388824 lines=270 END +moesel call=78 dev=1 T=1 E=512 k=10 sel=6c8796420f9bb75c minulps=0 mintok=0 x=1470.86962 logit=3133.32031 exp=867.480005 shr=7.81114797 lines=271 END +moesel call=79 dev=1 T=1 E=512 k=10 sel=eb29872faed407da minulps=3 mintok=0 x=1827.32727 logit=3074.15625 exp=1186.14958 shr=10.3065294 lines=272 END +moesel call=80 dev=1 T=1 E=512 k=10 sel=21524f9969403d65 minulps=4 mintok=0 x=1763.7714 logit=3055.25 exp=1054.69468 shr=9.87494844 lines=273 END +moesel call=81 dev=1 T=1 E=512 k=10 sel=4a574cc2fc7fb9fb minulps=5 mintok=0 x=1602.81626 logit=3315.92188 exp=1103.81968 shr=6.46347584 lines=274 END +moesel call=82 dev=1 T=1 E=512 k=10 sel=b3eaa9c6c6eb3130 minulps=0 mintok=0 x=1433.33069 logit=3340.53125 exp=980.073327 shr=6.11113988 lines=275 END +moesel call=83 dev=1 T=1 E=512 k=10 sel=2e6166f480eec102 minulps=2 mintok=0 x=1680.14681 logit=3316.28125 exp=1233.40858 shr=7.08982886 lines=276 END +moesel call=84 dev=1 T=1 E=512 k=10 sel=e74550e15d7c5824 minulps=2 mintok=0 x=1999.0076 logit=3225.98438 exp=1862.59425 shr=19.151212 lines=277 END +moesel call=85 dev=1 T=1 E=512 k=10 sel=b589ab458c3fcf8f minulps=6 mintok=0 x=1922.31729 logit=3259.39062 exp=1476.00827 shr=13.1864017 lines=278 END +moesel call=86 dev=1 T=1 E=512 k=10 sel=f1fdb7b403ff173f minulps=2 mintok=0 x=1897.10037 logit=3267.20312 exp=2262.07617 shr=18.4758842 lines=279 END +moesel call=87 dev=1 T=1 E=512 k=10 sel=e281339c7be2009f minulps=2 mintok=0 x=2020.75961 logit=3055.8125 exp=1835.15941 shr=28.9277753 lines=280 END +moesel call=88 dev=1 T=1 E=512 k=10 sel=5d15905bdd77a9b8 minulps=3 mintok=0 x=2077.11619 logit=3079.07812 exp=1727.19587 shr=23.2208766 lines=281 END +moesel call=89 dev=1 T=1 E=512 k=10 sel=b5354b95743d475f minulps=0 mintok=0 x=2059.48909 logit=3224.5 exp=1896.18855 shr=12.8156054 lines=282 END +moesel call=90 dev=1 T=1 E=512 k=10 sel=e18d448a6e9d074f minulps=0 mintok=0 x=2088.69188 logit=3183.14062 exp=2517.64052 shr=23.6908675 lines=283 END +moesel call=91 dev=1 T=1 E=512 k=10 sel=196c9873cebb2a27 minulps=6 mintok=0 x=1798.19436 logit=3094.70312 exp=1263.77772 shr=21.8022825 lines=284 END +moesel call=92 dev=1 T=1 E=512 k=10 sel=ae96ba2bf0c60fc0 minulps=1 mintok=0 x=2015.2857 logit=3143.8125 exp=1677.70815 shr=18.9001883 lines=285 END +moesel call=93 dev=1 T=1 E=512 k=10 sel=0770365869cd05cc minulps=6 mintok=0 x=2180.65905 logit=3273.34375 exp=1535.39289 shr=23.4513273 lines=286 END +moesel call=94 dev=1 T=1 E=512 k=10 sel=195829ec6b1c4fae minulps=1 mintok=0 x=2207.75889 logit=3264.14062 exp=1543.09526 shr=31.3433605 lines=287 END +moesel call=95 dev=1 T=1 E=512 k=10 sel=2e60b1df07b0c22c minulps=0 mintok=0 x=1998.70799 logit=3184.03125 exp=1309.73513 shr=134.910462 lines=288 END +moesel call=96 dev=1 T=1 E=512 k=10 sel=0148dfd4911f327c minulps=0 mintok=0 x=711.229391 logit=3391.71875 exp=608.896065 shr=38.3388032 lines=289 END +moesel call=97 dev=1 T=1 E=512 k=10 sel=2a8a39b413aa2418 minulps=2 mintok=0 x=813.289912 logit=3060.14062 exp=458.857198 shr=5.10810651 lines=290 END +moesel call=98 dev=1 T=1 E=512 k=10 sel=7091a270949c8cf6 minulps=0 mintok=0 x=793.840794 logit=3058.3125 exp=305.347046 shr=7.26283523 lines=291 END +moesel call=99 dev=1 T=1 E=512 k=10 sel=d3488e89a2656e11 minulps=0 mintok=0 x=1148.27415 logit=3025.26562 exp=563.083738 shr=3.97451648 lines=292 END +moesel call=100 dev=1 T=1 E=512 k=10 sel=ca64ce98d54c003b minulps=2 mintok=0 x=1142.11562 logit=2944.34375 exp=527.39017 shr=4.94533142 lines=293 END +moesel call=101 dev=1 T=1 E=512 k=10 sel=82c9513497c8dab5 minulps=3 mintok=0 x=1046.18565 logit=3038.15625 exp=467.063415 shr=7.69106738 lines=294 END +moesel call=102 dev=1 T=1 E=512 k=10 sel=f49dbc59c7d107e1 minulps=1 mintok=0 x=1204.76578 logit=3084 exp=647.277392 shr=6.21173823 lines=295 END +moesel call=103 dev=1 T=1 E=512 k=10 sel=cb566956732e4c89 minulps=3 mintok=0 x=1098.74821 logit=3016 exp=524.098303 shr=5.28041228 lines=296 END +moesel call=104 dev=1 T=1 E=512 k=10 sel=b4292d57aaf8616d minulps=1 mintok=0 x=1126.13285 logit=3031.78125 exp=580.574672 shr=6.37354497 lines=297 END +moesel call=105 dev=1 T=1 E=512 k=10 sel=efc8328b96fbbf59 minulps=3 mintok=0 x=1151.31993 logit=3056.15625 exp=546.376722 shr=5.93742776 lines=298 END +moesel call=106 dev=1 T=1 E=512 k=10 sel=ecc57d51c9e6ad13 minulps=3 mintok=0 x=1062.67196 logit=3091.28125 exp=490.783311 shr=3.7371625 lines=299 END +moesel call=107 dev=1 T=1 E=512 k=10 sel=684bde3e5f1bf1aa minulps=3 mintok=0 x=1047.96809 logit=3081.14062 exp=455.223676 shr=3.03726482 lines=300 END +moesel call=108 dev=1 T=1 E=512 k=10 sel=bf127b4544c277d0 minulps=1 mintok=0 x=1065.00896 logit=3091.96875 exp=480.718076 shr=5.86843657 lines=301 END +moesel call=109 dev=1 T=1 E=512 k=10 sel=e598f26e2c516d21 minulps=1 mintok=0 x=952.62907 logit=3055.71875 exp=409.174747 shr=5.98165156 lines=302 END +moesel call=110 dev=1 T=1 E=512 k=10 sel=fee8e61f5b205d8d minulps=1 mintok=0 x=1190.50853 logit=3174.10938 exp=639.398977 shr=5.0496688 lines=303 END +moesel call=111 dev=1 T=1 E=512 k=10 sel=38ddb3334bf52011 minulps=7 mintok=0 x=1581.64626 logit=3088.82812 exp=915.981432 shr=7.10401408 lines=304 END +moesel call=112 dev=1 T=1 E=512 k=10 sel=197a929c90933e08 minulps=4 mintok=0 x=1315.26594 logit=3188.75 exp=665.381997 shr=4.12329758 lines=305 END +moesel call=113 dev=1 T=1 E=512 k=10 sel=5bf394db409c91de minulps=0 mintok=0 x=1086.86942 logit=3070.97656 exp=464.323578 shr=4.31994963 lines=306 END +moesel call=114 dev=1 T=1 E=512 k=10 sel=db2f2b90ff1ca8c4 minulps=6 mintok=0 x=1057.06034 logit=3195.5 exp=501.827474 shr=3.50922626 lines=307 END +moesel call=115 dev=1 T=1 E=512 k=10 sel=bf85a0dd154e12b1 minulps=1 mintok=0 x=1216.69313 logit=3169 exp=602.579264 shr=5.34388182 lines=308 END +moesel call=116 dev=1 T=1 E=512 k=10 sel=4df92e9c25fb4d51 minulps=5 mintok=0 x=1465.47069 logit=3190.48438 exp=796.757087 shr=8.43680063 lines=309 END +moesel call=117 dev=1 T=1 E=512 k=10 sel=cf786bdd0222e90f minulps=3 mintok=0 x=1447.98005 logit=3265.48438 exp=816.96201 shr=5.35224335 lines=310 END +moesel call=118 dev=1 T=1 E=512 k=10 sel=38c8a9ac128de4be minulps=1 mintok=0 x=1388.9577 logit=3321 exp=767.592594 shr=3.67117262 lines=311 END +moesel call=119 dev=1 T=1 E=512 k=10 sel=f85c4bb9161d1c08 minulps=1 mintok=0 x=1667.9041 logit=3130.51562 exp=928.467204 shr=9.59148346 lines=312 END +moesel call=120 dev=1 T=1 E=512 k=10 sel=4c1604415ca197b4 minulps=1 mintok=0 x=1192.32415 logit=3203.71875 exp=573.681941 shr=5.08476549 lines=313 END +moesel call=121 dev=1 T=1 E=512 k=10 sel=bd9f8afedc5e920f minulps=2 mintok=0 x=1218.67761 logit=3186.4375 exp=614.268691 shr=4.43214091 lines=314 END +moesel call=122 dev=1 T=1 E=512 k=10 sel=5ca6d5be21cfbf38 minulps=1 mintok=0 x=1105.24921 logit=3256.57812 exp=569.026838 shr=4.74037996 lines=315 END +moesel call=123 dev=1 T=1 E=512 k=10 sel=1b10475baf5406c8 minulps=24 mintok=0 x=1185.73088 logit=3211.54688 exp=593.23451 shr=5.2065296 lines=316 END +moesel call=124 dev=1 T=1 E=512 k=10 sel=d3fc82f065d25b5d minulps=3 mintok=0 x=1193.8325 logit=3253.59375 exp=564.272067 shr=5.23822185 lines=317 END +moesel call=125 dev=1 T=1 E=512 k=10 sel=1be218557f73f10e minulps=0 mintok=0 x=1199.97556 logit=3197.375 exp=687.191725 shr=6.24312269 lines=318 END +moesel call=126 dev=1 T=1 E=512 k=10 sel=f42323df10eb3bda minulps=1 mintok=0 x=1467.88619 logit=3279.4375 exp=972.377591 shr=8.87083261 lines=319 END +moesel call=127 dev=1 T=1 E=512 k=10 sel=d4ca388514099f7b minulps=0 mintok=0 x=1939.54626 logit=3197.4375 exp=1292.16474 shr=10.8208374 lines=320 END +moesel call=128 dev=1 T=1 E=512 k=10 sel=44d3f5dabd163fe4 minulps=0 mintok=0 x=1775.87708 logit=3192.46875 exp=1133.96296 shr=9.4649017 lines=321 END +moesel call=129 dev=1 T=1 E=512 k=10 sel=45d1e46900535c92 minulps=4 mintok=0 x=1562.9893 logit=3186.5625 exp=1062.0221 shr=8.61333513 lines=322 END +moesel call=130 dev=1 T=1 E=512 k=10 sel=7ecefc81d491db2f minulps=1 mintok=0 x=1355.86098 logit=3281.95312 exp=829.080054 shr=5.78759143 lines=323 END +moesel call=131 dev=1 T=1 E=512 k=10 sel=a1b8c880f2e38e41 minulps=4 mintok=0 x=1603.41966 logit=3165.34375 exp=989.260696 shr=7.96881433 lines=324 END +moesel call=132 dev=1 T=1 E=512 k=10 sel=ef214577e0fb46df minulps=2 mintok=0 x=1876.34944 logit=3195.42188 exp=1582.1276 shr=23.6369063 lines=325 END +moesel call=133 dev=1 T=1 E=512 k=10 sel=546bd831da771053 minulps=3 mintok=0 x=1755.0382 logit=3223.59375 exp=1334.19155 shr=11.9084479 lines=326 END +moesel call=134 dev=1 T=1 E=512 k=10 sel=4ba4625a67b48d5a minulps=3 mintok=0 x=1782.05633 logit=3165 exp=1762.52148 shr=15.7946019 lines=327 END +moesel call=135 dev=1 T=1 E=512 k=10 sel=33563f0a816e3ea5 minulps=3 mintok=0 x=2061.72572 logit=3058.8125 exp=1854.22584 shr=22.7734525 lines=328 END +moesel call=136 dev=1 T=1 E=512 k=10 sel=215e261e955d132a minulps=2 mintok=0 x=2105.59185 logit=3110.26562 exp=1579.11116 shr=17.9604087 lines=329 END +moesel call=137 dev=1 T=1 E=512 k=10 sel=c8237987eca74795 minulps=0 mintok=0 x=2075.90125 logit=3203.89062 exp=1873.71327 shr=11.5481218 lines=330 END +moesel call=138 dev=1 T=1 E=512 k=10 sel=be9e1f90de0ea44d minulps=0 mintok=0 x=2101.67663 logit=3193.0625 exp=1893.70736 shr=22.0316356 lines=331 END +moesel call=139 dev=1 T=1 E=512 k=10 sel=36ba76b551a2124f minulps=3 mintok=0 x=1861.83108 logit=3099.35938 exp=1341.23117 shr=21.6560817 lines=332 END +moesel call=140 dev=1 T=1 E=512 k=10 sel=0adbb6f1f0e23878 minulps=1 mintok=0 x=2019.18243 logit=3169.25 exp=1736.76268 shr=23.2480535 lines=333 END +moesel call=141 dev=1 T=1 E=512 k=10 sel=834801528ea79c8e minulps=1 mintok=0 x=2156.12024 logit=3188.3125 exp=1707.46686 shr=30.6757116 lines=334 END +moesel call=142 dev=1 T=1 E=512 k=10 sel=ac6441082a2c0801 minulps=0 mintok=0 x=2061.68134 logit=3139.15625 exp=1626.7381 shr=34.1050006 lines=335 END +moesel call=143 dev=1 T=1 E=512 k=10 sel=7576e41aaf7ace32 minulps=3 mintok=0 x=1918.08546 logit=3193.03125 exp=1586.48832 shr=159.199129 lines=336 END +moesel call=144 dev=1 T=1 E=512 k=10 sel=09c0aa1fa6c8c877 minulps=0 mintok=0 x=709.265829 logit=3419.98438 exp=588.563679 shr=44.0349371 lines=337 END +moesel call=145 dev=1 T=1 E=512 k=10 sel=a916136dafb38ba5 minulps=2 mintok=0 x=806.718358 logit=3072.34375 exp=521.929052 shr=5.52216164 lines=338 END +moesel call=146 dev=1 T=1 E=512 k=10 sel=c3ae50cfa21dff8e minulps=1 mintok=0 x=834.998063 logit=3038.6875 exp=383.940281 shr=8.0326896 lines=339 END +moesel call=147 dev=1 T=1 E=512 k=10 sel=a8708ad99713b1fa minulps=1 mintok=0 x=1166.77777 logit=3165.46875 exp=531.862158 shr=3.76528293 lines=340 END +moesel call=148 dev=1 T=1 E=512 k=10 sel=a07ce5b4d7e4c0f4 minulps=1 mintok=0 x=1143.24499 logit=3043.60938 exp=600.16254 shr=6.28855085 lines=341 END +moesel call=149 dev=1 T=1 E=512 k=10 sel=41ce2a00299c7b4e minulps=0 mintok=0 x=1077.6009 logit=3098.03125 exp=449.393738 shr=5.79224983 lines=342 END +moesel call=150 dev=1 T=1 E=512 k=10 sel=40068dc1a2211280 minulps=1 mintok=0 x=1197.02317 logit=3176.82812 exp=636.113586 shr=7.11001849 lines=343 END +moesel call=151 dev=1 T=1 E=512 k=10 sel=91b6b39d22f98b4f minulps=2 mintok=0 x=1124.00188 logit=3106.10938 exp=573.867174 shr=4.49665631 lines=344 END +moesel call=152 dev=1 T=1 E=512 k=10 sel=6249f6dc603c35af minulps=1 mintok=0 x=1121.45561 logit=3045.1875 exp=552.051332 shr=4.80011374 lines=345 END +moesel call=153 dev=1 T=1 E=512 k=10 sel=2718d09e060a4646 minulps=7 mintok=0 x=1165.85487 logit=3095.45312 exp=521.999165 shr=5.80828868 lines=346 END +moesel call=154 dev=1 T=1 E=512 k=10 sel=fc3615395040373a minulps=0 mintok=0 x=1045.45938 logit=3144.29688 exp=502.656306 shr=3.53277464 lines=347 END +moesel call=155 dev=1 T=1 E=512 k=10 sel=3b73bf134ebe2733 minulps=5 mintok=0 x=1054.21848 logit=3071.26562 exp=447.021474 shr=3.02292909 lines=348 END +moesel call=156 dev=1 T=1 E=512 k=10 sel=677f10c147284eaa minulps=1 mintok=0 x=1097.53947 logit=3078.78125 exp=529.79108 shr=6.68490016 lines=349 END +moesel call=157 dev=1 T=1 E=512 k=10 sel=8e80159dfedac81d minulps=1 mintok=0 x=980.618567 logit=3005.90625 exp=421.948717 shr=5.0479212 lines=350 END +moesel call=158 dev=1 T=1 E=512 k=10 sel=3901af792cd4064e minulps=0 mintok=0 x=1203.96706 logit=3163.9375 exp=603.212165 shr=4.67670434 lines=351 END +moesel call=159 dev=1 T=1 E=512 k=10 sel=ecea62446c550089 minulps=3 mintok=0 x=1616.5708 logit=3171.79688 exp=950.795252 shr=5.66884657 lines=352 END +moesel call=160 dev=1 T=1 E=512 k=10 sel=6dea3b8b688439cf minulps=3 mintok=0 x=1383.46333 logit=3119.78125 exp=659.9029 shr=4.44192258 lines=353 END +moesel call=161 dev=1 T=1 E=512 k=10 sel=1ab7e1e78d5fd1d8 minulps=13 mintok=0 x=1165.40147 logit=3048.26562 exp=542.088381 shr=4.76746796 lines=354 END +moesel call=162 dev=1 T=1 E=512 k=10 sel=2dcd573eb4fc01b4 minulps=1 mintok=0 x=1037.56161 logit=3133.90625 exp=471.669929 shr=3.77397194 lines=355 END +moesel call=163 dev=1 T=1 E=512 k=10 sel=dc42ee3aef8a215f minulps=1 mintok=0 x=1257.9455 logit=3177.53125 exp=625.847851 shr=5.1642314 lines=356 END +moesel call=164 dev=1 T=1 E=512 k=10 sel=c30e84a97933daad minulps=1 mintok=0 x=1477.93942 logit=3180.95312 exp=819.896465 shr=9.15391125 lines=357 END +moesel call=165 dev=1 T=1 E=512 k=10 sel=938a2967cafb3bb9 minulps=1 mintok=0 x=1389.86459 logit=3218.40625 exp=728.211013 shr=5.84739663 lines=358 END +moesel call=166 dev=1 T=1 E=512 k=10 sel=024eef52bb54717a minulps=6 mintok=0 x=1302.85019 logit=3321.03125 exp=674.083279 shr=4.14695835 lines=359 END +moesel call=167 dev=1 T=1 E=512 k=10 sel=eb52d195dc1e4d9d minulps=0 mintok=0 x=1583.46851 logit=3157.1875 exp=810.325841 shr=9.69447611 lines=360 END +moesel call=168 dev=1 T=1 E=512 k=10 sel=3b2fbe3418eff725 minulps=2 mintok=0 x=1167.98357 logit=3179.67188 exp=586.495793 shr=5.39254894 lines=361 END +moesel call=169 dev=1 T=1 E=512 k=10 sel=7ce39d832fb4a4c3 minulps=2 mintok=0 x=1282.43682 logit=3241.10938 exp=670.836109 shr=5.9029246 lines=362 END +moesel call=170 dev=1 T=1 E=512 k=10 sel=f8c3febace9a2e04 minulps=2 mintok=0 x=1120.09287 logit=3198.40625 exp=554.365411 shr=4.36842306 lines=363 END +moesel call=171 dev=1 T=1 E=512 k=10 sel=922aedd0aa6e7c97 minulps=3 mintok=0 x=1166.64225 logit=3210.28125 exp=550.763535 shr=4.82317674 lines=364 END +moesel call=172 dev=1 T=1 E=512 k=10 sel=88201025b87ff1aa minulps=1 mintok=0 x=1210.73197 logit=3291.85938 exp=608.724044 shr=6.43653818 lines=365 END +moesel call=173 dev=1 T=1 E=512 k=10 sel=242f52e1d2aea9b4 minulps=1 mintok=0 x=1174.16624 logit=3169.625 exp=684.683064 shr=7.11342804 lines=366 END +moesel call=174 dev=1 T=1 E=512 k=10 sel=97b8993e354862fe minulps=2 mintok=0 x=1443.55906 logit=3327.95312 exp=906.495923 shr=7.82015212 lines=367 END +moesel call=175 dev=1 T=1 E=512 k=10 sel=ecceaaf787f4e8a2 minulps=1 mintok=0 x=1907.92532 logit=3234.75 exp=1303.53765 shr=11.7175276 lines=368 END +moesel call=176 dev=1 T=1 E=512 k=10 sel=623fa0d91696b26a minulps=0 mintok=0 x=1772.42972 logit=3182.95312 exp=1102.16113 shr=12.0974883 lines=369 END +moesel call=177 dev=1 T=1 E=512 k=10 sel=a805094da310ac0d minulps=1 mintok=0 x=1562.21337 logit=3161.53125 exp=1013.94754 shr=8.14480484 lines=370 END +moesel call=178 dev=1 T=1 E=512 k=10 sel=ea7214f25d79013e minulps=6 mintok=0 x=1408.19263 logit=3322.875 exp=888.958483 shr=6.42585049 lines=371 END +moesel call=179 dev=1 T=1 E=512 k=10 sel=ca8d28b5f880044a minulps=3 mintok=0 x=1603.33182 logit=3187.57812 exp=981.019062 shr=9.40827595 lines=372 END +moesel call=180 dev=1 T=1 E=512 k=10 sel=5fa37274e14a4555 minulps=0 mintok=0 x=1830.96466 logit=3197.96875 exp=1497.51121 shr=24.2480717 lines=373 END +moesel call=181 dev=1 T=1 E=512 k=10 sel=0af4c7f5941e2562 minulps=4 mintok=0 x=1783.48712 logit=3279.78125 exp=1305.22535 shr=12.6527024 lines=374 END +moesel call=182 dev=1 T=1 E=512 k=10 sel=b2a982321d88e522 minulps=0 mintok=0 x=1841.23434 logit=3210.3125 exp=1817.83536 shr=16.2897464 lines=375 END +moesel call=183 dev=1 T=1 E=512 k=10 sel=985cef7fde795954 minulps=3 mintok=0 x=1939.8259 logit=3074.29688 exp=1636.95547 shr=29.4986321 lines=376 END +moesel call=184 dev=1 T=1 E=512 k=10 sel=e0639f29d57181b6 minulps=2 mintok=0 x=1947.12249 logit=3111.59375 exp=1375.78275 shr=18.5198702 lines=377 END +moesel call=185 dev=1 T=1 E=512 k=10 sel=083ec11d6668dd3b minulps=6 mintok=0 x=2043.42286 logit=3206.75 exp=1591.7568 shr=12.1486009 lines=378 END +moesel call=186 dev=1 T=1 E=512 k=10 sel=a5b8b376d111d99d minulps=1 mintok=0 x=2183.88591 logit=3210.40625 exp=1958.24461 shr=21.903844 lines=379 END +moesel call=187 dev=1 T=1 E=512 k=10 sel=6118f474e0432f43 minulps=5 mintok=0 x=1861.45516 logit=3113.1875 exp=1351.57273 shr=23.5180559 lines=380 END +moesel call=188 dev=1 T=1 E=512 k=10 sel=cbe071c09189f847 minulps=1 mintok=0 x=2028.40356 logit=3157.65625 exp=1554.31609 shr=25.4878428 lines=381 END +moesel call=189 dev=1 T=1 E=512 k=10 sel=132689ebfca1f5fe minulps=0 mintok=0 x=2282.88352 logit=3248.3125 exp=1786.49834 shr=23.95741 lines=382 END +moesel call=190 dev=1 T=1 E=512 k=10 sel=861266fd7f73af51 minulps=2 mintok=0 x=2142.89643 logit=3205.82812 exp=1843.56673 shr=45.3861183 lines=383 END +moesel call=191 dev=1 T=1 E=512 k=10 sel=4199bb65a00cd9d9 minulps=1 mintok=0 x=1941.61238 logit=3194.5 exp=1618.01128 shr=154.832057 lines=384 END +moesel call=192 dev=1 T=1 E=512 k=10 sel=779637335a2e27ec minulps=0 mintok=0 x=715.972754 logit=3468.28125 exp=487.24113 shr=51.3272271 lines=385 END +moesel call=193 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=765.311812 logit=3039.48438 exp=448.088648 shr=5.45426782 lines=386 END +moesel call=194 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=1 mintok=0 x=790.771747 logit=3074.59375 exp=368.835797 shr=6.01973237 lines=387 END +moesel call=195 dev=1 T=1 E=512 k=10 sel=55b427fc8ccf7832 minulps=0 mintok=0 x=1146.39526 logit=3150.625 exp=574.587783 shr=3.67641241 lines=388 END +moesel call=196 dev=1 T=1 E=512 k=10 sel=e75f20bfc53651de minulps=1 mintok=0 x=1086.01609 logit=2952.48438 exp=527.473318 shr=6.07862944 lines=389 END +moesel call=197 dev=1 T=1 E=512 k=10 sel=e199dea9a8bf81ec minulps=3 mintok=0 x=1096.07753 logit=3080 exp=518.36322 shr=6.0544898 lines=390 END +moesel call=198 dev=1 T=1 E=512 k=10 sel=d09df72dfb6f0590 minulps=0 mintok=0 x=1232.41076 logit=3124.51562 exp=686.97776 shr=6.92919293 lines=391 END +moesel call=199 dev=1 T=1 E=512 k=10 sel=cc090d387c768f94 minulps=1 mintok=0 x=1112.01891 logit=3108.51562 exp=549.316119 shr=6.26023278 lines=392 END +moesel call=200 dev=1 T=1 E=512 k=10 sel=aff67bb07407ae85 minulps=1 mintok=0 x=1098.923 logit=3076.9375 exp=563.466729 shr=5.53606613 lines=393 END +moesel call=201 dev=1 T=1 E=512 k=10 sel=325e66aef4c1aa8a minulps=3 mintok=0 x=1152.64936 logit=3080.92188 exp=520.811862 shr=6.24086326 lines=394 END +moesel call=202 dev=1 T=1 E=512 k=10 sel=c1e76752a2f342fb minulps=1 mintok=0 x=1049.53183 logit=3127.5 exp=494.600163 shr=3.69966551 lines=395 END +moesel call=203 dev=1 T=1 E=512 k=10 sel=a446f5b4aa564dce minulps=4 mintok=0 x=1059.41607 logit=3035.5 exp=429.340645 shr=3.23255975 lines=396 END +moesel call=204 dev=1 T=1 E=512 k=10 sel=6084dcd936031c37 minulps=3 mintok=0 x=1100.06998 logit=3073.10938 exp=535.282465 shr=7.49345727 lines=397 END +moesel call=205 dev=1 T=1 E=512 k=10 sel=9afbc29fbb70741f minulps=3 mintok=0 x=971.840316 logit=3056.85938 exp=427.407292 shr=5.83559805 lines=398 END +moesel call=206 dev=1 T=1 E=512 k=10 sel=e54efc1726ad547b minulps=4 mintok=0 x=1171.57439 logit=3153.32812 exp=611.380009 shr=5.64469967 lines=399 END +moesel call=207 dev=1 T=1 E=512 k=10 sel=7d0859b4f61a0cbc minulps=1 mintok=0 x=1560.24919 logit=3178.28125 exp=960.064218 shr=5.07720075 lines=400 END +moesel call=208 dev=1 T=1 E=512 k=10 sel=3ba030b2004253b6 minulps=0 mintok=0 x=1368.99124 logit=3038.89062 exp=700.1013 shr=4.16129598 lines=401 END +moesel call=209 dev=1 T=1 E=512 k=10 sel=1c33e4ec13c54172 minulps=11 mintok=0 x=1163.93996 logit=3016.45312 exp=531.942349 shr=4.26704559 lines=402 END +moesel call=210 dev=1 T=1 E=512 k=10 sel=25a8a4bc71d12bfa minulps=1 mintok=0 x=1086.59974 logit=3191.73438 exp=490.604975 shr=3.71147528 lines=403 END +moesel call=211 dev=1 T=1 E=512 k=10 sel=f5fbc184160044b4 minulps=1 mintok=0 x=1260.99685 logit=3202.71875 exp=612.229754 shr=4.33470666 lines=404 END +moesel call=212 dev=1 T=1 E=512 k=10 sel=c725a272191ace12 minulps=2 mintok=0 x=1533.60942 logit=3210.03125 exp=873.32988 shr=10.9670092 lines=405 END +moesel call=213 dev=1 T=1 E=512 k=10 sel=180fb3de7452b92a minulps=1 mintok=0 x=1426.39367 logit=3195.25 exp=709.281718 shr=5.5830484 lines=406 END +moesel call=214 dev=1 T=1 E=512 k=10 sel=f02ddca3898fa612 minulps=1 mintok=0 x=1375.95914 logit=3256.51562 exp=709.251355 shr=3.75101862 lines=407 END +moesel call=215 dev=1 T=1 E=512 k=10 sel=7bb2417988ed8e59 minulps=3 mintok=0 x=1674.38684 logit=3193.26562 exp=937.319845 shr=9.48965186 lines=408 END +moesel call=216 dev=1 T=1 E=512 k=10 sel=194d48a748c629c6 minulps=3 mintok=0 x=1206.20548 logit=3190.48438 exp=606.87991 shr=5.17114846 lines=409 END +moesel call=217 dev=1 T=1 E=512 k=10 sel=8d5fbf947b98670c minulps=1 mintok=0 x=1293.95078 logit=3205.15625 exp=650.469607 shr=5.57221315 lines=410 END +moesel call=218 dev=1 T=1 E=512 k=10 sel=ac4f5ffd3e72d566 minulps=2 mintok=0 x=1131.50172 logit=3202.33594 exp=550.584461 shr=4.01715562 lines=411 END +moesel call=219 dev=1 T=1 E=512 k=10 sel=82ef23facb2f8412 minulps=5 mintok=0 x=1201.47886 logit=3131.3125 exp=616.733163 shr=4.2928885 lines=412 END +moesel call=220 dev=1 T=1 E=512 k=10 sel=b644f72fad00f1ed minulps=2 mintok=0 x=1252.86975 logit=3241.01562 exp=669.556914 shr=5.77976474 lines=413 END +moesel call=221 dev=1 T=1 E=512 k=10 sel=93bd0ee37e07e15c minulps=1 mintok=0 x=1213.96096 logit=3224.45312 exp=694.328718 shr=7.16312608 lines=414 END +moesel call=222 dev=1 T=1 E=512 k=10 sel=eee468c72d623319 minulps=1 mintok=0 x=1494.2228 logit=3272.51562 exp=933.00448 shr=8.65537377 lines=415 END +moesel call=223 dev=1 T=1 E=512 k=10 sel=3a8eb23f98e69453 minulps=5 mintok=0 x=1945.89427 logit=3196.32812 exp=1337.74517 shr=9.30025354 lines=416 END +moesel call=224 dev=1 T=1 E=512 k=10 sel=8e7ef235d23ba453 minulps=1 mintok=0 x=1792.11004 logit=3157.46875 exp=1103.74508 shr=8.14367199 lines=417 END +moesel call=225 dev=1 T=1 E=512 k=10 sel=cd708bff21223dab minulps=1 mintok=0 x=1550.37602 logit=3131.53125 exp=943.768813 shr=6.73874525 lines=418 END +moesel call=226 dev=1 T=1 E=512 k=10 sel=a2592ab30223fbd7 minulps=1 mintok=0 x=1454.51285 logit=3359.14062 exp=880.687199 shr=4.52738642 lines=419 END +moesel call=227 dev=1 T=1 E=512 k=10 sel=7605e279ebbc9aa6 minulps=0 mintok=0 x=1718.42733 logit=3280.0625 exp=1116.49685 shr=7.29068101 lines=420 END +moesel call=228 dev=1 T=1 E=512 k=10 sel=0def7dfdc3e3fd69 minulps=1 mintok=0 x=2138.24944 logit=3269.1875 exp=1975.03493 shr=20.0015562 lines=421 END +moesel call=229 dev=1 T=1 E=512 k=10 sel=8be3456b4875ad2e minulps=0 mintok=0 x=2007.44902 logit=3257.98438 exp=1637.01024 shr=10.4202962 lines=422 END +moesel call=230 dev=1 T=1 E=512 k=10 sel=f9b2fcbf4e94b8c9 minulps=0 mintok=0 x=2039.25974 logit=3090.07812 exp=1880.78895 shr=14.8957555 lines=423 END +moesel call=231 dev=1 T=1 E=512 k=10 sel=44e72e18b98bed12 minulps=1 mintok=0 x=2088.80882 logit=3082.3125 exp=1784.81803 shr=22.1053065 lines=424 END +moesel call=232 dev=1 T=1 E=512 k=10 sel=09f0b82e8cf730f3 minulps=0 mintok=0 x=2106.75727 logit=3139.67188 exp=1539.96297 shr=14.7666921 lines=425 END +moesel call=233 dev=1 T=1 E=512 k=10 sel=0dff399b4770ee10 minulps=3 mintok=0 x=2241.38857 logit=3236.9375 exp=1788.18843 shr=12.8611548 lines=426 END +moesel call=234 dev=1 T=1 E=512 k=10 sel=e2b1245a1d963c28 minulps=0 mintok=0 x=2270.07075 logit=3207.75 exp=1944.56974 shr=23.0768353 lines=427 END +moesel call=235 dev=1 T=1 E=512 k=10 sel=34ecff52f133b24d minulps=1 mintok=0 x=1875.09357 logit=3110.0625 exp=1343.50007 shr=21.7689148 lines=428 END +moesel call=236 dev=1 T=1 E=512 k=10 sel=2ca538f02e32bb3c minulps=1 mintok=0 x=2224.76821 logit=3237.51562 exp=1791.08428 shr=27.2459818 lines=429 END +moesel call=237 dev=1 T=1 E=512 k=10 sel=3b5325c7ff30efba minulps=0 mintok=0 x=2308.26959 logit=3261.21875 exp=1658.76712 shr=30.9008512 lines=430 END +moesel call=238 dev=1 T=1 E=512 k=10 sel=4e3559090e65c272 minulps=0 mintok=0 x=2163.29301 logit=3203.39062 exp=1631.63319 shr=37.0887284 lines=431 END +moesel call=239 dev=1 T=1 E=512 k=10 sel=08c9ed7a4760f705 minulps=1 mintok=0 x=1978.85175 logit=3204.96875 exp=1603.01215 shr=144.048051 lines=432 END +moesel call=240 dev=1 T=1 E=512 k=10 sel=61b49097fc368a4f minulps=1 mintok=0 x=719.735275 logit=3487.75 exp=482.559906 shr=54.8279501 lines=433 END +moesel call=241 dev=1 T=1 E=512 k=10 sel=dde584f7f77e8cb5 minulps=2 mintok=0 x=771.466531 logit=3071.32812 exp=416.046963 shr=5.94673173 lines=434 END +moesel call=242 dev=1 T=1 E=512 k=10 sel=aa7456477447c9db minulps=3 mintok=0 x=787.406551 logit=3065.8125 exp=400.104052 shr=7.02664079 lines=435 END +moesel call=243 dev=1 T=1 E=512 k=10 sel=fcadc8fa2617fb34 minulps=1 mintok=0 x=1125.81332 logit=3166.0625 exp=581.746621 shr=3.30102856 lines=436 END +moesel call=244 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1082.08723 logit=2970.46875 exp=526.837637 shr=5.57845163 lines=437 END +moesel call=245 dev=1 T=1 E=512 k=10 sel=8bc65aa86c856784 minulps=1 mintok=0 x=1124.86482 logit=3119.34375 exp=504.797825 shr=5.05789583 lines=438 END +moesel call=246 dev=1 T=1 E=512 k=10 sel=62cc47e80cc73128 minulps=2 mintok=0 x=1321.76281 logit=3161.79688 exp=674.849031 shr=8.12079952 lines=439 END +moesel call=247 dev=1 T=1 E=512 k=10 sel=b7771ebaf5fb393a minulps=0 mintok=0 x=1168.06668 logit=3084.57812 exp=564.712501 shr=6.24083661 lines=440 END +moesel call=248 dev=1 T=1 E=512 k=10 sel=0c140a8e49a781fd minulps=3 mintok=0 x=1171.76134 logit=3066.1875 exp=599.744879 shr=6.866799 lines=441 END +moesel call=249 dev=1 T=1 E=512 k=10 sel=507f458d7a0dd61f minulps=1 mintok=0 x=1196.92192 logit=3047.9375 exp=566.048916 shr=5.14612947 lines=442 END +moesel call=250 dev=1 T=1 E=512 k=10 sel=11fefbb433e81ea5 minulps=0 mintok=0 x=1073.28715 logit=3136.92188 exp=504.051555 shr=3.50103059 lines=443 END +moesel call=251 dev=1 T=1 E=512 k=10 sel=b01f165a2aed8d4b minulps=3 mintok=0 x=1061.5559 logit=3038 exp=453.041076 shr=2.42076577 lines=444 END +moesel call=252 dev=1 T=1 E=512 k=10 sel=92e084d08d5d3f4e minulps=1 mintok=0 x=1111.23433 logit=3064.73438 exp=519.329495 shr=5.51540724 lines=445 END +moesel call=253 dev=1 T=1 E=512 k=10 sel=d906c5f6f48f0d41 minulps=1 mintok=0 x=984.467606 logit=3042.57812 exp=418.224721 shr=5.32077738 lines=446 END +moesel call=254 dev=1 T=1 E=512 k=10 sel=e73cb09b45b079b4 minulps=2 mintok=0 x=1224.14796 logit=3088.85938 exp=596.884094 shr=4.78430417 lines=447 END +moesel call=255 dev=1 T=1 E=512 k=10 sel=036ca995574452fa minulps=1 mintok=0 x=1623.95516 logit=3098.45312 exp=885.750901 shr=6.6510445 lines=448 END +moesel call=256 dev=1 T=1 E=512 k=10 sel=2c97bea4acec81a4 minulps=1 mintok=0 x=1440.77504 logit=3077.23438 exp=691.724926 shr=6.9816874 lines=449 END +moesel call=257 dev=1 T=1 E=512 k=10 sel=5beb06c103212760 minulps=2 mintok=0 x=1211.88206 logit=3146.01562 exp=610.742811 shr=5.3325901 lines=450 END +moesel call=258 dev=1 T=1 E=512 k=10 sel=cf56b1fef245c1b2 minulps=1 mintok=0 x=1087.62227 logit=3217.76562 exp=469.492844 shr=3.7592778 lines=451 END +moesel call=259 dev=1 T=1 E=512 k=10 sel=f1f15ef0786e7166 minulps=1 mintok=0 x=1260.5826 logit=3234.1875 exp=592.127562 shr=5.12547102 lines=452 END +moesel call=260 dev=1 T=1 E=512 k=10 sel=8cc149cf77964f24 minulps=1 mintok=0 x=1488.14397 logit=3273.53125 exp=789.547055 shr=8.75632502 lines=453 END +moesel call=261 dev=1 T=1 E=512 k=10 sel=a6b70eb4e1969847 minulps=3 mintok=0 x=1418.41927 logit=3256.96875 exp=685.589609 shr=4.91994519 lines=454 END +moesel call=262 dev=1 T=1 E=512 k=10 sel=7b7a3168b2ff06a2 minulps=6 mintok=0 x=1356.91032 logit=3266.42188 exp=664.668233 shr=4.00647543 lines=455 END +moesel call=263 dev=1 T=1 E=512 k=10 sel=f5590947f667141f minulps=0 mintok=0 x=1612.18464 logit=3188.35938 exp=874.37565 shr=7.22029203 lines=456 END +moesel call=264 dev=1 T=1 E=512 k=10 sel=7434b6bf2c085707 minulps=9 mintok=0 x=1213.43486 logit=3209.92188 exp=587.864345 shr=5.35805188 lines=457 END +moesel call=265 dev=1 T=1 E=512 k=10 sel=1d5b6fd1434dd984 minulps=4 mintok=0 x=1300.87791 logit=3272.32812 exp=610.208423 shr=5.07808681 lines=458 END +moesel call=266 dev=1 T=1 E=512 k=10 sel=933d7afaff8cc437 minulps=9 mintok=0 x=1113.05272 logit=3254.85156 exp=552.016545 shr=3.71943129 lines=459 END +moesel call=267 dev=1 T=1 E=512 k=10 sel=8ecaa8950e4d5f9d minulps=2 mintok=0 x=1190.53572 logit=3156.84375 exp=527.462512 shr=4.91369309 lines=460 END +moesel call=268 dev=1 T=1 E=512 k=10 sel=78e9d07fe08d6249 minulps=1 mintok=0 x=1243.89803 logit=3312.45312 exp=632.998852 shr=5.97065023 lines=461 END +moesel call=269 dev=1 T=1 E=512 k=10 sel=a4fdb20db9cc56a8 minulps=0 mintok=0 x=1238.8786 logit=3271.8125 exp=660.144664 shr=9.16139148 lines=462 END +moesel call=270 dev=1 T=1 E=512 k=10 sel=24eddf1056185207 minulps=2 mintok=0 x=1450.94733 logit=3379.29688 exp=770.611199 shr=7.09821257 lines=463 END +moesel call=271 dev=1 T=1 E=512 k=10 sel=6871cc1e230785ad minulps=2 mintok=0 x=1706.86826 logit=3224.35938 exp=1037.66972 shr=11.8690866 lines=464 END +moesel call=272 dev=1 T=1 E=512 k=10 sel=e3f5190e6d703653 minulps=2 mintok=0 x=1596.52113 logit=3170.98438 exp=911.640029 shr=11.5912727 lines=465 END +moesel call=273 dev=1 T=1 E=512 k=10 sel=2dfd19a0fc02b740 minulps=2 mintok=0 x=1459.34425 logit=3273.57812 exp=834.476069 shr=7.15768957 lines=466 END +moesel call=274 dev=1 T=1 E=512 k=10 sel=e08075eec512f324 minulps=6 mintok=0 x=1384.07997 logit=3403.46875 exp=800.853358 shr=5.02991112 lines=467 END +moesel call=275 dev=1 T=1 E=512 k=10 sel=5447e5522c9978ba minulps=0 mintok=0 x=1565.34497 logit=3261.125 exp=949.541952 shr=7.78171557 lines=468 END +moesel call=276 dev=1 T=1 E=512 k=10 sel=52face1eca90650c minulps=3 mintok=0 x=1849.35504 logit=3320.70312 exp=1248.77513 shr=18.1390751 lines=469 END +moesel call=277 dev=1 T=1 E=512 k=10 sel=aba9dd5f26da312b minulps=3 mintok=0 x=1773.53765 logit=3290.03125 exp=1154.50591 shr=9.91195093 lines=470 END +moesel call=278 dev=1 T=1 E=512 k=10 sel=06c460b9e38d97eb minulps=2 mintok=0 x=1860.26612 logit=3194.70312 exp=1618.3548 shr=12.8954418 lines=471 END +moesel call=279 dev=1 T=1 E=512 k=10 sel=ab07fba4d130b9e5 minulps=0 mintok=0 x=1894.73928 logit=3038.07812 exp=1262.50809 shr=21.319008 lines=472 END +moesel call=280 dev=1 T=1 E=512 k=10 sel=f51bd81f34330756 minulps=0 mintok=0 x=2039.29698 logit=3141.40625 exp=1378.85824 shr=14.5558079 lines=473 END +moesel call=281 dev=1 T=1 E=512 k=10 sel=38ba42a21cb96827 minulps=5 mintok=0 x=2034.52425 logit=3255.6875 exp=1556.42675 shr=10.1677739 lines=474 END +moesel call=282 dev=1 T=1 E=512 k=10 sel=caea884e4f8a421b minulps=3 mintok=0 x=2201.27892 logit=3181.125 exp=1767.24185 shr=21.6390068 lines=475 END +moesel call=283 dev=1 T=1 E=512 k=10 sel=85126cca847fe0ae minulps=0 mintok=0 x=1888.35798 logit=3154.57812 exp=1305.97696 shr=18.1589696 lines=476 END +moesel call=284 dev=1 T=1 E=512 k=10 sel=73334fc721c55436 minulps=3 mintok=0 x=2119.56377 logit=3171.78125 exp=1850.2893 shr=21.9173639 lines=477 END +moesel call=285 dev=1 T=1 E=512 k=10 sel=e51cb50a0f7c4c64 minulps=2 mintok=0 x=2235.23249 logit=3218.25 exp=1667.06572 shr=24.3306805 lines=478 END +moesel call=286 dev=1 T=1 E=512 k=10 sel=3ff312bf57506b4c minulps=0 mintok=0 x=2157.19093 logit=3175.09375 exp=1583.27752 shr=35.8604115 lines=479 END +moesel call=287 dev=1 T=1 E=512 k=10 sel=a71eb83b635761a7 minulps=2 mintok=0 x=1994.61945 logit=3186.25 exp=1613.20293 shr=139.88128 lines=480 END +moesel call=288 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=1 mintok=0 x=720.908846 logit=3493.3125 exp=470.299978 shr=56.0763303 lines=481 END +moesel call=289 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=0 mintok=0 x=770.535929 logit=3054.75 exp=390.753702 shr=6.59788024 lines=482 END +moesel call=290 dev=1 T=1 E=512 k=10 sel=19a4aa22fd55ceb9 minulps=1 mintok=0 x=797.194718 logit=3063.875 exp=449.164568 shr=7.77219334 lines=483 END +moesel call=291 dev=1 T=1 E=512 k=10 sel=2a01e5c40d6cc0a3 minulps=0 mintok=0 x=1130.49359 logit=3148.71875 exp=630.243819 shr=3.22358381 lines=484 END +moesel call=292 dev=1 T=1 E=512 k=10 sel=e2bc5fe0c2cd0efa minulps=2 mintok=0 x=1109.70885 logit=3062.89062 exp=617.032947 shr=4.58339378 lines=485 END +moesel call=293 dev=1 T=1 E=512 k=10 sel=25a7c1bf08883893 minulps=1 mintok=0 x=1195.05093 logit=3230.92188 exp=558.708276 shr=5.87885763 lines=486 END +moesel call=294 dev=1 T=1 E=512 k=10 sel=fe9752fc405e7f45 minulps=1 mintok=0 x=1351.35708 logit=3224.25 exp=758.855099 shr=8.13239998 lines=487 END +moesel call=295 dev=1 T=1 E=512 k=10 sel=23e39c29cd78e156 minulps=1 mintok=0 x=1231.14917 logit=3168.6875 exp=640.549042 shr=6.48122781 lines=488 END +moesel call=296 dev=1 T=1 E=512 k=10 sel=335dd0bd48b7fead minulps=0 mintok=0 x=1251.82667 logit=3166.5 exp=653.138235 shr=7.89882357 lines=489 END +moesel call=297 dev=1 T=1 E=512 k=10 sel=1e6ef2e2bd5bbe6e minulps=1 mintok=0 x=1291.58003 logit=3089.85938 exp=646.445662 shr=5.78292927 lines=490 END +moesel call=298 dev=1 T=1 E=512 k=10 sel=6d9a04e77a545573 minulps=1 mintok=0 x=1128.54297 logit=3227.82031 exp=573.681107 shr=4.35932608 lines=491 END +moesel call=299 dev=1 T=1 E=512 k=10 sel=431ad89c3a34fdb5 minulps=3 mintok=0 x=1160.22196 logit=3076.54688 exp=552.155695 shr=3.10065097 lines=492 END +moesel call=300 dev=1 T=1 E=512 k=10 sel=98cbb41200451908 minulps=2 mintok=0 x=1205.60989 logit=3072.40625 exp=588.010341 shr=4.60971649 lines=493 END +moesel call=301 dev=1 T=1 E=512 k=10 sel=cee73503e291c554 minulps=0 mintok=0 x=1049.94109 logit=3148.65625 exp=465.748932 shr=5.53982265 lines=494 END +moesel call=302 dev=1 T=1 E=512 k=10 sel=cbcf5808f868f2ee minulps=2 mintok=0 x=1300.08776 logit=3157.73438 exp=787.313637 shr=5.8626614 lines=495 END +moesel call=303 dev=1 T=1 E=512 k=10 sel=a176075303636b24 minulps=0 mintok=0 x=1626.80483 logit=3080.9375 exp=892.795173 shr=9.08575885 lines=496 END +moesel call=304 dev=1 T=1 E=512 k=10 sel=81f99954a1527b57 minulps=2 mintok=0 x=1465.01153 logit=3071.3125 exp=722.733711 shr=6.2803769 lines=497 END +moesel call=305 dev=1 T=1 E=512 k=10 sel=916f8776965ecba6 minulps=1 mintok=0 x=1268.95674 logit=3154.8125 exp=594.541258 shr=4.35259996 lines=498 END +moesel call=306 dev=1 T=1 E=512 k=10 sel=23826f8f1cfaf52c minulps=0 mintok=0 x=1170.08739 logit=3193.98438 exp=583.54809 shr=3.92149412 lines=499 END +moesel call=307 dev=1 T=1 E=512 k=10 sel=454397dc42f5b948 minulps=0 mintok=0 x=1291.48043 logit=3151.01562 exp=653.289927 shr=4.64947504 lines=500 END +moesel call=308 dev=1 T=1 E=512 k=10 sel=a693e98db6eaab69 minulps=1 mintok=0 x=1532.17177 logit=3179.4375 exp=895.965714 shr=9.50969727 lines=501 END +moesel call=309 dev=1 T=1 E=512 k=10 sel=e60de2e9e70d1f03 minulps=0 mintok=0 x=1490.63683 logit=3207.1875 exp=796.308669 shr=5.03227582 lines=502 END +moesel call=310 dev=1 T=1 E=512 k=10 sel=eda1b89e68411e03 minulps=5 mintok=0 x=1412.96221 logit=3241.04688 exp=750.6313 shr=4.91231269 lines=503 END +moesel call=311 dev=1 T=1 E=512 k=10 sel=492c4ee841cd0697 minulps=0 mintok=0 x=1547.72312 logit=3138.32812 exp=746.602557 shr=12.6945544 lines=504 END +moesel call=312 dev=1 T=1 E=512 k=10 sel=bb55ca07e4958d50 minulps=10 mintok=0 x=1232.44339 logit=3230.60938 exp=618.057561 shr=5.98809077 lines=505 END +moesel call=313 dev=1 T=1 E=512 k=10 sel=33bd5f8fd4274cde minulps=2 mintok=0 x=1307.53513 logit=3320.01562 exp=618.332616 shr=5.09537409 lines=506 END +moesel call=314 dev=1 T=1 E=512 k=10 sel=8d9f1ba184eaaf3c minulps=1 mintok=0 x=1146.61759 logit=3333.82812 exp=555.122777 shr=3.84388512 lines=507 END +moesel call=315 dev=1 T=1 E=512 k=10 sel=8f52b8b32a27c50c minulps=3 mintok=0 x=1203.73064 logit=3289.34375 exp=518.891168 shr=3.55087181 lines=508 END +moesel call=316 dev=1 T=1 E=512 k=10 sel=b06159110032b073 minulps=0 mintok=0 x=1267.07388 logit=3378.70312 exp=638.88724 shr=4.50843236 lines=509 END +moesel call=317 dev=1 T=1 E=512 k=10 sel=5ac221458ae3f815 minulps=0 mintok=0 x=1248.944 logit=3323.96875 exp=663.817585 shr=6.28959796 lines=510 END +moesel call=318 dev=1 T=1 E=512 k=10 sel=3692915ef113b95c minulps=6 mintok=0 x=1543.87815 logit=3373.25 exp=906.074797 shr=4.30320768 lines=511 END +moesel call=319 dev=1 T=1 E=512 k=10 sel=5654c4f757f5adc0 minulps=1 mintok=0 x=1784.87576 logit=3225.09375 exp=1127.58558 shr=6.99982764 lines=512 END +moesel call=320 dev=1 T=1 E=512 k=10 sel=cce9d411277ac7a0 minulps=2 mintok=0 x=1731.88059 logit=3257.48438 exp=954.358576 shr=6.33040473 lines=513 END +moesel call=321 dev=1 T=1 E=512 k=10 sel=df028f29b5993b25 minulps=2 mintok=0 x=1564.45086 logit=3376.35938 exp=917.965805 shr=4.72212958 lines=514 END +moesel call=322 dev=1 T=1 E=512 k=10 sel=c2f29cfbcc871c32 minulps=1 mintok=0 x=1470.7524 logit=3463.03125 exp=889.904336 shr=4.56328436 lines=515 END +moesel call=323 dev=1 T=1 E=512 k=10 sel=a08c6e7a5893f6cd minulps=10 mintok=0 x=1659.91617 logit=3319.4375 exp=1033.99195 shr=5.26986123 lines=516 END +moesel call=324 dev=1 T=1 E=512 k=10 sel=f9ee760e41a3c91d minulps=0 mintok=0 x=2027.81188 logit=3398.8125 exp=1750.47563 shr=13.1384588 lines=517 END +moesel call=325 dev=1 T=1 E=512 k=10 sel=e178771deea6c5ce minulps=7 mintok=0 x=1956.54692 logit=3331.42188 exp=1368.12519 shr=6.88059388 lines=518 END +moesel call=326 dev=1 T=1 E=512 k=10 sel=0fafc2fab555f6e9 minulps=1 mintok=0 x=1991.47472 logit=3239 exp=1614.7162 shr=13.9327306 lines=519 END +moesel call=327 dev=1 T=1 E=512 k=10 sel=6c7e393659972261 minulps=1 mintok=0 x=1906.79834 logit=3088.10938 exp=1370.20132 shr=17.0168733 lines=520 END +moesel call=328 dev=1 T=1 E=512 k=10 sel=8f36dc7be08e3931 minulps=2 mintok=0 x=2039.20587 logit=3143.9375 exp=1405.13947 shr=11.7286562 lines=521 END +moesel call=329 dev=1 T=1 E=512 k=10 sel=9e648ffe259dfbae minulps=3 mintok=0 x=2057.31357 logit=3195.73438 exp=1494.29283 shr=9.69284485 lines=522 END +moesel call=330 dev=1 T=1 E=512 k=10 sel=2982a17c543ebe92 minulps=0 mintok=0 x=2169.32461 logit=3203.21875 exp=1812.67176 shr=20.3597129 lines=523 END +moesel call=331 dev=1 T=1 E=512 k=10 sel=91e5443eedd29105 minulps=3 mintok=0 x=1929.6716 logit=3095.28125 exp=1407.73551 shr=16.6108131 lines=524 END +moesel call=332 dev=1 T=1 E=512 k=10 sel=e656fd5283a60b62 minulps=1 mintok=0 x=2128.8884 logit=3177 exp=1691.97369 shr=23.9393592 lines=525 END +moesel call=333 dev=1 T=1 E=512 k=10 sel=44a985e32c2f6607 minulps=1 mintok=0 x=2143.73433 logit=3217.01562 exp=1485.41742 shr=30.2132944 lines=526 END +moesel call=334 dev=1 T=1 E=512 k=10 sel=7ffb8d605828846f minulps=1 mintok=0 x=2044.79437 logit=3170.875 exp=1378.33216 shr=33.7946244 lines=527 END +moesel call=335 dev=1 T=1 E=512 k=10 sel=4ff4040682a26031 minulps=0 mintok=0 x=1870.00026 logit=3166.34375 exp=1537.97981 shr=141.048263 lines=528 END +moesel call=336 dev=1 T=1 E=512 k=10 sel=44cde1097e7fef30 minulps=0 mintok=0 x=720.015304 logit=3494.46875 exp=464.331417 shr=56.7611626 lines=529 END +moesel call=337 dev=1 T=1 E=512 k=10 sel=dd39d1545eb3721e minulps=1 mintok=0 x=769.239557 logit=3052.85938 exp=383.075755 shr=6.90563814 lines=530 END +moesel call=338 dev=1 T=1 E=512 k=10 sel=da8470c4d884deef minulps=2 mintok=0 x=781.241141 logit=3044.625 exp=450.116965 shr=8.03351162 lines=531 END +moesel call=339 dev=1 T=1 E=512 k=10 sel=d71ea9ac6dcf42f0 minulps=0 mintok=0 x=1123.97446 logit=3109.09375 exp=528.433402 shr=3.14524879 lines=532 END +moesel call=340 dev=1 T=1 E=512 k=10 sel=11e9c52f473f20ca minulps=1 mintok=0 x=1108.30387 logit=3072 exp=595.020402 shr=4.04507779 lines=533 END +moesel call=341 dev=1 T=1 E=512 k=10 sel=ccab85420d64391b minulps=0 mintok=0 x=1140.24286 logit=3136.26562 exp=521.452564 shr=5.89650467 lines=534 END +moesel call=342 dev=1 T=1 E=512 k=10 sel=b0359bc2c09a1b3b minulps=2 mintok=0 x=1284.18058 logit=3175.59375 exp=695.144514 shr=7.5284667 lines=535 END +moesel call=343 dev=1 T=1 E=512 k=10 sel=b8766a294ccc5e15 minulps=3 mintok=0 x=1179.17826 logit=3163.95312 exp=592.168702 shr=7.01145951 lines=536 END +moesel call=344 dev=1 T=1 E=512 k=10 sel=39dd045c2a0bf426 minulps=1 mintok=0 x=1173.72122 logit=3133.8125 exp=598.289271 shr=8.36464005 lines=537 END +moesel call=345 dev=1 T=1 E=512 k=10 sel=c8b450035b8bad2b minulps=0 mintok=0 x=1196.105 logit=3062.15625 exp=590.992329 shr=6.52784037 lines=538 END +moesel call=346 dev=1 T=1 E=512 k=10 sel=2a7287b05a0cab3f minulps=2 mintok=0 x=1093.39694 logit=3217.6875 exp=582.437427 shr=3.78586096 lines=539 END +moesel call=347 dev=1 T=1 E=512 k=10 sel=9f3b3343e48ffbf5 minulps=0 mintok=0 x=1171.20684 logit=3088.54688 exp=539.94768 shr=2.9270583 lines=540 END +moesel call=348 dev=1 T=1 E=512 k=10 sel=7fe4fe5d8951d439 minulps=3 mintok=0 x=1199.96908 logit=3061.85938 exp=600.276196 shr=4.67941533 lines=541 END +moesel call=349 dev=1 T=1 E=512 k=10 sel=7b00267773828180 minulps=0 mintok=0 x=1068.80267 logit=3080.96875 exp=474.210937 shr=5.04527323 lines=542 END +moesel call=350 dev=1 T=1 E=512 k=10 sel=3c5c42aa762d7ed2 minulps=4 mintok=0 x=1269.14098 logit=3085.70312 exp=730.125101 shr=4.51368469 lines=543 END +moesel call=351 dev=1 T=1 E=512 k=10 sel=deaab1e1a7541b87 minulps=2 mintok=0 x=1619.64559 logit=3075.14062 exp=874.925515 shr=8.89459471 lines=544 END +moesel call=352 dev=1 T=1 E=512 k=10 sel=6e5d044cf1fc7022 minulps=1 mintok=0 x=1484.82882 logit=3053.96875 exp=733.953595 shr=6.29550232 lines=545 END +moesel call=353 dev=1 T=1 E=512 k=10 sel=aa42bad017dc8cbc minulps=2 mintok=0 x=1289.43275 logit=3151.95312 exp=615.826015 shr=4.31546008 lines=546 END +moesel call=354 dev=1 T=1 E=512 k=10 sel=315b9f288ff0a182 minulps=3 mintok=0 x=1178.73817 logit=3183.14062 exp=547.426989 shr=3.78970516 lines=547 END +moesel call=355 dev=1 T=1 E=512 k=10 sel=bdb81852e778bc90 minulps=0 mintok=0 x=1329.85278 logit=3159.40625 exp=636.104008 shr=4.7916096 lines=548 END +moesel call=356 dev=1 T=1 E=512 k=10 sel=7034143c6c58be41 minulps=0 mintok=0 x=1552.82118 logit=3206.71875 exp=872.151475 shr=7.76180266 lines=549 END +moesel call=357 dev=1 T=1 E=512 k=10 sel=4cc169bd11ce9a33 minulps=0 mintok=0 x=1496.55086 logit=3206.70312 exp=733.508479 shr=5.00650799 lines=550 END +moesel call=358 dev=1 T=1 E=512 k=10 sel=4862310eebbd815e minulps=1 mintok=0 x=1348.99547 logit=3195.28125 exp=690.420928 shr=4.49610108 lines=551 END +moesel call=359 dev=1 T=1 E=512 k=10 sel=d5ecc24ee49916ac minulps=0 mintok=0 x=1622.07154 logit=3201.65625 exp=865.30765 shr=8.74974447 lines=552 END +moesel call=360 dev=1 T=1 E=512 k=10 sel=963b0fbb9cb6e81f minulps=0 mintok=0 x=1255.61019 logit=3156.42188 exp=659.038896 shr=5.84109655 lines=553 END +moesel call=361 dev=1 T=1 E=512 k=10 sel=1edc464c7723fe54 minulps=1 mintok=0 x=1336.06516 logit=3293.21875 exp=630.459126 shr=4.41336972 lines=554 END +moesel call=362 dev=1 T=1 E=512 k=10 sel=2626a43dbee38f3b minulps=3 mintok=0 x=1133.96029 logit=3218.69531 exp=583.583901 shr=4.19799819 lines=555 END +moesel call=363 dev=1 T=1 E=512 k=10 sel=80f08432dc533ac3 minulps=0 mintok=0 x=1200.27544 logit=3197.39062 exp=550.233979 shr=4.26960437 lines=556 END +moesel call=364 dev=1 T=1 E=512 k=10 sel=4de5de32929854dd minulps=1 mintok=0 x=1255.59562 logit=3333.96875 exp=631.117934 shr=3.87706444 lines=557 END +moesel call=365 dev=1 T=1 E=512 k=10 sel=7d8ae07aa246693f minulps=0 mintok=0 x=1252.15723 logit=3249.26562 exp=671.915036 shr=7.54002076 lines=558 END +moesel call=366 dev=1 T=1 E=512 k=10 sel=2c34bfad57a0f504 minulps=1 mintok=0 x=1507.71261 logit=3304.70312 exp=841.009994 shr=6.10757782 lines=559 END +moesel call=367 dev=1 T=1 E=512 k=10 sel=68714ec82c41b541 minulps=0 mintok=0 x=1763.37123 logit=3169.03125 exp=1111.12199 shr=12.374807 lines=560 END +moesel call=368 dev=1 T=1 E=512 k=10 sel=1f22675012ab1ea4 minulps=1 mintok=0 x=1654.11779 logit=3173.51562 exp=915.490735 shr=12.4345397 lines=561 END +moesel call=369 dev=1 T=1 E=512 k=10 sel=fd883978fa602ccb minulps=1 mintok=0 x=1490.08333 logit=3295.10938 exp=841.559377 shr=6.13698708 lines=562 END +moesel call=370 dev=1 T=1 E=512 k=10 sel=1298f8a8c115ca6d minulps=0 mintok=0 x=1382.55747 logit=3363.67188 exp=795.375256 shr=4.93426613 lines=563 END +moesel call=371 dev=1 T=1 E=512 k=10 sel=fafa416a41510e2b minulps=1 mintok=0 x=1588.0379 logit=3263.5625 exp=955.51882 shr=6.98780198 lines=564 END +moesel call=372 dev=1 T=1 E=512 k=10 sel=7fce82859a778218 minulps=1 mintok=0 x=1938.96774 logit=3336.21875 exp=1294.53322 shr=15.6524524 lines=565 END +moesel call=373 dev=1 T=1 E=512 k=10 sel=bcc944daed06c12b minulps=2 mintok=0 x=1902.17242 logit=3328.4375 exp=1293.49587 shr=8.27988695 lines=566 END +moesel call=374 dev=1 T=1 E=512 k=10 sel=9565301592511e79 minulps=1 mintok=0 x=1912.38708 logit=3257.60938 exp=1721.68549 shr=12.9984499 lines=567 END +moesel call=375 dev=1 T=1 E=512 k=10 sel=11e0a25c4aae080d minulps=2 mintok=0 x=1976.4549 logit=3087.9375 exp=1416.52003 shr=16.9319087 lines=568 END +moesel call=376 dev=1 T=1 E=512 k=10 sel=93e6ccc0425767d4 minulps=3 mintok=0 x=2087.62383 logit=3137.17188 exp=1523.74264 shr=11.2343423 lines=569 END +moesel call=377 dev=1 T=1 E=512 k=10 sel=34ffeb576a4d4586 minulps=2 mintok=0 x=2120.68981 logit=3262.29688 exp=1654.40233 shr=9.1849984 lines=570 END +moesel call=378 dev=1 T=1 E=512 k=10 sel=b5579a72498d18d8 minulps=0 mintok=0 x=2282.62504 logit=3218.20312 exp=1797.38524 shr=18.6403076 lines=571 END +moesel call=379 dev=1 T=1 E=512 k=10 sel=ccb1f591af99da75 minulps=0 mintok=0 x=1924.06312 logit=3161.35938 exp=1337.21027 shr=12.6647484 lines=572 END +moesel call=380 dev=1 T=1 E=512 k=10 sel=f239f060355c60fd minulps=0 mintok=0 x=2133.93015 logit=3152.23438 exp=1598.71725 shr=18.1373025 lines=573 END +moesel call=381 dev=1 T=1 E=512 k=10 sel=f83d76c3622c532f minulps=1 mintok=0 x=2125.87638 logit=3165.73438 exp=1455.09365 shr=19.723922 lines=574 END +moesel call=382 dev=1 T=1 E=512 k=10 sel=d81279237fcb3ff7 minulps=5 mintok=0 x=2100.57419 logit=3147.04688 exp=1383.72128 shr=29.6997827 lines=575 END +moesel call=383 dev=1 T=1 E=512 k=10 sel=dafd3703f38e1353 minulps=0 mintok=0 x=1889.96341 logit=3089.34375 exp=1556.76733 shr=114.923461 lines=576 END diff --git a/docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt b/docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt new file mode 100644 index 000000000..af660cbf0 --- /dev/null +++ b/docs/bench-evidence/qwen4exp-moe-selection-20260902/results.txt @@ -0,0 +1,75 @@ +RESULT GPU: NVIDIA Thor, 11.0, 595.78 +RESULT NVCC: Cuda compilation tools, release 13.0, V13.0.88 Build cuda_13.0.r13.0/compiler.36424714_0 +RESULT SOURCE TARBALL sha256=be5f31ec1d0b3c7ce4ed724647cd9295caa48db62dbff1689e501654d775fd44 +RESULT SOURCE TARBALL HEAD: 8e2ba6931dd5aae33955ffdd8e7258ff5d90eef0 +RESULT INSTRUMENT IN SOURCE: MoeSelFp=2(2) env=1(1) +RESULT ARTIFACT STAGE REUSED (no copy) +RESULT ARTIFACT VERIFIED: shard1 sha256=88a1420825a9304063e882ada29d438263617f51ac8923d438d927496693bafd bytes=72546461650 +RESULT CMAKE rc=0 +RESULT BUILD rc=0 wall=921s objects=585 +RESULT BINARY: /tmp/moediv-bld/examples/vllm-server sha256=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf built=2026-09-02T07:06:21Z +RESULT BINARY CARRIES THE INSTRUMENT: strings_hits=1 +RESULT TAP GATE OFF rc=0 [doctest] assertions: 121 | 121 passed | 0 failed | +RESULT TAP GATE ON rc=0 [doctest] assertions: 172 | 172 passed | 0 failed | +RESULT ARM CPU-CTRL START bin_sha=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf device=cpu env='VT_CPU_QUANT_REPACK=0' +RESULT ARM CPU-CTRL HEALTH up=1 load_wall=45s +RESULT ARM CPU-CTRL REQ http=200 curl_rc=0 wall=6s +RESULT ARM CPU-CTRL TOKEN IDS: 11751 13 15767 411 2029 11 1092 369 +RESULT ARM CPU-CTRL MATCHES CONTROL: YES +RESULT ARM CPU-CTRL MOESEL LINES: 960 +RESULT ARM CPU-CTRL MOESEL DIGESTS: 384 +RESULT ARM CPU-CTRL MOESEL LAST lines=: lines=576 +RESULT ARM CPU-CTRL PREFILL CALLS (T=5): 48 +RESULT ARM CPU-CTRL steps completed: 11 +RESULT ARM CPU-CTRL FIRST ERROR LINE: +RESULT ARM CPU-CTRL END +RESULT ARM CUDA-PROD START bin_sha=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf device=cuda env='VT_CPU_QUANT_REPACK=0' +RESULT ARM CUDA-PROD HEALTH up=1 load_wall=15s +RESULT ARM CUDA-PROD REQ http=200 curl_rc=0 wall=156s +RESULT ARM CUDA-PROD TOKEN IDS: 11751 271 271 271 271 271 0 0 +RESULT ARM CUDA-PROD MATCHES CONTROL: NO (control='11751 13 15767 411 2029 11 1092 369') +RESULT ARM CUDA-PROD MOESEL LINES: 960 +RESULT ARM CUDA-PROD MOESEL DIGESTS: 384 +RESULT ARM CUDA-PROD MOESEL LAST lines=: lines=576 +RESULT ARM CUDA-PROD PREFILL CALLS (T=5): 48 +RESULT ARM CUDA-PROD steps completed: 11 +RESULT ARM CUDA-PROD FIRST ERROR LINE: +RESULT ARM CUDA-PROD END +RESULT ARM CUDA-GDNSEQ START bin_sha=b3d5d97c86ffa75f6de9e70e7a036f495962c39c8d76d8ed182b9a44c83beacf device=cuda env='VT_CPU_QUANT_REPACK=0 VT_GDN_CHUNKED=0' +RESULT ARM CUDA-GDNSEQ HEALTH up=1 load_wall=15s +RESULT ARM CUDA-GDNSEQ REQ http=200 curl_rc=0 wall=223s +RESULT ARM CUDA-GDNSEQ TOKEN IDS: 11751 271 271 11751 271 271 271 0 +RESULT ARM CUDA-GDNSEQ MATCHES CONTROL: NO (control='11751 13 15767 411 2029 11 1092 369') +RESULT ARM CUDA-GDNSEQ MOESEL LINES: 960 +RESULT ARM CUDA-GDNSEQ MOESEL DIGESTS: 384 +RESULT ARM CUDA-GDNSEQ MOESEL LAST lines=: lines=576 +RESULT ARM CUDA-GDNSEQ PREFILL CALLS (T=5): 48 +RESULT ARM CUDA-GDNSEQ steps completed: 11 +RESULT ARM CUDA-GDNSEQ FIRST ERROR LINE: +RESULT ARM CUDA-GDNSEQ END +RESULT CMP CUDA-PROD rc=0 lines=80 +RESULT CMP CUDA-PROD COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +RESULT CMP CUDA-PROD COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-PROD COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-PROD COUNT PREFILL calls=48 sel-hash MISMATCH=40 +RESULT CMP CUDA-PROD COUNT PREFILL token-slots=240 FLIPPED=78 +RESULT CMP CUDA-PROD FIRST PREFILL FLIP call=0 tok=2 +RESULT CMP CUDA-PROD MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +RESULT CMP CUDA-PROD COUNT DECODE calls=336 sel-hash MISMATCH=336 +RESULT CMP CUDA-PROD COUNT DECODE token-slots=336 FLIPPED=336 +RESULT CMP CUDA-PROD FIRST DECODE FLIP call=48 tok=0 +RESULT CMP CUDA-PROD MIN DECODE base minulps over calls: min=0 p10=0 median=1 +RESULT CMP CUDA-PROD VERDICT prefill expert-selection SET EQUALITY: DIFFERS (78 flipped token-slots) +RESULT CMP CUDA-GDNSEQ rc=0 lines=80 +RESULT CMP CUDA-GDNSEQ COUNT base_calls=384 other_calls=384 base_tok_lines=576 other_tok_lines=576 +RESULT CMP CUDA-GDNSEQ COUNTED-PROPERTY base: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-GDNSEQ COUNTED-PROPERTY other: last digest lines=576, value lines seen=576, agree=True +RESULT CMP CUDA-GDNSEQ COUNT PREFILL calls=48 sel-hash MISMATCH=38 +RESULT CMP CUDA-GDNSEQ COUNT PREFILL token-slots=240 FLIPPED=75 +RESULT CMP CUDA-GDNSEQ FIRST PREFILL FLIP call=1 tok=2 +RESULT CMP CUDA-GDNSEQ MIN PREFILL base minulps over calls: min=0 p10=0 median=0 +RESULT CMP CUDA-GDNSEQ COUNT DECODE calls=336 sel-hash MISMATCH=336 +RESULT CMP CUDA-GDNSEQ COUNT DECODE token-slots=336 FLIPPED=336 +RESULT CMP CUDA-GDNSEQ FIRST DECODE FLIP call=48 tok=0 +RESULT CMP CUDA-GDNSEQ MIN DECODE base minulps over calls: min=0 p10=0 median=1 +RESULT CMP CUDA-GDNSEQ VERDICT prefill expert-selection SET EQUALITY: DIFFERS (75 flipped token-slots) diff --git a/scripts/env-doc-allowlist.txt b/scripts/env-doc-allowlist.txt index cafe80bcb..df283d91d 100644 --- a/scripts/env-doc-allowlist.txt +++ b/scripts/env-doc-allowlist.txt @@ -154,6 +154,7 @@ VT_MOE_FUSED_W13 VT_MOE_GLUE_FUSE VT_MOE_HOST_FREE VT_MOE_PAD_STATS +VT_MOE_SEL_FP VT_MOE_SHARED_AUX_STREAM VT_MOE_SHARED_AUX_THRESHOLD VT_MOE_SPLIT_K diff --git a/src/vllm/model_executor/models/qwen3_5.cpp b/src/vllm/model_executor/models/qwen3_5.cpp index 269154c86..4c890c8af 100644 --- a/src/vllm/model_executor/models/qwen3_5.cpp +++ b/src/vllm/model_executor/models/qwen3_5.cpp @@ -6925,6 +6925,163 @@ DBuf MoeBlockBf16Cuda(Dev d, const MoeBlockWeights& w, const HfConfig& cfg, return dout; } +// ─── VT_MOE_SEL_FP — the SELECTED-EXPERT-ID tap (MOEDIV, #2552) ────────────── +// +// `VT_Q4EXP_LAYER_FP` (#2547) taps VALUES, and a value tap cannot separate an +// expert-selection FLIP from re-association inside the expert GEMM. A discrete +// selection has BIMODAL error and not a tolerance: a token's top-k set either +// changes — and that token's MoE output moves by an O(1) amount — or it does not +// change at all and the residue is rounding. One `rel(sum|x|)` averages the two +// together and destroys exactly the bit that says which happened. +// +// So this prints the SELECTION. Per token: the selected ids SORTED, because the +// assertion between two arms is SET equality and the selection ORDER is not part +// of it (`vt::MoeCombine` sums over the k slots and is order-invariant given the +// weights). Per call: `sel`, an FNV-1a hash over every token's sorted list, so +// comparing two arms' selections at one layer is ONE string comparison. +// +// THE MARGIN IS IN LOGIT SPACE AND ITS UNIT IS bf16 ULPS. Probability space is +// the wrong space: the softmax denominator is a device-order f32 reduction and +// differs between the arms, while the bf16 logits are what the selection is +// actually a function of. And a probability difference reads as "small" for a +// gap of one representable step and for a gap of fifty. `ulps` is the number of +// representable bf16 values between the largest REJECTED logit and the smallest +// SELECTED one under the sign-magnitude total order; `ulps == 0` means the two +// are the SAME bf16 value and the boundary was decided by the lowest-index +// tie-break, which one ulp of re-association anywhere upstream will flip. +// +// `lines=` IS THE COUNTED PROPERTY. An instrument that never ran and two arms +// whose taps agreed look identical in a diff. The measuring job asserts +// `lines == calls * T` and refuses to report a comparison otherwise. +// +// IT IS ON THE REFERENCE ARM ONLY, which is the arm a stacked keep-quant +// checkpoint takes and the only arm any `qwen4_exp` checkpoint reaches. The +// three fused CUDA arms keep their ids on device; adding a readback to a +// capturable path, to instrument a model that cannot enter it, would be dead +// code by construction. See the spec's "Wave MOEDIV" section. +int64_t MoeSelFpCalls() { + static const int64_t n = [] { + const char* e = std::getenv("VT_MOE_SEL_FP"); + if (e == nullptr || e[0] == '\0') return static_cast(0); + const long long parsed = std::atoll(e); + return parsed > 0 ? static_cast(parsed) : static_cast(0); + }(); + return n; +} +int64_t& MoeSelFpCall() { + static int64_t call = 0; + return call; +} +int64_t& MoeSelFpLines() { + static int64_t lines = 0; + return lines; +} + +// bf16 under the sign-magnitude TOTAL ORDER, as a signed key. Monotone in the +// value it encodes, so the difference of two keys counts representable steps. +// -0 and +0 both map to 0, which is what makes an exact tie read `ulps=0`. +int32_t Bf16Ordered(uint16_t b) { + return (b & 0x8000u) != 0 ? -static_cast(b & 0x7fffu) + : static_cast(b & 0x7fffu); +} + +double SumAbsBf16(const std::vector& v) { + double s = 0.0; + for (uint16_t b : v) { + const float f = vt::BF16ToF32(b); + if (std::isfinite(f)) s += std::fabs(static_cast(f)); + } + return s; +} + +// One tapped MoE block invocation. Every buffer here is one the reference path +// had ALREADY downloaded, so the tap moves no extra bytes off the device except +// the one guarded shared-expert read its caller passes in. +void MoeSelFp(int dev_type, int64_t T, int64_t E, int64_t top_k, + const std::vector& h, const std::vector& logits, + const std::vector& ids, + const std::vector& expert_out, + const std::vector* shared) { + const int64_t call = MoeSelFpCall(); + if (call >= MoeSelFpCalls()) return; + uint64_t sel = 1469598103934665603ull; // FNV-1a offset basis + int64_t min_ulps = -1; + int64_t min_tok = -1; + std::vector sorted(static_cast(top_k)); + std::vector picked(static_cast(E)); + for (int64_t t = 0; t < T; ++t) { + for (int64_t j = 0; j < top_k; ++j) + sorted[static_cast(j)] = ids[static_cast(t * top_k + j)]; + std::sort(sorted.begin(), sorted.end()); + std::string idstr; + for (int64_t j = 0; j < top_k; ++j) { + if (j != 0) idstr += ","; + idstr += std::to_string(sorted[static_cast(j)]); + // The hash is over the SORTED list, so it is a set identity and not a + // selection-order identity. + const uint32_t v = static_cast(sorted[static_cast(j)]); + for (int b = 0; b < 4; ++b) { + sel ^= static_cast((v >> (8 * b)) & 0xffu); + sel *= 1099511628211ull; + } + } + // The boundary: the smallest SELECTED logit against the largest REJECTED + // one, read from the bf16 the router GEMM actually stored. + for (int64_t e = 0; e < E; ++e) picked[static_cast(e)] = 0; + for (int64_t j = 0; j < top_k; ++j) + picked[static_cast(sorted[static_cast(j)])] = 1; + int64_t lo_e = -1, hi_e = -1; + for (int64_t e = 0; e < E; ++e) { + const uint16_t b = logits[static_cast(t * E + e)]; + if (picked[static_cast(e)] != 0) { + if (lo_e < 0 || Bf16Ordered(b) < Bf16Ordered(logits[static_cast(t * E + lo_e)])) + lo_e = e; + } else { + if (hi_e < 0 || Bf16Ordered(b) > Bf16Ordered(logits[static_cast(t * E + hi_e)])) + hi_e = e; + } + } + const uint16_t lo_b = lo_e >= 0 ? logits[static_cast(t * E + lo_e)] : 0; + const uint16_t hi_b = hi_e >= 0 ? logits[static_cast(t * E + hi_e)] : 0; + // top_k == E leaves nothing rejected; the boundary is then undefined and is + // reported as such rather than as a zero margin. + const int64_t ulps = hi_e >= 0 && lo_e >= 0 + ? static_cast(Bf16Ordered(lo_b)) - + static_cast(Bf16Ordered(hi_b)) + : -1; + if (ulps >= 0 && (min_ulps < 0 || ulps < min_ulps)) { + min_ulps = ulps; + min_tok = t; + } + ++MoeSelFpLines(); + std::fprintf(stderr, + "moesel call=%lld dev=%d tok=%lld ids=%s lo_e=%lld lo=%.9g " + "lo_raw=0x%04x hi_e=%lld hi=%.9g hi_raw=0x%04x margin=%.9g " + "ulps=%lld\n", + static_cast(call), dev_type, static_cast(t), + idstr.c_str(), static_cast(lo_e), + static_cast(vt::BF16ToF32(lo_b)), static_cast(lo_b), + static_cast(hi_e), static_cast(vt::BF16ToF32(hi_b)), + static_cast(hi_b), + static_cast(vt::BF16ToF32(lo_b)) - + static_cast(vt::BF16ToF32(hi_b)), + static_cast(ulps)); + } + // The digest. `x`/`logit`/`exp`/`shr` decompose the block's own output, so + // with the selection sets equal these four say WHICH GEMM carries the residue. + std::fprintf(stderr, + "moesel call=%lld dev=%d T=%lld E=%lld k=%lld sel=%016llx " + "minulps=%lld mintok=%lld x=%.9g logit=%.9g exp=%.9g shr=%.9g " + "lines=%lld END\n", + static_cast(call), dev_type, static_cast(T), + static_cast(E), static_cast(top_k), + static_cast(sel), static_cast(min_ulps), + static_cast(min_tok), SumAbsBf16(h), SumAbsBf16(logits), + SumAbsBf16(expert_out), shared != nullptr ? SumAbsBf16(*shared) : 0.0, + static_cast(MoeSelFpLines())); + ++MoeSelFpCall(); +} + DBuf MoeBlock(Dev d, const MoeBlockWeights& w, const HfConfig& cfg, const Tensor& dh, int64_t T) { const int64_t H = cfg.hidden_size; @@ -7050,6 +7207,19 @@ DBuf MoeBlock(Dev d, const MoeBlockWeights& w, const HfConfig& cfg, std::optional shared; if (has_shared) shared.emplace(SharedExpert(d, w, cfg, dh, T, fp4)); + // MOEDIV (#2552): the selected-expert-id tap. Inert unless VT_MOE_SEL_FP is + // set; the outer guard is what keeps the shared-expert readback off the + // default path. Everything else it reads was already downloaded above. + if (MoeSelFpCalls() > 0) { + std::vector shr; + if (has_shared) { + shr.resize(static_cast(T) * H); + shared->Download(d, shr.data()); + } + MoeSelFp(static_cast(dh.device.type), T, E, top_k, h, logits, ids, + expert_out, has_shared ? &shr : nullptr); + } + // Combine (moe-semantics.md §6): out = shared + sum_j w_j * expert_out_j. DBuf deo(d, DType::kBF16, {T, top_k, H}, expert_out.data()); DBuf dwt(d, DType::kF32, {T, top_k}, weights.data()); diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 6f81eb414..9134dd0ae 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -3659,6 +3659,21 @@ target_include_directories(test_qwen4_exp_forward PRIVATE ${CMAKE_SOURCE_DIR}/sr # the src/ grant. vllm_cpp_add_test(test_qwen4_exp_moe vllm/models/test_qwen4_exp_moe.cpp) target_include_directories(test_qwen4_exp_moe PRIVATE ${CMAKE_SOURCE_DIR}/src) +# MOEDIV (#2552): the selected-expert-id tap. `MoeSelFpCalls()` is another +# read-once static, so the default run above can only ever observe the switch +# UNSET -- where the tap case asserts the tap is INERT. Re-run the SAME binary +# with it set: that is the arm which gates the printed ids, the bf16-ulp margin +# and the set-equality hash against `MoeReference`. +# +# DELIBERATELY UNFILTERED. A `-tc=` pattern that matches nothing exits 0 having +# run nothing, and doctest reports that as `assertions: 0 ... SUCCESS!` -- a +# skip wearing a pass, which this row has already paid for. Running the whole +# binary cannot fail that way, and it buys a second property for free: every +# value case in the suite re-runs with the tap ON, so a tap that perturbed the +# block's own output would red them. +add_test(NAME test_qwen4_exp_moe_sel_fp COMMAND test_qwen4_exp_moe) +set_tests_properties(test_qwen4_exp_moe_sel_fp PROPERTIES + ENVIRONMENT "VT_MOE_SEL_FP=4096") # The five suites that reach src/vllm/support/platform_compat.h, granted per # target rather than globally: the file already carries 123 explicit diff --git a/tests/vllm/models/test_qwen4_exp_moe.cpp b/tests/vllm/models/test_qwen4_exp_moe.cpp index 13ecfc6ff..0625b6600 100644 --- a/tests/vllm/models/test_qwen4_exp_moe.cpp +++ b/tests/vllm/models/test_qwen4_exp_moe.cpp @@ -107,10 +107,19 @@ #include #include #include +#include #include #include #include +// MOEDIV (#2552): the selected-expert-id tap prints on stderr, so this suite +// captures the descriptor around ONE `RunBlock` call. POSIX only; the tap case +// refuses to report on a platform without it rather than passing vacuously. +#if defined(__unix__) || defined(__APPLE__) +#include +#define VLLM_MOE_SEL_FP_CAPTURE 1 +#endif + #include "vllm/model_executor/models/qwen4_exp.h" #include "vllm/model_executor/models/qwen4_exp_weights.h" #include "vt/backend.h" @@ -318,6 +327,10 @@ struct RefOut { std::vector y; // [T, H] std::vector> selected; // [T][top_k], oracle order std::vector margin; // [T] p_k - p_{k+1} + // MOEDIV (#2552): the bf16 BITS of the router logits. The tap's margin is in + // logit space and its unit is bf16 ulps, so gating it needs the bit patterns + // and not the doubles they were rounded from. + std::vector logit_bits; // [T * E] }; RefOut MoeReference(const MoeSource& s, const std::vector& xc) { @@ -325,6 +338,7 @@ RefOut MoeReference(const MoeSource& s, const std::vector& xc) { r.y.assign(static_cast(kT) * kH, 0.0); r.selected.resize(static_cast(kT)); r.margin.assign(static_cast(kT), 0.0); + r.logit_bits.assign(static_cast(kT) * kE, 0); for (int64_t t = 0; t < kT; ++t) { std::vector x(static_cast(kH)); for (int64_t h = 0; h < kH; ++h) @@ -339,6 +353,7 @@ RefOut MoeReference(const MoeSource& s, const std::vector& xc) { a += x[static_cast(h)] * (static_cast(s.router[static_cast(e * kH + h)]) * kWU); logit[static_cast(e)] = Bf16(a); + r.logit_bits[static_cast(t * kE + e)] = vt::F32ToBF16(static_cast(a)); } const double mx = *std::max_element(logit.begin(), logit.end()); std::vector prob(static_cast(kE)); @@ -710,3 +725,207 @@ TEST_CASE("Qwen4Exp MoE: the router and shared gate reach the seam as bf16") { if (vt::BF16ToF32(rg[i]) == static_cast(s.router[i]) * kWU) ++exact; CHECK(exact == s.router.size()); } + +// ─── MOEDIV (#2552): the SELECTED-EXPERT-ID tap ────────────────────────────── +// +// `VT_Q4EXP_LAYER_FP` taps values, and a value tap cannot separate an +// expert-selection FLIP from re-association inside the expert GEMM: a discrete +// selection has BIMODAL error and not a tolerance. `VT_MOE_SEL_FP` prints the +// selection instead. This case is what makes the printed line trustworthy — +// it gates the ids against `MoeReference`, this suite's independent +// double-precision reimplementation of the lane-pinned oracle, and it gates +// `ulps` against the reference's own bf16 logit BITS. +// +// THE CASE NAME CONTAINS NO COMMA. `-tc` splits its filter on commas, so a +// comma in a case name silently selects nothing and the run reports +// `assertions: 0 ... SUCCESS!` at rc 0 — a skip wearing a pass. The ctest entry +// `test_qwen4_exp_moe_sel_fp` re-runs this binary with the switch set and +// filters on this exact string. +// +// IT ASSERTS ITS OWN CAPTURE. A sentinel is written into the redirected +// descriptor and read back, so a capture that silently caught nothing fails +// here rather than reading as "the tap printed nothing", which is the polarity +// the OFF arm asserts and the one a broken instrument would fake. +namespace { + +#ifdef VLLM_MOE_SEL_FP_CAPTURE +// Everything written to stderr while `body` runs, as a string. +template +std::string CaptureStderr(F&& body) { + std::fflush(stderr); + int saved = ::dup(2); + char path[] = "/tmp/vllm_moesel_XXXXXX"; + int fd = ::mkstemp(path); + REQUIRE(saved >= 0); + REQUIRE(fd >= 0); + ::dup2(fd, 2); + body(); + std::fflush(stderr); + ::dup2(saved, 2); + ::close(saved); + ::lseek(fd, 0, SEEK_SET); + std::string out; + char buf[4096]; + ssize_t n = 0; + while ((n = ::read(fd, buf, sizeof(buf))) > 0) out.append(buf, static_cast(n)); + ::close(fd); + ::unlink(path); + return out; +} + +std::vector LinesWithPrefix(const std::string& s, const std::string& pfx) { + std::vector out; + size_t i = 0; + while (i < s.size()) { + const size_t e = s.find('\n', i); + const std::string line = s.substr(i, e == std::string::npos ? std::string::npos : e - i); + if (line.rfind(pfx, 0) == 0) out.push_back(line); + if (e == std::string::npos) break; + i = e + 1; + } + return out; +} + +// `key=` of a `moesel` line, as a string. +std::string Field(const std::string& line, const std::string& key) { + const std::string k = " " + key + "="; + const size_t i = line.find(k); + if (i == std::string::npos) return std::string(); + const size_t b = i + k.size(); + const size_t e = line.find(' ', b); + return line.substr(b, e == std::string::npos ? std::string::npos : e - b); +} + +// bf16 under the sign-magnitude total order — the tap's own `Bf16Ordered`, +// re-derived here rather than shared, so a sign-handling defect in the tap has +// nothing on this side agreeing with it. +int32_t RefBf16Ordered(uint16_t b) { + return (b & 0x8000U) != 0 ? -static_cast(b & 0x7fffU) + : static_cast(b & 0x7fffU); +} +#endif // VLLM_MOE_SEL_FP_CAPTURE + +} // namespace + +TEST_CASE("Qwen4Exp MoE: the selected-expert-id tap names the oracle's own set") { +#ifndef VLLM_MOE_SEL_FP_CAPTURE + MESSAGE("no POSIX stderr capture on this platform; the tap is NOT gated here"); + return; +#else + MoeSource s = BuildSource(/*keep_quant=*/false); + const Qwen4ExpParams p = Params(); + MoeBlockWeights mw = vllm::Qwen4ExpMoeBlockWeights(s.w, p); + const std::vector xc = HiddenCodes(); + const RefOut ref = MoeReference(s, xc); + AssertRoutingIsNonTrivial(ref); + + std::vector got; + const std::string cap = CaptureStderr([&] { + std::fprintf(stderr, "moesel-capture-sentinel\n"); + got = RunBlock(mw, p, xc); + }); + // The capture's OWN precondition. Without this line the OFF arm below is an + // assertion of absence from a redirect that may never have happened. + REQUIRE(cap.find("moesel-capture-sentinel") != std::string::npos); + REQUIRE(got.size() == static_cast(kT) * kH); + + const std::vector lines = LinesWithPrefix(cap, "moesel call="); + const char* budget = std::getenv("VT_MOE_SEL_FP"); + if (budget == nullptr || budget[0] == '\0' || std::string(budget) == "0") { + // A DIAGNOSTIC MUST BE INERT ON THE DEFAULT PATH. The sentinel above proves + // the capture ran, so this is a measurement and not a failed grep. + MESSAGE("VT_MOE_SEL_FP unset: the tap must print nothing"); + CHECK(lines.empty()); + return; + } + + // ON. One value line per token plus one digest line. + REQUIRE(lines.size() == static_cast(kT) + 1); + const std::string& digest = lines.back(); + CHECK(Field(digest, "T") == std::to_string(kT)); + CHECK(Field(digest, "E") == std::to_string(kE)); + CHECK(Field(digest, "k") == std::to_string(kTopK)); + // `lines` is the PROCESS-WIDE running total, and this entry runs the whole + // binary, so other cases have already spent some of it. The exact per-call + // count is asserted above as `lines.size() == kT + 1` over the captured + // window; this asserts the running total is at least what this call added, + // which is the property the measuring job reads back. + CHECK(std::stoll(Field(digest, "lines")) >= static_cast(kT)); + + uint64_t want_sel = 1469598103934665603ULL; + int64_t want_min_ulps = -1; + int64_t want_min_tok = -1; + for (int64_t t = 0; t < kT; ++t) { + const std::string& ln = lines[static_cast(t)]; + CAPTURE(ln); + CHECK(Field(ln, "tok") == std::to_string(t)); + + // (1) THE SET. Sorted, because two arms selecting the same experts in a + // different order have made the same routing decision. + std::vector want = ref.selected[static_cast(t)]; + std::sort(want.begin(), want.end()); + std::string want_ids; + for (size_t j = 0; j < want.size(); ++j) { + if (j != 0) want_ids += ","; + want_ids += std::to_string(want[j]); + const uint32_t v = static_cast(want[j]); + for (int b = 0; b < 4; ++b) { + want_sel ^= static_cast((v >> (8 * b)) & 0xffU); + want_sel *= 1099511628211ULL; + } + } + CHECK(Field(ln, "ids") == want_ids); + + // (2) THE BOUNDARY, from the reference's OWN bf16 logit bits. + int lo_e = -1, hi_e = -1; + for (int64_t e = 0; e < kE; ++e) { + const bool picked = std::find(want.begin(), want.end(), static_cast(e)) != want.end(); + const uint16_t b = ref.logit_bits[static_cast(t * kE + e)]; + const int32_t o = RefBf16Ordered(b); + if (picked) { + if (lo_e < 0 || o < RefBf16Ordered(ref.logit_bits[static_cast(t * kE + lo_e)])) + lo_e = static_cast(e); + } else { + if (hi_e < 0 || o > RefBf16Ordered(ref.logit_bits[static_cast(t * kE + hi_e)])) + hi_e = static_cast(e); + } + } + REQUIRE(lo_e >= 0); + REQUIRE(hi_e >= 0); + const int64_t want_ulps = + static_cast(RefBf16Ordered(ref.logit_bits[static_cast(t * kE + lo_e)])) - + static_cast(RefBf16Ordered(ref.logit_bits[static_cast(t * kE + hi_e)])); + CHECK(Field(ln, "lo_e") == std::to_string(lo_e)); + CHECK(Field(ln, "hi_e") == std::to_string(hi_e)); + CHECK(Field(ln, "ulps") == std::to_string(want_ulps)); + // The margin has to be POSITIVE and the ulp count has to agree with it: a + // selected logit below a rejected one is a routing defect, not a rounding. + CHECK(want_ulps > 0); + if (want_min_ulps < 0 || want_ulps < want_min_ulps) { + want_min_ulps = want_ulps; + want_min_tok = t; + } + } + // (3) The digest hash IS the set-equality assertion two arms use. Recomputed + // here from the oracle's sets, so a tap that hashed the SELECTION ORDER (and + // would therefore report a false flip between two arms that agree) reds. + char want_hex[32]; + std::snprintf(want_hex, sizeof(want_hex), "%016llx", + static_cast(want_sel)); + CHECK(Field(digest, "sel") == std::string(want_hex)); + CHECK(Field(digest, "minulps") == std::to_string(want_min_ulps)); + CHECK(Field(digest, "mintok") == std::to_string(want_min_tok)); + + // (4) The four decomposition axes are POPULATED and finite. `x` is the block + // input and is checkable in closed form; the other three are only asserted + // non-zero, because their value is the measurement and not the contract. + double want_x = 0.0; + for (size_t i = 0; i < xc.size(); ++i) + want_x += std::fabs(static_cast( + vt::BF16ToF32(vt::F32ToBF16(static_cast(xc[i]) * kAU)))); + CHECK(std::stod(Field(digest, "x")) == doctest::Approx(want_x).epsilon(1e-9)); + CHECK(std::stod(Field(digest, "logit")) > 0.0); + CHECK(std::stod(Field(digest, "exp")) > 0.0); + CHECK(std::stod(Field(digest, "shr")) > 0.0); +#endif +}