feat: add gstools.mps — Multiple Point Statistics via Direct Sampling - #416
Draft
n0228a wants to merge 23 commits into
Draft
feat: add gstools.mps — Multiple Point Statistics via Direct Sampling#416n0228a wants to merge 23 commits into
n0228a wants to merge 23 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
gstools.mpssubpackage implementing Multiple Point Statistics (MPS)simulation via the Direct Sampling (DS) algorithm (Mariethoz et al. 2010) with
the DSBC parametrization (Juda et al. 2022). MPS is a fundamentally different
paradigm from two-point geostatistics: instead of a parametric covariance model,
spatial structure is learned directly from a training image (TI), enabling
reproduction of curvilinear, connected patterns — channels, fractures, categorical
facies — that variogram-based methods cannot capture.
The implementation is pure Python + NumPy. Unlike
field/krige/variogram,there is no
gstools_core(Rust) or Cython path formps; the per-node TI scan isvectorised over candidate blocks and parallelised across nodes with threads.
Node-level parallelism is a dependency DAG rather than a barrier — a node is
dispatched as soon as the neighbours it reads are settled — and the same machinery
drives the main path and every post-processing pass, so output is bit-identical for
any
num_threads.Supersedes #409.
Public API
Five classes exported from
gstools.mps.DirectSampling,MPSModel,TrainingImage, andZoneare also available at the top-levelgstoolsnamespace;
Variableis available asgstools.mps.Variable.VariableHolds one variable's data and its per-variable configuration:
data— n-D NumPy arraycategorical— whether to use weighted mismatch distance (Mariethoz2010 Eq. 3)distance— continuous metric:"l1"(Juda2022 Eq. 7, default),"l2","l<p>"(arbitrary Minkowski),"variation"or"variation<p>"(Mariethoz2010Eq. 9, with automatic mean-shift correction)
n_neighbors— data-event size (maximum neighbours, default 32)max_radius— optional Euclidean cap on neighbour selectionweight— relative weight in the multivariate joint distance;Nonefor equalshare; weights are automatically normalized and do not need to sum to 1
penalty_matrix— optional(C, C)per-category mismatch costT[u, v]replacing the flat binary mismatch for categorical variables, so that
"sand mistaken for silt" can cost less than "sand mistaken for shale":
Σ w_i · T[Z(x_i), Z(y_i)] / Σ w_i. Asymmetric matrices are allowed. Validatedto be 2-D square with an all-zero diagonal and off-diagonal entries in
(0, 1],and the variable's data must hold integer-valued category codes
< C— theseconstraints are what keep the distance in
[0, 1](there is nod_maxnormalization on this path)
TrainingImageThe MPS analogue of
CovModel. Three construction modes:Shared option:
distance_power— spatial-decay exponent δ for neighbourweighting (Mariethoz2010 Eq. 5);
0.0(default) gives uniform weights.window(slices)returns a sub-region view of a TI, carrying over every searchparameter and the distance kernel — useful for cutting several zone TIs out of one
large image.
NaN masking — undefined TI cells (NaN) are excluded from distance
comparisons and never pasted into the simulation grid.
MPSModelConfiguration object bundling a
TrainingImagewith the search parameters andevery optional feature:
scan_fraction,threshold,cond_weight,boundaryrotation,scalezonesZoneobjects for zonated simulationpost_processing,post_processing_factor,post_processing_pathGeometric non-stationarity —
rotation=/scale=accept a scalar, a vector,a full per-node array, or a callable evaluated on the grid coordinates, and apply
the lag transform of Mariethoz2010 §6.2. A stationary TI can therefore produce
patterns that rotate and stretch across the grid without modifying the TI itself.
rotationis rejected on 1-D grids (no rotation planes exist);scalestaysmeaningful.
Post-processing (
post_processing=p, Meerschman et al. 2013 §4) runsppasses after the main path, each re-simulating every non-conditioning node against
a fully informed neighbourhood to remove simulation noise. Me13 §7 advises at least
p=1for categorical simulations.post_processing_factor(p_f ≥ 1) dividesscan_fractionandn_neighborsduring the passes to trade fidelity for speed;Me13 finds little effect and recommends
p_f=1. Passes are Gauss–Seidel, but thatordering is carried by the same dependency DAG the main path uses, so they honour
num_threadswhile staying bit-identical to serial execution at any thread count.Each pass re-visits essentially the whole grid, so
p > 0dominates the cost of arun — budget accordingly.
post_processing=0(default) is bit-identical toomitting the parameter and consumes no extra RNG draws.
post_processing_pathcontrols the post-pass visit order and is animplementation extension — Me13 §4 does not prescribe one.
None(default)inherits the main-path mode;
"random","sequential","same"(reuse the mainpass's order), or an explicit
(N, dim)array are accepted.ZoneZone(ti, where)binds a training image to a region of the simulation grid forzonated (non-stationary) simulation, Mariethoz2010 para [40]. Nodes inside the
region scan this zone's TI instead of the primary one, while the data event is
still built from simulation-grid neighbours regardless of their zone — which keeps
zone boundaries coherent instead of cutting the pattern at the seam.
whereis a boolean array of the grid shape, or a callablef(x, [y, z, ...]) -> bool mask. Non-boolean arrays raise; there is no silent!= 0coercion, so an integer zonation raster is writtenZone(ti_k, where=(labels == k)).d_max).Every search hyper-parameter and the distance kernel —
n_neighbors,max_radius,distance,weight,penalty_matrix— always come from theprimary TI, so setting them on a zone TI has no effect.
deterministic and thread-count invariant.
DirectSamplingSubclasses
gstools.field.base.Fieldand follows the same call interface asSRF. Takes anMPSModel.All search parameters (
scan_fraction,threshold,cond_weight,boundary,n_neighbors,max_radius) are read-only onDirectSampling— theyreflect the underlying
MPSModelandVariableobjects.n_neighborsandmax_radiuscollapse to a scalar when all variables share the same value,otherwise return a
{name: value}dict. To change them, mutate theMPSModelor
Variabledirectly.Key capabilities:
"sequential"or an explicit(N, dim)visit-order array via thepathargument
scan_fractioncaps the per-node TI scan (Juda2022 §2)threshold=0.0— DSBC mode: full scan, best-candidate selectionboundary="strict"(default) or"partial"— partial drops lags that exceedthe TI extent and searches with the reduced template (Mariethoz2010 §6.2)
set_condition(cond_pos, cond_val)— hard-data conditioning with nearest-nodesnapping and per-variable collision resolution;
cond_valaccepts a{variable: array}dict for multivariate TIs, withnp.nanto leave avariable unconditioned at a point; conditioning weight is set via
MPSModelset_mv_transforms()— per-variable mean / normalizer / trend post-processing.Note this is the
Fieldpipeline and is unrelated toMPSModel's DSpost_processingpassesnum_threads(orgstools.config.NUM_THREADS)path_seed,node_seed) for independent control of visitorder vs. TI scan randomness
{variable: ndarray}dict; each variable alsostored as a named field accessible via
ds["facies"]/ds.all_fieldsUsage
Multivariate TIs are built from a list of
Variableobjects and return a{variable: ndarray}dict;rotation/scale,zones, andpost_processingareall
MPSModelarguments layered on the same call. See the examples below for each.Module layout
training_image.pyVariable,TrainingImage— data and distance functionsdata_event.pyDataEvent— internal value object bundling per-variable data-event arraysdistance.pymodel.pyMPSModel— validated search-parameter configzone.pyZone— a training image bound to a region of the gridneighbors.pyscan.pyrunner.pynonstationary.pysimulate.py_DirectSamplingEngine+ds_simulateentry pointdirect_sampling.pyDirectSampling— publicFieldsubclassExamples (
examples/13_mps/)examples/13_mps/is now wired intodocs/source/tutorials.rstandsphinx_gallery_conf, so these run at documentation build time. Several fetchtheir training images from public sources at first run and cache them next to the
script.
00_simple_unconditional.py01_conditional.pyset_condition()02_continuous.py03_channel_strebelle.py04_channel_strebelle_nonstat.pyMPSModel(rotation=...)05_channel_strebelle_radial.py06_channel_multivariate_fig8.py07_variation_distance_fig6.py08_bivariate_joint_fig4.py09_continuous_stone_fig3.py10_zonated_facies.pyZone— a separate TI per region, viawindow()views00_mps_overview.ipynbChanges outside
gstools.mpsMPS's per-node rotation/scale non-stationarity (Mariethoz2010 §6.2) needs a
per-axis scale with no pinned axis — including uniform dilation (M10's
affinity
r), which the existinganisconvention (set_anis, axis 0 fixedto ratio 1, every other axis a ratio to it) cannot express.
gstools.toolsgains a second, more general primitive family for this:
set_scale— validates a scalar-or-dim-vector scale, no padding, no pinmatrix_scale,matrix_transform,matrix_detransform— diagonal scalematrix, rotate∘scale, and its exact inverse
great_circle_to_chordaladded totools/geometric.py's module__all__,alongside
chordal_to_great_circle(already present, just not exported)gstools.mps.neighborsusesmatrix_detransformdirectly for the per-nodeSG→TI lag pullback. Separately, the existing
anis-pinned helpers —matrix_isotropify,matrix_anisotropify,matrix_isometrize, andmatrix_anisometrize— are now all expressed as the axis-0-pinned specialcase of the same general primitives, instead of each duplicating its own
diag/matmul construction.
matrix_isometrizeandmatrix_anisometrizeareCovModel's actual call path (covmodel/base.py::isometrize/anisometrize),so this is verified behaviour-preserving two ways:
tests/test_geometric.py::TestWrapperBitIdenticalchecks exact (
assert_array_equal, notallclose) equality against a frozencopy of every pre-refactor implementation over 150 randomized cases across
dims 2–4, and the full
test_covmodel.pysuite passes unmodified.References
to perform multiple-point geostatistical simulations. Water Resources Research.
https://doi.org/10.1029/2008WR007621
Renard, P. (2013). A practical guide to performing multiple-point statistical
simulations with the Direct Sampling algorithm. Computers & Geosciences.
https://doi.org/10.1016/j.cageo.2012.09.019
the Direct Sampling algorithm for multiple-point statistical simulations.
Applied Computing and Geosciences.
https://doi.org/10.1016/j.acags.2022.100091
using multiple-point statistics. Mathematical Geology.