Collection of Python scripts for parsing/analysis of reduced representation sequencing data (e.g. RAD-seq, nextRAD). While many of the scripts are functional, some still need considerable cleaning up and more thorough testing - and this repository therefore very much represents a work in progress.
These scripts all require Python 3, with some requiring additional packages (BioPython, NumPy and SciPy - which can be easily installed using the Miniconda or Anaconda installers, or PyVCF - which can be installed using e.g. pip install PyVCF). Usage information for each script can be obtained using the -h or --help flag (e.g. python3 name_of_script.py -h, or is also listed in this README.
This documentation is dynamically generated using the listed README_compile.py script, extracting purpose, usage and links to example files from the argparse information of each script.
vcf_cluster_explore.py - clone-insensitive exploration of genetic clusters/lineages (UPGMA tree, per-K cluster assignment and a multi-panel differentiation report)
vcf_cluster_explore_all.py - batch-run vcf_cluster_explore.py over every VCF found recursively (auto-detecting a matching .loci), writing default-named outputs beside each VCF
ipyrad_reduce_loci_to_match_vcf.py - reduce an ipyrad .loci file to just the loci present in a .vcf (matched by locus id)
vcf_remap2genome.py - script to remap VCF from de novo RAD assembly back to a reference genome
pyrad_find_caps_markers.py - search PyRAD output file for diagnostic CAPS loci that can distinguish two groups (or one group and all other samples)
vcf_clone_detect.py - script to facilitate identification of clones in dataset
vcf_remap.py - Remaps variants in VCF format to new CHROM and POS as obtained through the
mapping_get_bwa_matches.py scripts. Positions are rough estimates because:
(1) new position is simply an offset of the mapping position + 0-based
position in locus (and e.g. do not take into account reference insertions),
(2) one standard contig length is used to determine pos in reverse mapping
reads (flag 16). [File did not pass PEP8 check]
usage: vcf_remap.py [-h] vcf_file mapping_file locus_length
positional arguments:
vcf_file vcf input file
mapping_file file with mapping results
locus_length length of query loci
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf_missing_data.py - Outputs list of missing data (# and % of SNPs) for each sample in VCF, to identify poor-performing samples to eliminate prior to SNP filtering. Takes vcf_filename as argument. Outputs to STDOUT (no output file). [File did not pass PEP8 check]
usage: vcf_missing_data.py [-h] vcf_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf_rename_loci.py - Renames CHROMS in .vcf file according to list with old/new names, and only
outputs those loci that are listed. [File did not pass PEP8 check]
usage: vcf_rename_loci.py [-h] vcf_file locusnames_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
locusnames_file text file (tsv or csv) with old and new name for each locus
(/CHROM)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, locusnames_file.txt.
vcf_find_clones.py - Script compares the allelic similarity of individuals in a VCF, and outputs
all pairwise comparisons. This can be used to detect potential clones based on
percentage match. Note: highest matches can be assessed in the output file by
using $ sort -rn --key=5 output_file.txt | head -n 50 in the terminal. [File did not pass PEP8 check]
usage: vcf_find_clones.py [-h] vcf_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf_get_chrom_pos_from_number.py - Translates sequential marker numbers back to CHROM/POS from original .vcf
file. Several programs only allow for integers to identify markers, this
script is to restore the original CHROM/POS for markers that were identified. [File did not pass PEP8 check]
usage: vcf_get_chrom_pos_from_number.py [-h] vcf_file markernumbers_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
markernumbers_file text file with SNP numbers that were identified
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, markernumbers_file.txt.
vcf_spider.py - Wrapper for PGDspider on Mac OS to convert .vcf files to various formats.
Note : set PGDSPIDER_PATH constant before use, and make script executable in
terminal with $ chmod +x vcf_spider.py.
usage: vcf_spider.py [-h] vcf_filename pop_filename output_filename
positional arguments:
vcf_filename original vcf file
pop_filename pop filename (.txt)
output_filename output filename (extension used to determine file format
(.genepop, .bayescan, .structure or .arlequin)
optional arguments:
-h, --help show this help message and exit
vcf_cluster_explore.py - Explores the deeper genetic clusters (lineages) in a .vcf dataset in a way
that is NOT biased by the presence of clones. Where vcf_clone_detect.py asks
"which samples are near-identical (clones)", this script asks "what are the
genetic groups, how many of them are there, and how differentiated are they". [File did not pass PEP8 check]
usage: vcf_cluster_explore.py [-h] [-v vcf_file] [-p pop_file]
[-o cluster_file]
[-m {ibs,het-masked,dosage,single-read}]
[--auto-clone] [--clone-list file]
[--clone-threshold pct] [--max-k K]
[--min-cluster-size N] [--k K]
[--tree {upgma,nj}] [--ordination {pca,pcoa}]
[--loci loci_file] [--pops-from-sample-id]
[--fields names] [--split-output K]
[--pdf-output pdf_file] [--no-pdf]
Clones are non-independent samples that distort per-group allele frequencies and
therefore bias essentially every population-genetic measure (Fst, private and
fixed-private alleles, heterozygosity, ordination, and even the clustering
itself). By default the script does NOT remove clones - it runs on all
samples as given (use this when the input is already clone-corrected, or to
inspect the raw structure). Optional CLONE-CORRECTION reduces each clonal genet
to a single representative ramet (the one with the least missing data) and runs
the entire analysis - similarity, tree, K-evaluation and all differentiation
statistics - on that clone-corrected (genet) set. Enable it with `--auto-clone`
(clonal genets detected internally by reusing `vcf_clone_detect.py` at an auto-
inferred threshold), `--clone-threshold PCT` (manual threshold), or
`--clone-list FILE` (external list of samples to drop, e.g. the "samples to
remove" output of `vcf_clone_detect.py`).
The script (1) computes pairwise genetic similarities (`--method`, default
`ibs`; `dosage` is the documented alternative), (2) builds a UPGMA / average-
linkage tree, (3) evaluates K = 2 .. `--max-k`, reporting for each K the
genetic-similarity cut-off that splits the tree into K groups, the merge-height
separation gap, and the silhouette width, and picks the best K by silhouette
(robust to between-cluster GD overlap, unlike the raw gap), (4) assigns every
sample to a cluster at each K, and (5) produces a two-page PDF. PAGE 1 is a
full-page-width tree with readable tip labels, the per-K cluster-assignment
columns, a % genotyped bar and - when a popfile (or `--pops-from-sample-id`) is
given - one dot-strip panel per metadata track (population from column 2 and,
if present, a lineage/species/region from column 3; or the 2nd/3rd/4th `_`-
delimited fields of the sample name) so each sample's category memberships line
up with its tip; the selected-K column is boxed, and tracks can be renamed with
`--fields`. PAGE 2 is a fixed A4 page (for supplementary docs), a row-based
grid: (row 1) a fanned circular tree with cluster-coloured branches/tips, then
ordination axes 1-vs-2 and 2-vs-3 with hulls; (row 2) the metric-vs-K support
curve and per-sample silhouette; (row 3, with tracks) one stacked bar per field
showing how the clusters distribute across that field's categories; (row 4)
white-yellow-red differentiation heatmaps - shared loci (or shared genotyped
SNPs without a `.loci` file; diagonal = each cluster's own total), pairwise
private alleles excluding singletons, and alternatively fixed SNPs (all counted
over sites with >= 2 samples per cluster).
The UPGMA linkage is the single source of truth: it is drawn as the tree AND
cut to give every K-assignment, so the tree and the columns are always coherent.
With `--tree nj` a neighbour-joining tree (as in `vcf_clone_detect.py`) is drawn
for display instead, but the cluster assignments still come from UPGMA.
Every PDF page is labelled with the VCF basename (top-left). With `--split-output
K` the dataset is split into the K clusters at that K and the whole analysis is
re-run separately for each (writing `_C<n>` csv/pdf outputs), to explore the
sub-structure within each cluster.
Example:
python3 vcf_cluster_explore.py --vcf vcf_file.vcf --pop pop_file.txt --output clusters.csv
options:
-h, --help show this help message and exit
-v vcf_file, --vcf vcf_file
input file with SNP data (`.vcf`)
-p pop_file, --pop pop_file
optional text file (tsv/csv): col 1 = sample, col 2 =
population, optional col 3 = lineage/species/region.
Drawn as dot-strip annotation tracks beside the tree
and cross-tabulated against clusters (not used for
clustering itself)
-o cluster_file, --output cluster_file
output file (csv) for per-sample cluster assignments
across K (default: derived from vcf)
-m {ibs,het-masked,dosage,single-read}, --method {ibs,het-masked,dosage,single-read}
similarity measure (default: ibs; dosage is the
documented alternative)
--auto-clone clone-correct using an auto-inferred clone threshold
(default: clones are NOT removed)
--clone-list file external list of samples to drop for clone-correction
(e.g. vcf_clone_detect "to remove" output)
--clone-threshold pct
clone-correct using this manual similarity % threshold
above which samples are clones
--max-k K maximum K to evaluate (default: 10)
--min-cluster-size N clusters smaller than this exclude a K from being
chosen as best (default: 2)
--k K force which K drives the differentiation panels
(default: best-supported K)
--tree {upgma,nj} tree to draw (default: upgma; nj is display only,
clusters still come from upgma)
--ordination {pca,pcoa}
ordination for the scatter panel (default: pca)
--loci loci_file optional ipyrad `.loci` file; adds a shared/unique
loci panel (private loci per cluster + pairwise
Jaccard of recovered loci)
--pops-from-sample-id
derive up to three annotation tracks (drawn beside the
tree) from each sample name, splitting on "_" and
using the 2nd, 3rd and 4th fields (the 1st is
skipped); used instead of a popfile. A field with more
than 20 distinct values (e.g. a per-sample id) is
dropped
--fields names comma-separated names for the annotation tracks in
order (e.g. "location,depth"); overrides the default
track titles
--split-output K after the full analysis, split the dataset into the K
clusters at this K and re-run the whole analysis
separately for each (writing `_C<n>` csv/pdf outputs
alongside the main ones; needs K>=2)
--pdf-output pdf_file
filename for the PDF report
--no-pdf do not generate the PDF report (text only)
vcf_cluster_explore_all.py - Batch-run vcf_cluster_explore.py over every .vcf found recursively under a
directory. For each VCF a sibling .loci file with the same basename is used
automatically when present, and the CSV + PDF outputs are written with their
default names into the folder that contains the VCF (so each dataset's results
sit next to its input). [File did not pass PEP8 check]
usage: vcf_cluster_explore_all.py [-h] [--postfix str] [--pattern glob]
[--script path] [--dry-run]
[-m {ibs,het-masked,dosage,single-read}]
[--auto-clone] [--clone-threshold pct]
[--max-k K] [--min-cluster-size N] [--k K]
[--tree {upgma,nj}]
[--ordination {pca,pcoa}]
[--pops-from-sample-id] [--fields names]
[--no-pdf]
[root]
Any `vcf_cluster_explore.py` option that is not file-specific is exposed here and
applied to every dataset (e.g. `--pops-from-sample-id`, `--fields`, `--method`,
`--max-k`, `--tree`, `--ordination`, `--auto-clone`, `--no-pdf`). The per-file
options (`--vcf`, `--loci`, `--output`, `--pop`, `--pdf-output`) are handled
per-VCF by this wrapper and are intentionally not forwarded.
Use `--postfix` to target a specific set of VCFs by basename suffix (e.g.
`--postfix _filtered` matches `*_filtered.vcf`), or `--pattern` for a full glob.
Each VCF is run in its own subprocess, so one failing dataset does not stop the
batch; a summary of successes/failures is printed at the end.
Example:
python3 vcf_cluster_explore_all.py datasets/ --postfix _filtered --pops-from-sample-id --fields species,location,depth
positional arguments:
root directory to search recursively for VCFs (default:
current directory)
options:
-h, --help show this help message and exit
--postfix str only use VCFs whose basename ends with this before
`.vcf` (e.g. --postfix _filtered matches
*_filtered.vcf); default: all *.vcf
--pattern glob full filename glob for VCFs (overrides --postfix;
default: *<postfix>.vcf)
--script path path to vcf_cluster_explore.py (default: the copy
alongside this wrapper)
--dry-run list the VCFs and the commands without running them
-m {ibs,het-masked,dosage,single-read}, --method {ibs,het-masked,dosage,single-read}
similarity measure (default: ibs)
--auto-clone clone-correct using an auto-inferred threshold
--clone-threshold pct
clone-correct using this manual similarity % threshold
--max-k K maximum K to evaluate (default: 10)
--min-cluster-size N clusters smaller than this cannot be chosen as best K
(default: 2)
--k K force which K drives the differentiation panels
(default: best-supported K)
--tree {upgma,nj} tree to draw (default: upgma)
--ordination {pca,pcoa}
ordination for the scatter panels (default: pca)
--pops-from-sample-id
derive annotation tracks from the sample name fields
(see vcf_cluster_explore.py)
--fields names comma-separated names for the annotation tracks, e.g.
"species,location,depth"
--no-pdf do not generate the PDF reports (text only)
ipyrad_reduce_loci_to_match_vcf.py - Reduce an ipyrad .loci file to just the loci that are present in a .vcf
(i.e. the loci that still contributed at least one SNP to the VCF, e.g. after
filtering). Each locus block in the .loci file is kept or dropped as a whole,
so the output is a valid .loci file containing only the retained loci. [File did not pass PEP8 check]
usage: ipyrad_reduce_loci_to_match_vcf.py [-h] -v vcf_file -l loci_file
[-o loci_file]
Loci are matched by their integer locus id. ipyrad writes that id into the VCF
`ID` column as `loc<N>_pos<M>` (and into each `.loci` separator line as
`// ... |<N>...|`). When the VCF has no `loc<N>` ids (e.g. a de novo VCF whose
`CHROM` is the locus number) the `CHROM` column is used as the fallback key.
By default the reduced file is named after the VCF with a `.loci` extension (so
`--vcf FAVIINAE_filtered.vcf` writes `FAVIINAE_filtered.loci`, ready to pair with
that VCF); if that name already exists it falls back to `<vcf>_invcf.loci`.
Example:
python3 ipyrad_reduce_loci_to_match_vcf.py --vcf FAVIINAE_filtered.vcf --loci FAVIINAE.loci
options:
-h, --help show this help message and exit
-v vcf_file, --vcf vcf_file
input `.vcf` file (defines which loci to keep)
-l loci_file, --loci loci_file
input ipyrad `.loci` file to reduce
-o loci_file, --output loci_file
output `.loci` file (default: the vcf name with a
`.loci` extension, e.g. FAVIINAE_filtered.vcf ->
FAVIINAE_filtered.loci; falls back to
`<vcf>_invcf.loci` if that already exists)
vcf_clone_detect.py - Identify groups of clones (near-identical samples) in a .vcf dataset. The
script:
(1) computes a pairwise genetic similarity between every pair of individuals,
(2) prints a text histogram of those similarities,
(3) lists the highest-similarity pairs and, unless --threshold is given,
infers a candidate clonal threshold (see "Threshold inference" below),
(4) clusters individuals into clonal groups at the threshold,
(5) lists, per group, the members to remove so that the member with the most
genotyped loci is retained, and
(6) writes an A4 PDF report: a neighbour-joining tree with clonal groups
coloured and a per-sample % genotyped bar panel, plus similarity
histograms (full, and zoomed to the threshold). [File did not pass PEP8 check]
usage: vcf_clone_detect.py [-h] [-v vcf_file] [-p pop_file] [-i compare_file]
[-o compare_file] [-t threshold]
[-m {ibs,het-masked,dosage,single-read}] [-k K]
[--no-lineage] [--pdf-output pdf_file] [--no-pdf]
Usage is two-step. First compute and save the pairwise comparisons from a VCF:
vcf_clone_detect.py --vcf data.vcf --pop pops.txt --output compare.csv
Then re-threshold without recomputing by reading that file back:
vcf_clone_detect.py --input compare.csv --threshold 94.5
A popfile (sample, population per line) sets the matrix/tree order and splits
the histograms into within- vs between-population comparisons.
Similarity measures (--method)
------------------------------
Each measure is computed only over sites genotyped in both individuals of a
pair, and reported as a 0-100 % similarity (100 = identical). The tree distance
is 1 - similarity/100. da, db are alt-allele dosages (0, 1, 2).
ibs (default) Mean allele-sharing per site: score 1 if the two
genotypes are identical, 0.5 if they share one allele, 0 if they
share none (a heterozygote pair scores 1). Equals PLINK
`--distance 1-ibs flat-missing` (and the default
`--distance`/`allele-ct`, rescaled).
het-masked As ibs but using only sites where BOTH individuals are
homozygous; the per-site score is 1 if the genotypes are
identical, else 0. No PLINK/ANGSD equivalent.
dosage Mean squared dosage difference per site, 1 - (da-db)^2 / dmax^2
(dmax = maximum dosage, 2 for biallelic data). Squared-Euclidean
distance on dosage; equals PLINK `--make-rel cov` via
sum((da-db)^2) = M*(Cjj+Ckk-2*Cjk). Not the standardised GRM.
single-read Mean of p1 + p2 - 2*p1*p2 per site, where p is the alt-allele
fraction: alt/(ref+alt) from the AD field when present, otherwise
the genotype dosage/2. This is the expected mismatch when one
allele is drawn at random from each individual; it emulates ANGSD
`-doIBS 1 -makeMatrix 1`. With called genotypes a heterozygote
pair has an expected per-site mismatch of 0.5. (ANGSD itself
needs read-level data; this approximates it from a VCF.)
With --method ibs the result matches the previous version of this script
exactly. PLINK/ANGSD equivalences are exact only on identical SNP sets: this
script uses pairwise-complete sites while PLINK/ANGSD apply their own
missing-data scaling, so values can differ slightly when data are missing.
Threshold inference (when --threshold is not given)
---------------------------------------------------
Pairs are sorted by descending similarity. Considering only pairs at or above
85 %, the largest drop in similarity between two consecutive pairs is taken as
the clone/non-clone break. The threshold is the integer part of the similarity
just above that drop if that integer lies within the drop, otherwise the
midpoint of the drop. This heuristic assumes clones form a tight cluster at the
top of the distribution and is calibrated to the ibs %-scale; for the other
measures, or whenever the inferred value looks wrong, set --threshold.
Multiple cryptic lineages
-------------------------
When a dataset contains several genetic lineages, a single clonal threshold is
unreliable: each lineage has a different similarity distribution (different
informative-SNP counts and per-pair denominators), so clones are best called
within each lineage on its own scale. Two ways to do this:
- Supply a 3-column popfile (sample, population, lineage): clone detection
runs independently within each lineage, each with its own histogram and
inferred/supplied threshold.
- Or give `-k/--n-lineages K`: the dataset is split into K lineages by
average-linkage clustering of this script's own distance matrix (which is
clone-robust, unlike STRUCTURE/SNAPCLUST), the inferred 3-column popfile is
written to `<base>_lineages.txt` for review, and detection then runs per
lineage.
Each lineage gets its own comparison CSV and PDF; a combined list of all
samples to remove across the dataset is written to `<base>_clones_remove.txt`.
`--no-lineage` forces a single global run. Only within-lineage pairs are ever
eligible to be clones. The comparison CSV is always written (default name from
the input basename if `--output` is omitted).
Output columns (--output / --input CSV)
---------------------------------------
ind1, ind2, ind1_snps, ind2_snps, both_snps, match, match_perc, pop.
ind1_snps/ind2_snps: each sample's genotyped-loci count. both_snps: number of
sites the chosen measure used for the pair (genotyped in both; for het-masked,
homozygous in both; for AD-based single-read, with reads in both). match: summed
per-site similarity score over those sites; match_perc = round(100 * match /
both_snps, 2). pop: the shared population, 'popA-popB' for a between-population
pair, or 'NA'.
options:
-h, --help show this help message and exit
-v vcf_file, --vcf vcf_file
input VCF file with SNP data
-p pop_file, --pop pop_file
population file (sample, population[, lineage] per
line); a 3rd lineage column enables per-lineage
detection
-i compare_file, --input compare_file
read pairwise comparisons from a prior --output CSV
instead of a VCF
-o compare_file, --output compare_file
comparisons CSV (default: derived from input basename;
always written)
-t threshold, --threshold threshold
minimum % similarity to call two samples clones
(default: inferred per set)
-m {ibs,het-masked,dosage,single-read}, --method {ibs,het-masked,dosage,single-read}
similarity measure (default: ibs); see above
-k K, --n-lineages K delimit K lineages from the distance matrix and run
clone detection within each (writes an inferred 3-col
popfile); ignored if --pop already has a lineage
column
--no-lineage force a single global run even with a 3-col popfile or
--n-lineages
--pdf-output pdf_file
PDF report filename (default: from input basename;
per-lineage names are auto-derived)
--no-pdf skip the PDF report (text output only)
vcf_minrep_filter_abs.py - Filters .vcf file for SNPs that are genotyped for a minimum number of
individuals in each of the populations (rather than overall proportion of
individuals). This can help to guarantee a minimum number of individuals to
calculate population-based statistics, and eliminate loci that might be
suffering from locus drop-out in particular populations. Note: only
individuals that are listed in popfile are taken into account to determine
number of individuals genotyped (but all indivs are outputted). [File did not pass PEP8 check]
usage: vcf_minrep_filter_abs.py [-h]
vcf_file pop_file min_proportion
output_filename
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file text file (tsv or csv) with individuals and populations
min_proportion proportion of individuals required to be genotyped in each
population for a SNP to be included (e.g `0.8` for 80
percent of individuals)
output_filename name of output file (`.vcf`)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, pop_file.txt.
vcf_minrep_filter.py - Filters .vcf file for SNPs that are genotyped for a minimum proportion of
individuals in each of the populations (rather than overall proportion of
individuals). This can help to guarantee a minimum number of individuals to
calculate population-based statistics, and eliminate loci that might be
suffering from locus drop-out in particular populations. Note: only
individuals that are listed in popfile are taken into account to determine
proportion of individuals genotyped (but all indivs are outputted). [File did not pass PEP8 check]
usage: vcf_minrep_filter.py [-h]
vcf_file pop_file min_proportion output_filename
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file text file (tsv or csv) with individuals and populations
min_proportion proportion of individuals required to be genotyped in each
population for a SNP to be included (e.g `0.8` for 80
percent of individuals)
output_filename name of output file (`.vcf`)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, pop_file.txt.
vcf_remove_chrom.py - Excludes those loci (/CHROMs) in .vcf that are listed in exclusion list.
Also outputs a logfile with loci that were listed but not present in .vcf. [File did not pass PEP8 check]
usage: vcf_remove_chrom.py [-h] vcf_file exclusion_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
exclusion_file text file loci to be excluded
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, exclusion_file.txt.
vcf_remap2genome.py - Remap VCF to genome. * Currently only works with single-line FASTA files (but easy to change) * Works best when using the optional alignment output in sam2tsv [File did not pass PEP8 check]
usage: vcf_remap2genome.py [-h] [-v vcf_file] [-f fasta_file] [-t samtsv_file]
[-o output_vcf_file] [-pid pid_threshold]
optional arguments:
-h, --help show this help message and exit
-v vcf_file, --vcf vcf_file
original vcf file, where the CHROM correspond to the
sequence name in the supplied fasta (perfect match),
and the POS the position within that sequence (also
counting gaps if present).
-f fasta_file, --fasta fasta_file
fasta file representing a single sequence for each
locus/CHROM in the original vcf (including gaps if
present in the original aligment that the fasta is
based on). Note that these gaps need to be removed
before mapping these sequences back to the genome with
bwa mem, but need to be present in this file.
-t samtsv_file, --t samtsv_file
The sam file converted to tsv. The sam file represents
the mapping outcome of the supplied fasta (but then
without gaps) to the genome with bwa mem, e.g.
through: bwa mem ref_genome fasta_file_no_gaps.fa >
samfile.sam. This samfile then needs to be converted
to a tsv, using sam2tsv from the jVarKit toolkit
(http://lindenb.github.io/jvarkit/Sam2Tsv.html).
-o output_vcf_file, --output_vcf output_vcf_file
remapped vcf file with genome scaffold/chroms and
positions within those scaffold/chroms.
-pid pid_threshold, --pid pid_threshold
optional pid alignment threshold, to exclude loci
aligning to the genome with a percent id (PID) score
below the indicated value.
vcf_append_simulated_crosses.py - Generates artificial crosses between individuals from two indicated (in a
popfile) parentalgroups, and appends crossed individuals to .vcf file. Note:
individual SNPs on a single CHROM are independently crossed as if they are not
physically linked - therefore only use when subsampling a single SNP / CHROM. [File did not pass PEP8 check]
usage: vcf_append_simulated_crosses.py [-h] [--n_crosses n_crosses]
[--prefix prefix] [--parentalnames]
vcf_file pop_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file text file (tsv or csv) with the names of the
individuals used for the simulated crosses, and in the
second column which parental population they belong to
(any name can be chosen - as long as there are exactly
two distinct values)
optional arguments:
-h, --help show this help message and exit
--n_crosses n_crosses, -n n_crosses
number of crosses to simulate (should be no higher
than the number of individuals in each of the two
parental populations)
--prefix prefix prefix for crosses (used only if --parentalnames is
not set)
--parentalnames set flag to use names of both parents for cross
Example input file(s): vcf_file.vcf, pop_file.txt.
vcf2hapmatrix.py - Converts .vcf file to Tag Haplotype Matrix (with Chrom), with order of
individuals as indicated in optional file. Note: not yet properly tested. SNPs
of same CHROM (first column) in .vcf should be grouped
together/sequentially, and all individuals need to be listed in order_file. [File did not pass PEP8 check]
usage: vcf2hapmatrix.py [-h] [-o order_file] vcf_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
optional arguments:
-h, --help show this help message and exit
-o order_file, --order_file order_file
text file with preferred output order of individuals
Example input file(s): vcf_file.vcf.
vcf_genotype_freqs.py - Outputs genotype frequencies for specific SNPs in each population, organised by group. [File did not pass PEP8 check]
usage: vcf_genotype_freqs.py [-h] vcf_file factor_file SNP_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
factor_file text file (tsv or csv) with individuals, their population
assignment and group assignment
SNP_file text file (tsv or csv) with CHROM/POS of each SNP to be
outputted
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, factor_file.txt.
popfile_match_vcf.py - Cleans up popfile by eliminating any individuals that are not in .vcf file. [File did not pass PEP8 check]
usage: popfile_match_vcf.py [-h] vcf_file pop_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file text file (tsv or csv) with individuals and populations
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, pop_file.txt.
vcf2introgress.py - Converts .vcf file to INTROGRESS input files (4 files). Splits data into
three categories: parental1, parental2 and admixed based on cluster assignment
(provided in separate file; e.g. STRUCTURE output) and given threshold, and
outputs data for loc that exceed a certain frequency difference between the
two 'parental' categories. Note: not yet properly tested. also see similar
vcf_ancestry_matrix.py script. I use the formatted CLUMPP output
(clumpp_K2.out.csv) from the structure_mp wrapper as assignment file (max.
of 2 clusters). [File did not pass PEP8 check]
usage: vcf2introgress.py [-h] [--include]
vcf_file assignment_file assign_cut_off freq_cut_off
output_prefix
positional arguments:
vcf_file input file with SNP data (`.vcf`)
assignment_file text file (tsv or csv) with assignment values for each
individual (max. 2 clusters); e.g. a reformatted STRUCTURE
output file
assign_cut_off min. assignment value for an individual to be included in
the allele frequency calculation (i.e. putative purebred)
freq_cut_off min. allele frequency difference between the 2 clusters for
a locus to be included in the output
output_prefix prefix for output files
optional arguments:
-h, --help show this help message and exit
--include, -i set this flag if parental pops need to be included in
output
Example input file(s): vcf_file.vcf, assignment_file.csv.
vcf_pos_count.py - Counts SNP occurrence frequency for each POS in .vcf file. [File did not pass PEP8 check]
usage: vcf_pos_count.py [-h] vcf_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf_reference_loci.py - Lists all loci (using CHROM column) in .vcf that are genotyped for at least
one of the indicated samples/individuals. This can be used to reduce the
dataset to loci matching an included reference (e.g. aposymbiotic) samples.
Note: vcf can subsequently be filtered by using the output as inclusion_file
for vcf_include_chrom.py. [File did not pass PEP8 check]
usage: vcf_reference_loci.py [-h]
vcf_file
[reference_samples [reference_samples ...]]
positional arguments:
vcf_file input file with SNP data (`.vcf`)
reference_samples sample(s) against which the remainder of the dataset will
be compared
optional arguments:
-h, --help show this help message and exit
vcf_contrast_samples.py - Contrast all samples in .vcf file against certain reference sample(s) (e.g.
outgroup samples), to assess for fixed / private alleles. [File did not pass PEP8 check]
usage: vcf_contrast_samples.py [-h]
vcf_file
[reference_samples [reference_samples ...]]
positional arguments:
vcf_file input file with SNP data (`.vcf`)
reference_samples sample(s) against which the remainder of the dataset will
be compared
optional arguments:
-h, --help show this help message and exit
vcf_gdmatrix.py - Calculates Genetic Distance (Hamming / p-distance) for each pair of
individuals in a .vcf file and outputs as matrix. Popfile is supplied to
indicate order in matrix. [File did not pass PEP8 check]
usage: vcf_gdmatrix.py [-h] vcf_file pop_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file text file (tsv or csv) with individuals and populations
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, pop_file.txt.
vcf_single_snp.py - Reduces .vcf file to a single 'random' SNP per locus/chrom. Use for analyses
that require SNPs that are not physically linked. (although note that they of
course still may be - particularly so when dealing with short loci) [File did not pass PEP8 check]
usage: vcf_single_snp.py [-h] [-d distance_threshold] vcf_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
optional arguments:
-h, --help show this help message and exit
-d distance_threshold, --distance distance_threshold
optional custom distance threshold between SNPs
(default is 2500; not relevant for short de-novo loci
not mapped to reference scaffolds)
Example input file(s): vcf_file.vcf.
vcf_splitfst.py - Filter original SNP dataset (.vcf) for a particular Fst percentile bin.
Note: order in Fst file needs to correspond with (.vcf) file, currently set
(see script CONSTANTS) to work with LOSITAN output file, and output filename
automatically generated from percentile bins. [File did not pass PEP8 check]
usage: vcf_splitfst.py [-h] vcf_file fst_file min_percentile max_percentile
positional arguments:
vcf_file input file with SNP data (`.vcf`)
fst_file text file (tsv or csv) with Fst values for each SNP (same
order as in vcf) - currently set to work with LOSITAN output
file
min_percentile min. Fst value for a SNP to be included
max_percentile max. Fst value for a SNP to be included
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, fst_file.txt.
vcf_ragoo_order.py - Adds ordering indexes to a file with CHROM and POS columns, based on RaGOO mapping to a chromosome-level assembly. These ordering indexes (chromosome, scaffold and SNP order) can be used for Manhattan-style plots, without having to remap coordinates of the vcf. CHROM and POS columns should be the first and second column in the input file. In the output, three extra columns are inserted (after the CHROM and POS columns) that correspond to the chromosome ID, scaffold order on that chromosome, and position order within the scaffold (based on the RaGOO mapping orientation). [File did not pass PEP8 check]
usage: vcf_ragoo_order.py [-h] filename ragoo_orderings_path
positional arguments:
filename input file
ragoo_orderings_path path with ragoo orderings files
optional arguments:
-h, --help show this help message and exit
popfile_from_vcf.py - Creates tab-separated popfile from .vcf, using a subset of the sample name
as population. For example, to use the substring MGD from AFMGD6804H as
population designation, run script as python3 popfile_from_vcf vcf_file 3 5. [File did not pass PEP8 check]
usage: popfile_from_vcf.py [-h] vcf_file start_pos end_pos
positional arguments:
vcf_file input file with SNP data (`.vcf`)
start_pos character start position in sample name to be used for
population name (one-based)
end_pos character end position in sample name to be used for population
name (one-based)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf_read_trim.py - Removes SNPs from .vcf that are above a certain POS value. [File did not pass PEP8 check]
usage: vcf_read_trim.py [-h] vcf_file highest_pos
positional arguments:
vcf_file input file with SNP data (`.vcf`)
highest_pos max. POS value allowed in `.vcf`
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf_ancestry_matrix.py - Creates a genotype matrix for loci that have a large allele frequency
difference between two genetic clusters (as identified with e.g. STRUCTURE).
The script takes both a .vcf file and a text file with the assignment
probabilities as input. An assignment threshold (e.g. 0.98) needs to be
supplied to identify the reference individuals in the two clusters, and an
allele frequency cut-off needs to be supplied to identify divergent loci. An
optional file can be supplied with a list of loci that need to be included
regardless (e.g. previously identified outliers). Note: I use the formatted
CLUMPP output (clumpp_K2.out.csv) from the structure_mp wrapper as
assignment file (max. of 2 clusters). [File did not pass PEP8 check]
usage: vcf_ancestry_matrix.py [-h] [--include inclusion_file]
vcf_file assignment_file assign_cut_off
freq_cut_off
positional arguments:
vcf_file input file with SNP data (`.vcf`)
assignment_file text file (tsv or csv) with assignment values for each
individual (max. 2 clusters); e.g. a reformatted
STRUCTURE output file
assign_cut_off min. assignment value for an individual to be included
in the allele frequency calculation (i.e. putative
purebred
freq_cut_off min. allele frequency difference between the 2
clusters for a locus to be included in the output
optional arguments:
-h, --help show this help message and exit
--include inclusion_file, -i inclusion_file
text file with loci to be included in output
regardless of allele frequency differences
Example input file(s): vcf_file.vcf, assignment_file.csv.
vcf_include_chrom.py - Retains only those loci (/CHROMs) in .vcf that are given in file. [File did not pass PEP8 check]
usage: vcf_include_chrom.py [-h] vcf_file inclusion_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
inclusion_file text file with loci to be retained
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, inclusion_file.txt.
vcf_afd_filter.py - Calculate allele frequency differentials between groups, and flag those loci that have AFDs exceeding threshold between all subgroups of those groups. Note: so far only used with 3 groups - yet to be tested for more. [File did not pass PEP8 check]
usage: vcf_afd_filter.py [-h] vcf_file group_file afd_threshold
positional arguments:
vcf_file input file with SNP data (`.vcf`)
group_file text file (tsv or csv) separating individuals (first column)
into groups (second column)
afd_threshold allele frequency differential threshold
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf.
vcf2tess.py - Converts .vcf file to TESS input files (genotypes and coordinates). Requires
a popfile and a file with coordinates for each population (decimal ]degrees),
a then simulates individual coordinates by adding a certain amount of noise.
Note: outputs individuals in the same order as popfile. [File did not pass PEP8 check]
usage: vcf2tess.py [-h] [--noise noise]
vcf_file pop_file coord_file output_prefix
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file text file (tsv or csv) with individuals and
populations
coord_file text file (tsv or csv) with populations and their lats
and longs (in decimal degrees)
output_prefix name prefix for output files
optional arguments:
-h, --help show this help message and exit
--noise noise, -n noise
max. amount of noise to be added (default = 1e-10)
Example input file(s): vcf_file.vcf, pop_file.txt.
vcf_rename_samples.py - Renames sample in .vcf file according to list with old/new names; also
outputs samples that are not listed in name_change file. [File did not pass PEP8 check]
usage: vcf_rename_samples.py [-h] vcf_file samplenames_file
positional arguments:
vcf_file input file with SNP data (`.vcf`)
samplenames_file text file (tsv or csv) with old and new name for each
sample (not all samples have to be listed)
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, samplenames_file.txt.
pyradclust2fasta.py - Creates one large FASTA from all PyRAD clustS files in directory. Only outputs clusters that exceed size threshold (min. number of sequences in cluster). First sequence of each cluster is outputted (together with size of overall cluster - note: not of that specific sequence). Prints the outputted and total number of clusters to STDOUT. [File did not pass PEP8 check]
usage: pyradclust2fasta.py [-h] path cluster_threshold output_file
positional arguments:
path path that contains PyRAD `.clustS` files
cluster_threshold minimum size of cluster to be included
output_file name of output FASTA file
optional arguments:
-h, --help show this help message and exit
pyrad_find_caps_markers.py - Search PyRAD output file for diagnostic CAPS loci that can distinguish two groups (or one group and all other samples). [File did not pass PEP8 check]
usage: pyrad_find_caps_markers.py [-h] [-i pyrad_file] [-g group_file]
[-r re_site_file] [-g1 group1] [-g2 group2]
[-m min_samples] [-o output_folder]
optional arguments:
-h, --help show this help message and exit
-i pyrad_file, --input pyrad_file
input pyrad .loci or .alleles file
-g group_file, --groups group_file
text file (tsv or csv) separating individuals (first
column) into groups (second column)
-r re_site_file, --re re_site_file
text file (tsv or csv) listing restriction site names
(first column) and their recognition sequences (second
column)
-g1 group1, --group1 group1
first group that is targeted in marker search
-g2 group2, --group2 group2
second group (optional) that is targeted in marker
search (if none given, group1 is contrasted against
all other samples in group_file
-m min_samples, --min_samples min_samples
minimum number of genotyped samples in each group for
a marker to be considered
-o output_folder, --output output_folder
name of output folder for individual seqs of each
diagnostic locus
pyrad_shared_loci.py - Calculates the mean/min/max of shared loci for each sample [File did not pass PEP8 check]
usage: pyrad_shared_loci.py [-h] loci_file
positional arguments:
loci_file (i)pyrad .loci file
optional arguments:
-h, --help show this help message and exit
Example input file(s): loci_file.txt.
pyrad2fasta.py - Create FASTA file with a representative sequence (using first sample) or all
sequences (when --all_seqs flag is set) for each locus in pyRAD/ipyrad .loci
or .allele file. [File did not pass PEP8 check]
usage: pyrad2fasta.py [-h] [-a] pyrad_file
positional arguments:
pyrad_file PyRAD allele file (`.loci` or `.allele`)
optional arguments:
-h, --help show this help message and exit
-a, --all_seqs set flag to output all sequences
Example input file(s): pyrad_file.loci.
pyrad2concat_fasta.py - Concatenates PyRAD/ipyrad sequences from .loci file for each individual and
outputs as FASTA (order by popfile). Note: missing data are filled with gaps
(N) [File did not pass PEP8 check]
usage: pyrad2concat_fasta.py [-h] pyrad_file pop_file
positional arguments:
pyrad_file PyRAD file (`.loci`)
pop_file text file (tsv or csv) with individuals and populations
optional arguments:
-h, --help show this help message and exit
Example input file(s): pyrad_file.loci, pop_file.txt.
pyrad_filter.py - Filters PyRAD output file (.loci) for those loci (1) present or absent
(using --exclude flag) in supplied list, (2) genotyped for at least X number
of samples, and (3) with at least Y number of informative sites. Note: can
also be used for .alleles file but then 2x the number of samples should be
given (assuming diploid individual). [File did not pass PEP8 check]
usage: pyrad_filter.py [-h] [-e]
pyrad_file loci_file sample_threshold snp_threshold
positional arguments:
pyrad_file PyRAD file (`.loci`)
loci_file text file with PyRAD loci to be included
sample_threshold min. number of samples genotyped for a locus to be
included
snp_threshold min. number of SNPs for a locus to be included
optional arguments:
-h, --help show this help message and exit
-e, --exclude use the loci in loci_file as exclusion list
Example input file(s): pyrad_file.loci, loci_file.txt.
pyrad_include.py - Reduces (i)pyrad file to only those samples listed in supplied text file. [File did not pass PEP8 check]
usage: pyrad_include.py [-h] loci_file inclusion_file min_samples
positional arguments:
loci_file samples input file
inclusion_file text file with names of samples to be included
min_samples minimum number of samples for locus to be included
optional arguments:
-h, --help show this help message and exit
Example input file(s): loci_file.txt, inclusion_file.txt.
pyrad_trim.py - Trims sequence length in PyRAD/ipyrad .alleles or .loci file. [File did not pass PEP8 check]
usage: pyrad_trim.py [-h] pyrad_file seq_length
positional arguments:
pyrad_file PyRAD allele file (`.loci` or `.allele`)
seq_length length to which sequences are trimmed
optional arguments:
-h, --help show this help message and exit
Example input file(s): pyrad_file.loci.
pyrad2migrate.py - Converts PyRAD .allele file to migrate-n input file (population designated
indicated in supplied popfile). Note: only appropriate for PyRAD .allele
file (not .loci). [File did not pass PEP8 check]
usage: pyrad2migrate.py [-h] allele_file pop_file
positional arguments:
allele_file PyRAD allele file (.allele)
pop_file text file (tsv or csv) with individuals and populations
optional arguments:
-h, --help show this help message and exit
Example input file(s): pop_file.txt.
fastq_barcodes2samplenames.py - Renames barcoded .fastq.gz files in a folder to sample/individual names. Takes relative or absolute path as first argument (e.g. 'samples'; without trailing slash) and a text file as second argument. The latter should be a tab- separated text file, with the barcode in the first column, and the new sample/individual name in the second column. The script conducts a trial run first, listing the files to be renamed, and then asks for confirmation before doing the actual renaming. [File did not pass PEP8 check]
usage: fastq_barcodes2samplenames.py [-h] path barcode_file
positional arguments:
path path (for current directory use `.`)
barcode_file text file (tsv or csv) with barcodes and sample names
optional arguments:
-h, --help show this help message and exit
fastq_seqcount.py - Outputs number of reads for each fastq.gz sample to text file, and prints
mean/min/max to STDOUT. Note: Does not work with all FASTQ formats, and
correct depending on OS zcat or gzcat needs to be set in COMPRESS_UTIL
constant. [File did not pass PEP8 check]
usage: fastq_seqcount.py [-h] path output_filename
positional arguments:
path path (for current directory use `.`)
output_filename name of text output file
optional arguments:
-h, --help show this help message and exit
fastq_bin_paired_reads.py - Clusters reads of paired-end RAD-seq data for downstream contig assembly. It
maps R1 reads to a reference, and then outputs those reads and the
corresponding R2 reads to a separate 'shuffled' FASTQ file per locus. Note:
when using an existing output folder, reads are being appended to existing
files (use this to append data from multiple samples). BWA needs to be
installed and accessible through PATH environmental variable. [File did not pass PEP8 check]
usage: fastq_bin_paired_reads.py [-h]
r1_fastq_file r2_fastq_file ref_fasta_file
threads output_folder
positional arguments:
r1_fastq_file file in FASTQ format with R1 reads
r2_fastq_file file in FASTQ format with R2 reads
ref_fasta_file file in FASTA format with reference contigs
threads number of threads to be used by BWA
output_folder name of output folder
optional arguments:
-h, --help show this help message and exit
mapping_get_bwa_matches.py - Extracts a list of succesfully mapped loci from .sam file (produced with
bwa mem). Successfully mapped loci are identified by default identified as
those with flags 0 and 16 (can be adjusted in MATCH_FLAGS constant), and a
mapping quality of >=20. Configured for use with single-end reads. [File did not pass PEP8 check]
usage: mapping_get_bwa_matches.py [-h] sam_file
positional arguments:
sam_file `bwa mem` output file (`.sam`)
optional arguments:
-h, --help show this help message and exit
Example input file(s): sam_file.sam.
mapping_identify_blast_matches.py - Extracts a list of loci that have a blastn e-value below a certain threshold,
and outputs the (first) matching reference locus, as well as the alignment
length, nident, e-value and bitscore. It also compiles a set of all tax_ids,
which it uses to connect with the NCBI taxonomy database to get phylum ids for
each match using Entrez. Results are outputted to file with the chosen e-value
as post-fix, and STDOUT gives minimum alignment stats for filtered loci. Note:
fields in input file should be (in this order): query id, subject id,
alignment length, identity, perc. identity, evalue, bitscore, staxids, stitle.
This can be achieved by using blastn with the following argument: -outfmt 7 qseqid sseqid length nident pident evalue bitscore staxids stitle. [File did not pass PEP8 check]
usage: mapping_identify_blast_matches.py [-h] blastn_file evalue_cut_off email
positional arguments:
blastn_file blastn output file with the following fields (in that
order): query id, subject id, alignment length, identity,
perc. identity, evalue, bitscore
evalue_cut_off maximum e-value for match to be included
email email address to be used for NCBI connection
optional arguments:
-h, --help show this help message and exit
Example input file(s): blastn_file.txt.
mapping_get_blastn_matches.py - Extracts a list of loci that have a blastn e-value below a certain threshold,
and outputs the (first) matching reference locus, as well as the alignment
length, nident, and pident. Results are outputted to file with the chosen
e-value as post-fix, and STDOUT gives minimum alignment stats for filtered
loci. Note: fields in input file should be (in this order): query id, subject
id, alignment length, identity, perc. identity, evalue, bitscore (additional
fields beyond that are fine). This can be achieved by using blastn with the
following argument: -outfmt 7 qseqid sseqid length nident pident evalue bitscore. [File did not pass PEP8 check]
usage: mapping_get_blastn_matches.py [-h] blastn_file evalue_cut_off
positional arguments:
blastn_file blastn output file with the following fields (in that
order): query id, subject id, alignment length, identity,
perc. identity, evalue, bitscore
evalue_cut_off maximum e-value for match to be included
optional arguments:
-h, --help show this help message and exit
Example input file(s): blastn_file.txt.
popfile_toggleassign.py - Shuffle the assignment of individuals to populations by assigning the indivs sequentially to the different pops. The assignment is not completely random, but does generate equal population sizes (which otherwise differ substantially when using random assignment under originally small population sizes). [File did not pass PEP8 check]
usage: popfile_toggleassign.py [-h] pop_file
positional arguments:
pop_file text file (tsv or csv) with individuals and populations
optional arguments:
-h, --help show this help message and exit
Example input file(s): pop_file.txt.
popfile_randomize.py - Pseudo-randomize the assignment of individuals to populations. Note: with
small population sizes, this can lead to very uneven simulated population
sizes. See also the alternative: popfile_toggleassign.py. Individuals in
original popfile should be ordered by population. [File did not pass PEP8 check]
usage: popfile_randomize.py [-h] pop_file
positional arguments:
pop_file text file (tsv or csv) with individuals and populations
optional arguments:
-h, --help show this help message and exit
Example input file(s): pop_file.txt.
popfile_from_clusters.py - Outputs a popfile based on cluster assignment file (from e.g. STRUCTURE) and
outputs a popfile based on those assigments (using supplied assignment
threshold). Note: I use the formatted CLUMPP output (e.g. clumpp_K4.out.csv)
from the structure_mp wrapper as assignment file. [File did not pass PEP8 check]
usage: popfile_from_clusters.py [-h] [-p pop_filename]
assignment_file assign_cut_off
positional arguments:
assignment_file text file (tsv or csv) with assignment values for each
individual (max. 2 clusters); e.g. a reformatted
STRUCTURE output file
assign_cut_off min. assignment value for an individual to be assigned
to a cluster
optional arguments:
-h, --help show this help message and exit
-p pop_filename, --popfile pop_filename
optional popfile: use original popnames as assignment
prefix
Example input file(s): assignment_file.csv.
structure_mp_plot.py - Plots all the results from a structure_mp run to a multi-page PDF. [File did not pass PEP8 check]
usage: structure_mp_plot.py [-h] [-o order_filename] [-p] [-c] path
positional arguments:
path path to structure_mp results
optional arguments:
-h, --help show this help message and exit
-o order_filename, --orderfile order_filename
optional file specifying the output order of samples
-p, --popnames set flag to output population names
-c, --clumpp_only set flag to only plot CLUMPP summary
nexus_set_label_colors.py - Set the color of each label in a NEXUS tree file.
usage: nexus_set_label_colors.py [-h] nexus_filename color_filename
positional arguments:
nexus_filename nexus input file)
color_filename file with samples and corresponding colors
optional arguments:
-h, --help show this help message and exit
nexus_append_label_groups.py - Appends group to each label in a NEXUS tree file. [File did not pass PEP8 check]
usage: nexus_append_label_groups.py [-h] nexus_filename group_filename
positional arguments:
nexus_filename nexus input file)
group_filename file with samples and corresponding groups
optional arguments:
-h, --help show this help message and exit
gdmatrix2tree.py - Creates NJ tree from a genetic distance matrix. Outputs ASCII format to STDOUT
and a nexus-formatted tree to output file. Note: distance matrix can be
created from vcf using vcf_gdmatrix.py. [File did not pass PEP8 check]
usage: gdmatrix2tree.py [-h] matrix_file tree_output_file
positional arguments:
matrix_file text file (tsv or csv) with genetic distance matrix
tree_output_file nexus file with output tree
optional arguments:
-h, --help show this help message and exit
Example input file(s): matrix_file.txt.
README_compile.py - Compiles README markdown file for this repository (https://github.com/pimbongaerts/radseq). Categories are assigned based on prefix, usage information is extracted from argparse, and example input files are assigned based on argument names. [File did not pass PEP8 check]
usage: README_compile.py [-h]
optional arguments:
-h, --help show this help message and exit
fasta_exclude.py - Reduces FASTA file to those loci not listed in supplied text file. [File did not pass PEP8 check]
usage: fasta_exclude.py [-h] fasta_file exclusion_file
positional arguments:
fasta_file FASTA input file (`.fasta`/ `.fa`)
exclusion_file text file with names of loci to be excluded
optional arguments:
-h, --help show this help message and exit
Example input file(s): fasta_file.fa, exclusion_file.txt.
fasta_include.py - Reduces FASTA file to only those loci listed in supplied text file. [File did not pass PEP8 check]
usage: fasta_include.py [-h] fasta_file inclusion_file
positional arguments:
fasta_file FASTA input file (`.fasta`/ `.fa`)
inclusion_file text file with names of loci to be included
optional arguments:
-h, --help show this help message and exit
Example input file(s): fasta_file.fa, inclusion_file.txt.
bwa_distance_filter.py - Filters list of loci mapped to genome scaffolds, so that they are spaced at least a certain distance. Input file should be tab-separated with columns in the following order (no header): rad_locus, ref_scaffold, ref_start_pos, flag. Required spacing between POS will be spacing + max_locus_length. [File did not pass PEP8 check]
usage: bwa_distance_filter.py [-h] [-l loci_file]
filename spacing max_locus_length
positional arguments:
filename input file
spacing desired spacing between loci
max_locus_length max. length of loci
optional arguments:
-h, --help show this help message and exit
-l loci_file, --loci loci_file
file with loci to be considered
goterms_from_uniprot_blast.py - Creates a list of GO terms for each annotated gene. [File did not pass PEP8 check]
usage: goterms_from_uniprot_blast.py [-h]
gene2uniprot_filename uniprot2go_filename
positional arguments:
gene2uniprot_filename
input file (tsv) with the custom gene ids (first
column) and the corresponding UniProt IDs (second
column); when multiple UniProt IDs are given for each
gene, they should be sorted by highest match)
uniprot2go_filename input file (tsv) with UniProt gene ids (first column)
and the corresponding GO terms (second column)
separated by semi-colons (;)
optional arguments:
-h, --help show this help message and exit
itertools_combinations.py - Generate list with all unique pairwise combinations of values in file. Short script meant to allow use of itertools.combinations in bash. [File did not pass PEP8 check]
usage: itertools_combinations.py [-h] filename
positional arguments:
filename input file with values
optional arguments:
-h, --help show this help message and exit
structure_mp.py - Multi-processing STRUCTURE (Pritchard et al 2000) wrapper for RAD-seq data.
Takes a .vcf as input file and then creates a number of replicate datasets,
each with a different pseudo-random subsampling of one SNP per RAD contig.
Then, it runs the replicate datasets through STRUCTURE across multiple
threads, and summarises the outcome with CLUMPP (Jakobsson and Rosenberg
2007). Finally, it assesses the number of potential clusters using the
Puechmaille 2016 method (only suitable for certain datasets). Note: STILL
NEEDS TO BE MODIFIED FOR GENERAL USE. Input file (.vcf) should be sorted by
CHROM. The mainparams' and extraparamsfile for STRUCTURE need to be present in the current directory (with the desired settings - althoughNUMINDSandNUMLOCI` can be set to 0 as these will be supplied to STRUCTURE
by the script). The paramfile for CLUMPP will be generated by the script and
does not need to be supplied. [File did not pass PEP8 check]
usage: structure_mp.py [-h] vcf_file pop_file maxK replicates threads
positional arguments:
vcf_file input file with SNP data (`.vcf`)
pop_file population file (.txt)
maxK maximum number of K (expected clusters)
replicates number of replicate runs for each K
threads number of parallel threads
optional arguments:
-h, --help show this help message and exit
Example input file(s): vcf_file.vcf, pop_file.txt.