{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Singularity preprocessing\n", "\n", "Arguably the greatest strength of the dandelion package is a streamlined preprocessing setup making use of a variety of specialised single cell VDJ algorithms:\n", "\n", "1. V(D)J gene reannotation with `igblastn` and parsed to AIRR format with [changeo's](https://changeo.readthedocs.io/en/stable/examples/10x.html) `MakeDB.py` [[Gupta2015]](https://academic.oup.com/bioinformatics/article/31/20/3356/195677), with the pipeline strengthened by running `blastn` in parallel and using the best alignments\n", "2. Reassigning heavy chain IG V gene alleles with [TIgGER](https://tigger.readthedocs.io/en/stable/) [[Gadala-Maria15]](https://www.pnas.org/content/112/8/E862)\n", "3. Reassigning IG constant region calls by blasting against a curated set of highly specific C gene sequences\n", "4. Quantifying mutations via SHazaM's [observedMutations](https://shazam.readthedocs.io/en/stable/topics/observedMutations/)\n", "\n", "However, running this workflow requires a high number of dependencies and databases, which can be troublesome to set up. As such, we've put together a Singularity container that comes pre-configured with all of the required software and resources, allowing you to run the pre-processing pipeline with a single call and easy installation.\n", "\n", "## Setup and running\n", "\n", "Once you have [Singularity installed](https://sylabs.io/guides/3.0/user-guide/installation.html), you can download the [Dandelion container](https://cloud.sylabs.io/library/kt16/default/sc-dandelion). This command will create `sc-dandelion_latest.sif`, note its location.\n", "\n", " singularity pull library://kt16/default/sc-dandelion:latest\n", "\n", "In order to prepare your VDJ data for ingestion, create a folder for each sample you'd like to analyse, name it with your sample ID, and store the Cell Ranger `all_contig_annotations.csv` and `all_contig.fasta` output files inside.\n", "\n", " 5841STDY7998693\n", " ├── all_contig_annotations.csv\n", " └── all_contig.fasta\n", "\n", "Please ensure that the only subfolders present in your folder are such per-sample subfolders with the `.csv` and `.fasta` files.\n", "\n", "You can then navigate to the directory holding all your sample folders and run Dandelion pre-processing like so:\n", "\n", "```bash\n", "singularity run -B $PWD /path/to/sc-dandelion_latest.sif dandelion-preprocess\n", "```\n", "\n", "Any optional arguments get added at the end of this line.\n", "\n", "If you're running TR data rather than IG data, specify `--chain TR` to skip steps 2-4 in the preprocessing. This notably works with TRGD data, which most versions of Cell Ranger struggle to annotate correctly. However, the contigs are still reconstructed, so Dandelion's preprocessing can annotate them for you.\n", "\n", "You can provide the `--filter_to_high_confidence` flag to only keep the contigs that Cell Ranger has called as high confidence. If you wish to process files that have a different prefix than `all`, e.g. `filtered_contig_annotations.csv` and `filtered_contig.fasta`, provide the desired file prefix with `--file_prefix`. In that case, be sure that your input folder contains those files rather than the `all` ones. We use `all` as default as it's possible to subset contigs to relevant ones later.\n", "\n", "## Recommended parameterisation\n", "\n", "If in possession of gene expression data that the VDJ data will be integrated with, the following parameterisation is likely to yield the best results:\n", "\n", "```bash\n", "singularity run -B $PWD /path/to/sc-dandelion_latest.sif dandelion-preprocess \\\n", " --filter_to_high_confidence\n", "```\n", "\n", "Part of the Cell Ranger VDJ filtering criteria is whether the algorithm thinks the contig is in a cell or not, for which you will have superior information based on the gene expression data. The other half of the Cell Ranger VDJ filtering process, requiring the contig to be high confidence, is retained by providing the `--filter_to_high_confidence` flag.\n", "\n", "## Optional arguments\n", "\n", "By default, this workflow will analyse all provided IG samples jointly with TIgGER to maximise inference power, and in the event of multiple input folders will prepend the sample IDs to the cell barcodes to avoid erroneously merging barcodes overlapping between samples at this stage. TIgGER should be ran on a per-individual level. If running the workflow on multiple individuals' worth of data at once, or wanting to flag the cell barcodes in a non-default manner, information can be provided to the script in the form of a CSV file passed through the `--meta` argument:\n", "\n", "1. The first row of the CSV needs to be a header identifying the information in the columns, and the first column needs to contain sample IDs.\n", "2. Barcode flagging can be controlled by an optional `prefix`/`suffix` column. The pipeline will then add the specified prefixes/suffixes to the barcodes of the samples. This may be desirable, as corresponding gene expression samples are likely to have different IDs, and providing the matched ID will pre-format the VDJ output to match the GEX nomenclature.\n", "3. Individual information for TIgGER can be specified in an optional `individual` column. If specified, TIgGER will be ran for each unique value present in the column, pooling the corresponding samples.\n", "\n", "It's possible to just pass a prefix/suffix or individual information. An excerpt of a sample CSV file that could be used on input:\n", "\n", " sample,suffix,individual\n", " 5841STDY7998693,5841STDY7991475,A37\n", " 5841STDY7998694,5841STDY7991476,A37\n", " 5841STDY7998695,5841STDY7991477,A37\n", " WSSS_A_LNG9030827,WSSS_A_LNG8986832,A51\n", " WSSS8090101,WSSS8015042,A40\n", " WSSS8090102,WSSS8015043,A40\n", " [...]\n", "\n", "If specifying a metadata file, only subfolders with names provided in the sample column will be processed.\n", "\n", "The delimiter between the barcode and the prefix/suffix can be controlled with the `--sep` argument. By default, the workflow will strip out the trailing `\"-1\"` from the Cellranger ouput barcode names; pass `--keep_trailing_hyphen_number` if you don't want to do that. Pass `--clean_output` if you want to remove intermediate files and just keep the primary output. The intermediate files may be useful for more detailed inspection.\n", "\n", "For the full list of optional arguments, run:\n", "\n", "```bash\n", "singularity run -B $PWD /path/to/sc-dandelion_latest.sif dandelion-preprocess --help\n", "```\n", "\n", "```python\n", "usage: dandelion_preprocess.py [-h] [--meta META] [--chain CHAIN] [--org ORG]\n", " [--file_prefix FILE_PREFIX] [--db DB]\n", " [--strain STRAIN] [--sep SEP]\n", " [--flavour FLAVOUR]\n", " [--filter_to_high_confidence]\n", " [--keep_trailing_hyphen_number]\n", " [--skip_format_header] [--skip_tigger]\n", " [--skip_reassign_dj] [--skip_correct_c]\n", " [--clean_output]\n", "\n", "options:\n", " -h, --help show this help message and exit\n", " --meta META Optional metadata CSV file, header required, first\n", " column for sample ID matching folder names in the\n", " directory this is being ran in. Can have a\n", " \"prefix\"/\"suffix\" column for barcode alteration, and\n", " \"individual\" to provide tigger groupings that isn't\n", " analysing all of the samples jointly.\n", " --chain CHAIN Whether the data is TR or IG, as the preprocessing\n", " pipelines differ. Defaults to \"IG\".\n", " --org ORG organism for running the reannotation. human or mouse.\n", " --file_prefix FILE_PREFIX\n", " Which set of contig files to take for the folder. For\n", " a given PREFIX, will use PREFIX_contig_annotations.csv\n", " and PREFIX_contig.fasta. Defaults to \"all\".\n", " --db DB Which database to use for reannotation. imgt or ogrdb.\n", " --strain STRAIN Which mouse strain to use for running the\n", " reannotation. Only for ogrdb. Defaults to all (None)\n", " mouse strains.\n", " --sep SEP The separator to place between the barcode and\n", " prefix/suffix. Uses sample names as a prefix for BCR\n", " data if metadata CSV file absent and more than one\n", " sample to process. Defaults to \"_\".\n", " --flavour FLAVOUR The \"flavour\" for running igblastn reannotation.\n", " Accepts either \"strict\" or \"original\". strict will\n", " enforce evalue and penalty cutoffs.\n", " --filter_to_high_confidence\n", " If passed, limits the contig space to ones that are\n", " set to \"True\" in the high_confidence column of the\n", " contig annotation.\n", " --keep_trailing_hyphen_number\n", " If passed, do not strip out the trailing hyphen\n", " number, e.g. \"-1\", from the end of barcodes.\n", " --skip_format_header If passed, skips formatting of contig headers.\n", " --skip_tigger If passed, skips TIgGER reassign alleles step.\n", " --skip_reassign_dj If passed, skips reassigning d/j calls with blastn\n", " when flavour=strict.\n", " --skip_correct_c If passed, skips correcting c calls at assign_isotypes\n", " stage. Only if Chain == IG.\n", " --clean_output If passed, remove intermediate files that aren't the\n", " primary output from the run reults. The intermediate\n", " files may be occasionally useful for inspection.\n", "```\n", "\n", "## Output\n", "\n", "The main file of interest will be `dandelion/all_contig_dandelion.tsv`, stored in a new subfolder each sample folder. This is an AIRR formatted export of the corrected contigs, which can be used for downstream analysis by both dandelion itself, and other packages like [scirpy](https://icbi-lab.github.io/scirpy/generated/scirpy.io.read_airr.html) [[Sturm2020]](https://academic.oup.com/bioinformatics/article/36/18/4817/5866543) and changeo [[Gupta2015]](https://academic.oup.com/bioinformatics/article/31/20/3356/195677).\n", "\n", "The file above features a contig space filtered with immcantation. If this is not of interest to you and you wish to see the full contig space as provided on input, refer to `dandelion/tmp/all_contig_iblast_db-all.tsv`.\n", "\n", "The plots showing the impact of TIgGER are in `/_reassign_alleles.pdf`, for each TIgGER folder (one per unique individual if using `--meta`, `tigger` otherwise). The impact of C gene reannotation is shown in `dandelion/data/assign_isotype.pdf` for each sample.\n", "\n", "If you're interested in more detail about the pre-processing this offers, or wish to use the workflow in a more advanced manner (e.g. by using your own databases), proceed to the pre-processing section of the advanced guide." ] }, { "cell_type": "markdown", "metadata": {}, "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.8" } }, "nbformat": 4, "nbformat_minor": 4 }