Cell-Cell Communication Inference (Source-Target)
Detect interactions between source and target cell types
Description
The growing availability of single-cell data has sparked an increased interest in the inference of cell-cell communication (CCC), with an ever-growing number of computational tools developed for this purpose.
Different tools propose distinct preprocessing steps with diverse scoring functions, that are challenging to compare and evaluate. Furthermore, each tool typically comes with its own set of prior knowledge. To harmonize these, Dimitrov et al, 2022 recently developed the LIANA framework, which was used as a foundation for this task.
The challenges in evaluating the tools are further exacerbated by the lack of a gold standard to benchmark the performance of CCC methods. In an attempt to address this, Dimitrov et al use alternative data modalities, including the spatial proximity of cell types and downstream cytokine activities, to generate an inferred ground truth. However, these modalities are only approximations of biological reality and come with their own assumptions and limitations. In time, the inclusion of more datasets with known ground truth interactions will become available, from which the limitations and advantages of the different CCC methods will be better understood.
This subtask evaluates methods in their ability to predict interactions between spatially-adjacent source cell types and target cell types. This subtask focuses on the prediction of interactions from steady-state, or single-context, single-cell data.
Summary
Metrics
- Precision-recall AUC1: Area under the precision-recall curve for the binary classification task predicting interactions.
- Odds Ratio2: The odds ratio represents the ratio of true and false positives within a set of prioritized interactions (top ranked hits) versus the same ratio for the remainder of the interactions. Thus, in this scenario odds ratios quantify the strength of association between the ability of methods to prioritize interactions and those interactions assigned to the positive class.
Results
Results table of the scores per method, dataset and metric (after scaling). Use the filters to make a custom subselection of methods and datasets. The “Overall mean” dataset is the mean value across all datasets.
Details
Methods
- CellPhoneDB (max)3: CellPhoneDBv2 calculates a mean of ligand-receptor expression as a measure of interaction magnitude, along with a permutation-based p-value as a measure of specificity. Here, we use the former to prioritize interactions, subsequent to filtering according to p-value less than 0.05. Links: Docs.
- CellPhoneDB (sum)3: CellPhoneDBv2 calculates a mean of ligand-receptor expression as a measure of interaction magnitude, along with a permutation-based p-value as a measure of specificity. Here, we use the former to prioritize interactions, subsequent to filtering according to p-value less than 0.05. Links: Docs.
- NATMI (max)5: NATMI uses the product of ligand-receptor expression as a measure of magnitude. As a measure of specificity, NATMI proposes \(specificity.edge = \frac{l}{l_s} \cdot \frac{r}{r_s}\); where \(l\) and \(r\) represent the average expression of ligand and receptor per cell type, and \(l_s\) and \(r_s\) represent the sums of the average ligand and receptor expression across all cell types. We use its specificity measure, as recommended by the authors for single-context predictions. Links: Docs.
- NATMI (sum)5: NATMI uses the product of ligand-receptor expression as a measure of magnitude. As a measure of specificity, NATMI proposes \(specificity.edge = \frac{l}{l_s} \cdot \frac{r}{r_s}\); where \(l\) and \(r\) represent the average expression of ligand and receptor per cell type, and \(l_s\) and \(r_s\) represent the sums of the average ligand and receptor expression across all cell types. We use its specificity measure, as recommended by the authors for single-context predictions. Links: Docs.
Baseline methods
- Random Events: Random generation of cell-cell communication events by random selection of ligand, receptor, source, target, and score.
- True Events: Perfect prediction of cell-cell communication events from target data.
Datasets
- Mouse brain atlas8: A murine brain atlas with adjacent cell types as assumed benchmark truth, inferred from deconvolution proportion correlations using matching 10x Visium slides (see Dimitrov et al., 2022). 14249 cells x 34617 features with 23 cell type labels.
Download raw data
Task info Method info Metric info Dataset info Results Quality control
Quality control results
✓ All checks succeeded!