Data Processing with HPC

This is optional. If you have access to a High-Performance Computing system, you can process your data in HPC using the provided HPC pipeline to make an AnnData Object. You can also test multiple Leiden resolution values to determine which fits the dataset best. This is particularly useful if you have a large dataset.

Steps:

• Install CAFE (see the uv Way); this provides the cafe-hpc command
• Assign input and output directory
• Assign parameters. Note that some arguments accept multi-parameters such as Leiden resolution
• Run the pipeline

1. Choose the Parameters

Decide on the arguments to pass to the script:

• --input: Path to the input directory containing the CSV files.
• --output: Path to the output directory where results will be saved.
• --pca: PCA solver to use (choose from auto, full, arpack, randomized, or none).
• --cutoff: Explained variance cutoff for PCA (e.g., 95).
• --leiden: One or more resolutions for Leiden clustering (e.g., 0.5 1.0 2.0).
• --nneighbor: One or more n_neighbors values for UMAP and neighbors computation (e.g., 10 20 30).
• --distance: Distance metric to use for neighbors computation (default: euclidean).
• --min_dist: min_dist parameter for UMAP (default: 0.1).

2. Run the Pipeline

Run it in your terminal using the cafe-hpc command. For example:

cafe-hpc \
  --input /path/to/input_dir \
  --output /path/to/output_dir \
  --pca auto \
  --cutoff 95 \
  --leiden 0.5 1.0 \
  --nneighbor 15 30 \
  --distance euclidean \
  --min_dist 0.1

If you are running from a source checkout instead of an install, the equivalent is python3 CAFE/src/cafe_app/hpc.py ....