Skip to content

VeRCYe Documentation

This documentation aims to provide a comprehensive guide to running the Versatile Crop Yield Estimate (VeRCYe) pipeline. The original VeRCYe algorithm is published here.

Overview

The VeRCYe Repository contains a number of components:

  • The VerCYe Library: Contains all steps to run the VeRCYe algorithm as individual python scripts. The scripts are orchestrated into a pipeline using Snakemake. In general the library is split into two components:
    • LAI Generation: Downloads remotely sensed imagery and predicst Leaf Area Index (LAI) values per pixel.
    • Yield Simulation and Prediction: Simulate numerous likely configurations using APSIM and identify the best-matching simulations with the LAI data. This step also includes evaluation and reporting tools.
  • The VeRCYe Webapp: Provides a webapp wrapper around the core library. Runs a backend and a frontend service that facilitate using VeRCYe operationally.

VeRCYe Library Setup

0. Clone this repository

git clone https://github.com/JPLMLIA/vercye_ops.git
cd vercye_ops

1. Install the requirements

VeRCYe is distributed as a Python package with a conda-managed runtime.

Conda (recommended) - Conda is the only fully supported installation method. It bundles GDAL, Snakemake, Earth Engine, and all other native dependencies into a single self-contained environment. A requirements.txt is provided for reference, but pip-only installs are not supported - GDAL and other system-level libraries are difficult to install correctly through pip alone.

# Production environment
conda env create -f environment/environment.yaml
conda activate vercye

For development (includes linting, testing, and type-checking tools):

# Development environment
conda env create -f environment/environment.dev.yaml
conda activate vercye-dev

2. Install the VeRCYe package

From the root directory, run:

pip install -e .

3. Install APSIMX

VeRCYe depends on process-based APSIM NextGen model for yield & phenology(LAI) simulation. Two modes are supported: - Docker-based APSIM - Local APSIM binary

See the the APSIM Section for details.

4. Authenticate Google Earth Engine

The ERA5 meteorological data is currently still fetched through Google Earth Engine. Authenticate yourself with earth engine and follow the browser flow:

ee authenticate

Running your first yield study

Quickstart

The VeRCYe CLI allows you to get your yield study up an running quickly. However, if you want more options to customize different hyperparameters in a more structured way, you might want to run the study manually, as outlined in the next section.

  1. Activate your virtual environment (depending on your venv setup). E.g:
conda activate vercye
  1. Initialize a new yield study.
vercye init --name your-study-name --dir /path/to/study/store

  1. [Optional] Download remotely sensed imagery & create LAI.

  2. Only required if the LAI data for your region of interest is not yet available locally.

  3. Fill in the lai configuration under /path/to/study/store/lai_config.yaml.
  4. Run the following command to download the imagery:
vercye lai --name your-study-name --dir /path/to/study/store

  1. Prepare your study

  2. Fill in the run congfiguration under /path/to/study/store/setup_config.yaml.

  3. Run the following command to create your study directory and config template.
vercye prep --name your-study-name --dir /path/to/study/store

  1. Set your run options

  2. Fill in the run congfiguration under /path/to/study/store/study/config.yaml.

  3. If you already had the LAI data available locally, ensure to adapt the lai_dir, lai_region and lai_resolution.

  4. [Optional] Download the chirps data.

  5. Only required if chirps data is not yet downloaded for the complete study range.

  6. Ensure you have completed step 4, and have set apsim_params.chirps_dir correctly in /path/to/study/store/study/config.yaml.
vercye chirps --name your-study-name --dir /path/to/study/store

  1. Run your study

  2. You might want to adapt the number of cores to use in /path/to/study/store depending on your system.

vercye run --name your-study-name --dir /path/to/study/store/profile/config.yaml

Running VeRCYe manually

While the CLI provides a convenient way to run a yield study, for larger experiments with different configurations, you might want more freedom. For this the general process is as follows:

  1. You will first have to generate LAI data from remotely sensed imagery. Refer to the LAI Creation Guide for details.

  2. Once you have generated the LAI data, you can run your yield study, by following the Running a Yieldstudy Guide.

VeRCYe Webapp Setup

On information for setting up and running the webapp, visit the Webapp Section.

Technical Details

VeRCYe Architecture Diagram

  • Library Details: The technical implementation details of the vercye library are outlined in the VeRCYe Architecture Section. Fore more details check out the code in vercye_ops.
  • Webapp Details: The details on architectural decisions of the webapp are documented under VeRCYe Webapp.

Development

Development tips and best practices are documented under Development Tipps