Skip to content

Reporting

Fridolin Glatter edited this page Sep 25, 2024 · 32 revisions

Caution

This Wiki was archived on September 25, 2024. Before that, this page was last updated on February 17, 2021.

The documentation on 'Reporting' in the MESSAGEix framework can now be found at https://docs.messageix.org/en/latest/reporting.html. We also offer tutorials on how to use our 'Reporting', see point 3 in this list.

This page describes a 2019 update of reporting features in the MESSAGEix ecosystem. Any enhancements required to ixmp or message_ix should be addressed by opening new issues or PRs.

See also:

Plan/strategy

The reporting update was planned to take place in two phases. It was later decided to skip phase 1.

Phase 1 (cancelled)

Main PR: #142 targeted for message_ix 1.2 / ixmp 0.2 (2019-04-01)

This phase will set the initial user API for reporting, using the use-case of the Westeros tutorials, and be released in time for classroom use in April 2019. Feedback from students will be used to inform changes to the API.

Phase 2

Main PRs: ixmp#150 (superseding #144, #126) / message_ix#206 (supserseding #176) / message_data#41 targeted for message_ix 1.2 / ixmp 0.2 (June 2019)

This phase will used dask to set up an internal architecture based on evaluating nodes on a graph, as described here. The operations (nodes) will rely on pyam as far as possible. The user API should be identical to that in Phase 1, i.e. the implementation details should be invisible to the (average) user, although advanced users may access features to do customized reporting.

Changes in API versus Phase 1 will be addressed with a deprecation-and-removal cycle.

Requirements

These were prepared 20 February 2019 based on:

These are sorted to the different components of the software stack. Refer using section and number, e.g. "A1". The following words and phrases have specific meanings, as do others in bold:

  • quantity: a scalar or array (i.e. over one or more dimensions) variable.
    • May be either model (i.e. present in a model while it is solved) or non-model/exogenous (not present in a model or its solution, but required for reporting).
    • Identified by a name.
    • May have associated units.
  • report: a set of quantities that is ready for some other use, e.g. to be used in plotting or analysis scripts for a paper.
  • operation: any manipulation of model or non-model data that takes some inputs (quantities, data, or parameters) and produces outputs.
    • calculation: a mathematical operation that computes a quantity from its inputs.

A. for the ix modeling platform

  1. Provide equal support for any type of model that can be built on the ixmp.
    • i.e. not specific to GAMS models.
  2. Support use of exogenous data.
  3. Support basic operations:
    1. Weighted sums using model or exogenous weights.
    2. Disaggregation using model or exogenous shares.
    3. Interpolation: linear, exponential, etc.
  4. Support operations using quantities at different levels of aggregation, e.g. for eq1, some operations should be able to use Axy whilst others use Axyz simultaneously.
  5. Support user-defined operations.
  6. Duplicate or clone existing operations for multiple other sets of inputs or outputs.
  7. Support renaming of quantities using multiple name mappings:
    1. …given programmatically.
    2. …loaded from file.
  8. Compute only a subset of quantities from a larger/complete report.
    1. …for testing/development use-cases.
    2. …for high-performance use-cases.
    3. …using command-line arguments or other straightforward inputs to specify the subset.
  9. Handle units for quantities.
    • Both quantities with provided units, and quantities without units.
    • Derive units for operations such as division and multiplication.
  10. Provide automated description of how quantities are computed, as text (console output) or image.
  11. Callable through retixmp.

B. for the MESSAGEix framework

  1. Provide equal support for any type of model that can be built in MESSAGEix, i.e. do not restrict based on specific features of any one model:
    • e.g. Clara's "hospital" model, i.e. where the 'node' set does not map to countries or geographical regions.
    • the Westeros and Austria tutorial models.
    • Single-country models, e.g. South Africa, Israel, others built using Behnam's tools.
    • Models with 'extra' features/detail added: MESSAGE-Access, MESSAGE-Transport.
    • the MESSAGE global model(s).
  2. Provide zero-config reporting, i.e. reporting that is automatically configured by introspection of the model structure. Including:
    • Calculate all outputs of commodities at all levels.
  3. Support user-defined operations:
    1. …on exogenous sub-sets of model sets.
    2. …that exclude quantities from aggregation operations.
    3. …that compute the same quantity in different ways.
    4. Cumulative quantities with defined start and end periods.
  4. Support user-defined reports.
    1. …defined programmatically.
    2. …loaded from a configuration file.
  5. Provide (either automatically or per user input) operations related to model features including:
    1. …sub-annual time steps.
    2. …bi-directional commodity flows.
    3. …user-defined relations. (Probably not automatic; user will need to define reporting.)

C. for reporting in the MESSAGE global model

  1. Support reports that can be output into formats corresponding to the templates for…
    1. CD-LINKS
    2. IAMC
  2. Improve performance, or provide an architecture that allows for future optimization.
    • Baseline: ~10 minutes on standard desktop hardware for the MESSAGE Global model. Longer for the basin-level model.
  3. Provide operations related to model features including:
    1. …trade between regions.
  4. Check consistency between the same quantities computed multiple ways.

D. General

  1. Provide output as objects that can be:
    1. …coerced into other formats.
    2. …written to common file formats including CSV and Excel.
  2. Limit the number of required configuration files.
  3. Balance the complexity/conciseness of configuration file format(s).
  4. Limit code duplication to make the code maintainable.
    1. Re-use functionality provided by pyam where appropriate.
    2. Use other packages where appropriate.

Use cases

Please add your own!

TODO add references, in parentheses, to the requirements that cover the use-cases.

  • Behnam develops a sub-annual version of the Austria tutorial, and wants to plot the results coming out of the resulting model.
  • Paul develops a MESSAGEix-Transport version of the main global model, and wants to report aggregate results across different vehicle types and transport technology groups.
  • Oliver uses the MESSAGE-GLOBIOM global model; and wants to define simple rules such as "every technology that takes coal commodity input from the primary level and outputs the electricity commodity at the secondary level" and have calculations performed for such groups.
  • Alice wants to handle reporting differently for two models in the MESSAGEix framework:
    • For Model X, she wants to use the zero-config/automatic MESSAGEix reporting, but then write her own custom operations to override a few of the calculations.
    • For Model Y, based on the MESSAGE global model, she wants to use the global model's default reports, but write her own custom operations to make tailored calculations for new technologies she has added.
  • Matt wants to calculate two quantities in a MESSAGEix framework model: (1) cumulative installed capacity of different technologies, and (2) total capital costs. He supplies per-vintage capital costs, and generates a report that includes both quantities; the reporting features use his capital costs for (2).
  • Charlie is reporting F-gases from the global model, which has 1 species. He configures reporting so that shares read from a file are used to disaggregate this 1 species to 6 different ones.
  • Daniel wants to compare two methods (AR5 and AR6) of aggregating emissions of Kyoto gases. He can define operations that calculate these aggregates by each method and allow them to be compared from a single report.
  • Edgar finds a report in an IAMC-format text file with the variable "Investment costs|FF&I". He uses the reporting tools to access a flowchart (or text description) of how this quantity was computed.

Earlier content (January 2019)

NB. this is superseded by the actual API as implemented in #142.

API usage example

pp = Reporting()
  .add_activity(...)
  .add_activity()
  .aggregate(...)
  .build()
// reuse processing instance for multiple scenarios
pp.process(scenario1)
pp.process(scenario2)

// or
pp = Reporting().load(...)

Comment by @danielhuppmann: For any realistic use case even in a simple tutorial like Westeros, this will be a very long command. Also, the "write down all configs at once and then run it" is less intuitive from a new-user perspective compared to a "one step after the other" (first let's do primary energy, then do capacity, etc.)

Also, re-using the reporting for multiple scenarios within a notebook is an unlikely use case. In such a case, a user should place the specifications in config file so that it can be used across notebooks.

Suggestion for using yaml as config as an extension of the current proposal

specs.yaml

- activity
    - Primary Energy|Wind
        - technology: wind_ppl
- aggregate
    - Primary Energy

Call reporting specs on scenario: Reporting(scenario, 'specs.yaml')

And/or add a function to the message_ix.Scenario class like scenario.reporting('specs.yaml')