README.Rmd

---
output: github_document
---

<!-- README.md is generated from README.Rmd. Please edit that file -->

```{r, include = FALSE}
library(substrait)
library(dplyr)

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)
```

# Substrait R Interface

<!-- badges: start -->
[![R-CMD-check](https://github.com/voltrondata/substrait-r/workflows/R-CMD-check/badge.svg)](https://github.com/voltrondata/substrait-r/actions)
[![Codecov test coverage](https://codecov.io/gh/voltrondata/substrait-r/branch/main/graph/badge.svg)](https://app.codecov.io/gh/voltrondata/substrait-r?branch=main)
<!-- badges: end -->

[Substrait](https://substrait.io) is a cross-language specification for data compute operations. The Substrait specification creates a standard for compute expressions---or what should be done to data. 

The Substrait R package provides an R interface to Substrait, allowing users to construct a Substrait plan from R for evaluation by a Substrait consumer, such as Arrow or DuckDB. This is an experimental package that is under heavy development!

## Installation

You can install the development version of substrait from [GitHub](https://github.com/) with:

``` r
# install.packages("remotes")
remotes::install_github("voltrondata/substrait-r")
```

## Example

Basic construction of a Substrait plan and evaluating it using the DuckDB Substrait consumer:

```{r}
library(substrait)
library(dplyr)

mtcars %>% 
  duckdb_substrait_compiler() %>%
  mutate(mpg_plus_one = mpg + 1) %>% 
  select(mpg, wt, mpg_plus_one) %>% 
  collect()
```

You can inspect the Plan that will be generated by saving the result and calling `$plan()`:

```{r}
compiler <- data.frame(col1 = 1L) %>% 
  duckdb_substrait_compiler() %>%
  mutate(mpg_plus_one = col1 + 1)

compiler$plan()
```

You can also construct a Substrait plan and evaluate it using the Acero Substrait consumer. To use the Acero Substrait consumer you will need a [special build of the arrow package](https://arrow.apache.org/docs/r/articles/developers/setup.html) with Arrow configured using `-DARROW_SUBSTRAIT=ON`.

```{r}
library(substrait)
library(dplyr)

mtcars %>% 
  arrow_substrait_compiler() %>%
  mutate(mpg_plus_one = mpg + 1) %>% 
  select(mpg, wt, mpg_plus_one) %>% 
  collect()
```

## Create 'Substrait' proto objects

You can create Substrait proto objects using the `substrait` base object or using `substrait_create()`:

```{r}
substrait$Type$Boolean$create()
substrait_create("substrait.Type.Boolean")
```

You can convert an R object *to* a Substrait object using `as_substrait(object, type)`:

```{r}
(msg <- as_substrait(4L, "substrait.Expression"))
```

The `type` can be either a string of the qualified name or an object (which is needed to communicate certain types like `"substrait.Expression.Literal.Decimal"` which has a `precision` and `scale` in addition to the `value`).

Restore an R object *from* a Substrait object using `from_substrait(message, prototype)`:

```{r}
from_substrait(msg, integer())
```

Substrait objects are list-like (i.e., methods defined for `[[` and `$`), so you can get or set fields. Note that unset is different than `NULL` (just like an R list).

```{r}
msg$literal <- substrait$Expression$Literal$create(i32 = 5L)
msg
```

The constructors are currently implemented using about 1000 lines of auto-generated code made by inspecting the nanopb-compiled .proto files and RProtoBuf. This is probably not the best final approach but allows us to get started writing good `as_substrait()` and `from_substrait()` methods for various types of R objects.