Skip to content

Commit

Permalink
Merge pull request #2 from invenia/gm/readme
Browse files Browse the repository at this point in the history
Tidy up READM,  docs, and CI
  • Loading branch information
glennmoy authored Mar 23, 2020
2 parents de5d4ad + 37462d7 commit 384d219
Show file tree
Hide file tree
Showing 9 changed files with 81 additions and 53 deletions.
4 changes: 0 additions & 4 deletions .gitlab-ci.yml

This file was deleted.

28 changes: 28 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Documentation: http://docs.travis-ci.com/user/languages/julia/
language: julia
os:
- linux
- osx
julia:
- 1.0
- 1.3
- 1.4
- nightly
matrix:
allow_failures:
- julia: nightly
fast_finish: true
notifications:
email: false
after_success:
- julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit(process_folder())'
jobs:
include:
- stage: Documentation
julia: 1.3
script: julia --project=docs -e '
using Pkg;
Pkg.develop(PackageSpec(path=pwd()));
Pkg.instantiate();
include("docs/make.jl");'
after_success: skip
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
# Models

[![Latest](https://img.shields.io/badge/docs-latest-blue.svg)](https://invenia.pages.invenia.ca/research/Models.jl/)
[![Coverage](https://gitlab.invenia.ca/invenia/Models.jl/badges/master/coverage.svg)](https://gitlab.invenia.ca/invenia/Models.jl/commits/master)
[![Stable](https://img.shields.io/badge/docs-latest-blue.svg)](https://invenia.github.io/Models.jl/stable/)
[![Latest](https://img.shields.io/badge/docs-latest-blue.svg)](https://invenia.github.io/Models.jl/dev/)

[![Build Status](https://travis-ci.com/invenia/Models.jl.svg?branch=master)](https://travis-ci.com/invenia/Models.jl)
[![Codecov](https://codecov.io/gh/invenia/Models.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/invenia/Models.jl)

[![code style blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/invenia/BlueStyle)

## Why does this package exist?

[Models.jl](https://gitlab.invenia.ca/invenia/research/Models.jl) defines the [`Template`](@ref) and [`Model`](@ref) types as well as a common API for constructing a generic model in downstream packages, including:
[Models.jl](https://github.com/invenia/Models.jl) defines the `Template` and `Model` types as well as a common API for constructing a generic model in downstream packages, including:

* Calling `fit` on a `Template`.
* Calling `predict` on a `Model`.
* Assigning traits such as `EstimateTrait` and `OutputTrait`.
* Testing interfaces and downstream dependencies with `TestUtils`.

For common examples of the interface being implemented see [BaselineModels.jl](https://gitlab.invenia.ca/invenia/research/BaselineModels.jl).
7 changes: 6 additions & 1 deletion docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,14 @@ makedocs(;
"Design" => "design.md",
"TestUtils" => "testutils.md",
],
repo="https://gitlab.invenia.ca/invenia/Models.jl/blob/{commit}{path}#L{line}",
repo="https://github.com/invenia/Models.jl/blob/{commit}{path}#L{line}"
sitename="Models.jl",
authors="Invenia Technical Computing Corporation",
strict=true,
checkdocs=:exports,
)

deploydocs(;
repo="github.com/invenia/Models.jl",
push_preview=true,
)
60 changes: 29 additions & 31 deletions docs/src/design.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
## Design Documentation
# Design

This page details the key features of the design of BaselineModels.

BaselineModels exists to solve the issue highlighted by following quote:
This page details the key features of the design of [Models.jl](https://github.com/invenia/Models.jl), which exists to solve the issue highlighted by following quote:

> ML researchers tend to develop general purpose solutions as self-contained packages.
> A wide variety of these are available as open-source packages ...
Expand All @@ -13,22 +11,22 @@ BaselineModels exists to solve the issue highlighted by following quote:
-- [Sculley et al 2015](https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems)

BaselineModels provides a common API for mostly preexisting models to allow them to all be used in the same way.
[Models.jl](https://github.com/invenia/Models.jl) provides a common API for mostly preexisting models to allow them to all be used in the same way.
As such, the most important thing is that it itself has a common API.
Here are some facts about that API:

### Models and Templates
## Models and Templates

A **model** is an object that can be used to make predictions via calling `predict`.
A **template** is an object that can create a *model* by being `fit` to some data.
A [`Model`](@ref) is an object that can be used to make predictions via calling [`predict`](@ref).
A [`Template`](@ref) is an object that can create a [`Model`](@ref) by being [`fit`](@ref) to some data.

All information about how to perform `fit`, such as hyper-parameters, is stored inside the *template*.
This is different from some other APIs which might for example pass those as keyword arguments to `fit`.
The template based API is superior to these as it means `fit` is always the same.
One does not have to carry both a model type, and a varying collection of keyword arguments, which would get complicated when composing wrapper models.
All information about how to perform [`fit`](@ref), such as hyper-parameters, is stored inside the [`Template`](@ref).
This is different from some other APIs which might, for example, pass hyper-parameters as keyword arguments to [`fit`](@ref).
The [`Template`](@ref) based API is superior to these as it means [`fit`](@ref) is always the same.
One does not have to carry both a [`Model`](@ref) type, and a varying collection of keyword arguments, which would get complicated when composing wrapper models.


### `fit` and `predict`
## Calling `fit` and `predict`

```julia
model = StatsBase.fit(
Expand All @@ -46,30 +44,30 @@ outputs = StatsBase.predict(
)::AbstractMatrix # always Variates x Observations
```

`fit` takes in a *template* and some *data* and returns a `Model` that has been fit to the data.
`predict` takes a `Model` (that has been `fit` from a *template*) and produces a predicted output.
[`fit`](@ref) takes in a [`Template`](@ref) and some *data* and returns a [`Model`](@ref) that has been fit to the data.
[`predict`](@ref) takes a [`Model`](@ref) (that has been [`fit`](@ref) from a [`Template`](@ref)) and produces a predicted output.

Important facts about `fit` and `predict`:
Important facts about [`fit`](@ref) and [`predict`](@ref):
- `outputs` and `inputs` always have observations as the second dimension -- even if it is [`SingleOutput`](@ref) (that just means that it will be a `1 x num_obs` output. (See [Docs on Julia being column-major](https://docs.julialang.org/en/v1/manual/performance-tips/#Access-arrays-in-memory-order,-along-columns-1))
- The functions must accept any `AbstractMatrix` for the `inputs` and `outputs` (`fit` only). If the underlying implementation needs a plain dense `Matrix` then `fit`/`predict` should perform the conversion.
- `fit` always accepts a `weights` argument. If the underlying model does not support weighted fitting, then `fit` should throw and error if the weights that passed in and are not all equal.
- `fit`/`predict` take no keyword arguments, or any other arguments except the ones shown.
- The functions must accept any `AbstractMatrix` for the `inputs` and `outputs` ([`fit`](@ref) only). If the underlying implementation needs a plain dense `Matrix` then [`fit`](@ref)/[`predict`](@ref) should perform the conversion.
- [`fit`](@ref) always accepts a `weights` argument. If the underlying [`Model`](@ref) does not support weighted fitting, then [`fit`](@ref) should throw and error if the weights that passed in and are not all equal.
- [`fit`](@ref)/[`predict`](@ref) take no keyword arguments, or any other arguments except the ones shown.

### Traits
## Traits

This package largely avoids using complicated abstract types, or relying on a model having a particular abstract type.
Instead we use [traits](https://invenia.github.io/blog/2019/11/06/julialang-features-part-2/) to determine model behavior.
This package largely avoids using complicated abstract types, or relying on a [`Model`](@ref) having a particular abstract type.
Instead we use [traits](https://invenia.github.io/blog/2019/11/06/julialang-features-part-2/) to determine [`Model`](@ref) behavior.

Here are the current model traits in use and their possible values:
- `estimate_type` - determines what kinds of estimates the model outputs.
- `PointEstimate`: Predicts point-estimates of the most likely values.
- `DistributionEstimate`: Estimates distributions over possible values.
- `output_type` - determines how many output variates a model can learn
- `SingleOutput`: Fits and predicts on a single output only.
- `MultiOutput`: Fits and predicts on multiple outputs at a time.
Here are the current [`Model`](@ref) traits in use and their possible values:
- [`estimate_type`](@ref) - determines what kinds of estimates the [`Model`](@ref) outputs.
- [`PointEstimate`](@ref): Predicts point-estimates of the most likely values.
- [`DistributionEstimate`](@ref): Estimates distributions over possible values.
- [`output_type`](@ref) - determines how many output variates a [`Model`](@ref) can learn
- [`SingleOutput`](@ref): Fits and predicts on a single output only.
- [`MultiOutput`](@ref): Fits and predicts on multiple outputs at a time.

The traits always agree between the model and the template.
Every model and template should define all the listed traits.
The traits always agree between the [`Model`](@ref) and the [`Template`](@ref).
Every [`Model`](@ref) and [`Template`](@ref) should define all the listed traits.

This package uses traits implemented such that the trait function returns an `abstract type` (rather than an instance).
That means to check a trait one uses:
Expand Down
4 changes: 1 addition & 3 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,13 @@

## Why does this package exist?

[Models.jl](https://gitlab.invenia.ca/invenia/research/Models.jl) defines the [`Template`](@ref) and [`Model`](@ref) types as well as a common API for constructing a generic model in downstream packages, including:
[Models.jl](https://github.com/invenia/Models.jl) defines the [`Template`](@ref) and [`Model`](@ref) types as well as a common API for constructing a generic model in downstream packages, including:

* Calling [`fit`](@ref) on a [`Template`](@ref).
* Calling [`predict`](@ref) on a [`Model`](@ref).
* Assigning traits such as [`EstimateTrait`](@ref) and [`OutputTrait`](@ref).
* Testing interfaces and downstream dependencies with [`TestUtils`](@ref).

For common examples of the interface being implemented see [BaselineModels.jl](https://gitlab.invenia.ca/invenia/research/BaselineModels.jl).

## Contents
```@contents
Pages = ["api.md", "testutils.md"]
Expand Down
2 changes: 1 addition & 1 deletion docs/src/testutils.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# TestUtils

Provides test fakes, [`FakeTemplate`](@ref) and [`FakeModel`](@ref), that are useful for
testing downstream dependencies, and [`test_interface`](@ref) for testing the Model's API
testing downstream dependencies, and [`test_interface`](@ref) for testing the model's API
has been correctly implemented.

```@autodocs
Expand Down
10 changes: 5 additions & 5 deletions src/Models.jl
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ export OutputTrait, SingleOutput, MultiOutput
A Template is an untrained [`Model`](@ref) that can be [`fit`](@ref) to data.
Defined as well are the traits:
- [`output_type`](@ref): SingleOutput or MultiOutput
- [`estimate_type`](@ref): PointEstimate or DistributionEstimate
- [`output_type`](@ref): [`SingleOutput`](@ref) or [`MultiOutput`](@ref).
- [`estimate_type`](@ref): [`PointEstimate`](@ref) or [`DistributionEstimate`](@ref).
"""
abstract type Template end

Expand All @@ -22,8 +22,8 @@ abstract type Template end
A Model is a trained [`Template`](@ref) with which one can [`predict`](@ref) on inputs.
Defined as well are the traits:
- [`output_type`](@ref): SingleOutput or MultiOutput
- [`estimate_type`](@ref): PointEstimate or DistributionEstimate
- [`output_type`](@ref): [`SingleOutput`](@ref) or [`MultiOutput`](@ref).
- [`estimate_type`](@ref): [`PointEstimate`](@ref) or [`DistributionEstimate`](@ref).
"""
abstract type Model end

Expand All @@ -40,7 +40,7 @@ function fit end
Predict targets for the provided `input` and [`Model`](@ref).
Returns a predictive distribution or point estimates depending on the [`Model`](ref).
Returns a predictive distribution or point estimates depending on the [`Model`](@ref).
"""
function predict end

Expand Down
6 changes: 3 additions & 3 deletions test/traits.jl
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
@testset "traits.jl" begin
struct DummyTemplate <: Template end
struct DummyModel <: Model end

struct DummyTemplate <: Template end
struct DummyModel <: Model end
@testset "traits.jl" begin

estimates = (PointEstimate, DistributionEstimate)
outputs = (SingleOutput, MultiOutput)
Expand Down

2 comments on commit 384d219

@glennmoy
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/11496

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v0.1.0 -m "<description of version>" 384d219c6bc08350b81a0cd785a210d3eff2aefa
git push origin v0.1.0

Please sign in to comment.