Skip to content

Commit

Permalink
groupby example
Browse files Browse the repository at this point in the history
  • Loading branch information
lazarusA committed Mar 12, 2024
1 parent 2f24ee7 commit 149dc8a
Show file tree
Hide file tree
Showing 6 changed files with 63 additions and 33 deletions.
7 changes: 5 additions & 2 deletions docs/src/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export default defineConfig({
{ text: 'Setting chunks size', link: '/UserGuide/setchuncks' },
{ text: 'Apply functions on YAXArrays', link: '/UserGuide/applyfunctions' },
{ text: 'Create Cube from function', link: '/UserGuide/create_cube_from_function' },
{ text: 'Group by', link: '/UserGuide/group_by' },
{ text: 'Distributed computing', link: '/UserGuide/distributed' },
{ text: 'Open NetCDF', link: '/UserGuide/openNetCDF' },
{ text: 'Open Zarr (Store)', link: '/UserGuide/openZarr' },
Expand All @@ -64,7 +65,8 @@ export default defineConfig({
{ text: 'How do I?',
items: [
{ text: 'How do I ...', link: '/HowdoI/howdoi' },
{ text: 'Contribute to docs', link: '/HowdoI/contribute' }
{ text: 'Contribute to docs', link: '/HowdoI/contribute' },
{ text: 'Contributors', link: '/contributors' }
]},
],

Expand All @@ -78,6 +80,7 @@ export default defineConfig({
{ text: 'Setting chunks size', link: '/UserGuide/setchuncks' },
{ text: 'Apply functions on YAXArrays', link: '/UserGuide/applyfunctions' },
{ text: 'Create Cube from function', link: '/UserGuide/create_cube_from_function' },
{ text: 'Group by', link: '/UserGuide/group_by' },
{ text: 'Distributed computing', link: '/UserGuide/distributed' },
{ text: 'Open NetCDF', link: '/UserGuide/openNetCDF' },
{ text: 'Open Zarr (Store)', link: '/UserGuide/openZarr' },
Expand All @@ -91,7 +94,7 @@ export default defineConfig({
{ text: 'How do I?',
items: [
{ text: 'How do I ...', link: '/HowdoI/howdoi' },
{ text: 'Contribute to docs', link: '/HowdoI/contribute' }
{ text: 'Contribute to docs', link: '/HowdoI/contribute' },
]},
{ text: 'Contributors', link: '/contributors' },
{ text: 'API',
Expand Down
55 changes: 41 additions & 14 deletions docs/src/.vitepress/theme/style.css
Original file line number Diff line number Diff line change
@@ -1,26 +1,53 @@
@import url(https://fonts.googleapis.com/css?family=Space+Mono:regular,italic,700,700italic);
@import url(https://fonts.googleapis.com/css?family=Space+Grotesk:regular,italic,700,700italic);

/**
* Customize default theme styling by overriding CSS variables:
* https://github.com/vuejs/vitepress/blob/main/src/client/theme-default/styles/vars.css
/* Customize default theme styling by overriding CSS variables:
https://github.com/vuejs/vitepress/blob/main/src/client/theme-default/styles/vars.css
*/

/**
* Fonts
* --------------------------------------------------------------------------
*/
:root {
/* Layouts */

/*
:root {
--vp-layout-max-width: 1440px;
} */

.VPHero .clip {
white-space: pre;
max-width: 500px;
}

/* Fonts */

@font-face {
font-family: JuliaMono-Regular;
src: url("https://cdn.jsdelivr.net/gh/cormullion/juliamono/webfonts/JuliaMono-Regular.woff2");
}

:root {
/* Typography */
--vp-font-family-base: "Space Grotesk", "Inter var experimental", "Inter var",
--vp-font-family-base: "Barlow", "Inter var experimental", "Inter var",
-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu,
Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif;

/* Code Snippet font */
--vp-font-family-mono: "Space Mono", Menlo, Monaco, Consolas, "Courier New",
monospace;
--vp-font-family-mono: JuliaMono-Regular, monospace;

}

.mono {
/*
Disable contextual alternates (kind of like ligatures but different) in monospace,
which turns `/>` to an up arrow and `|>` (the Julia pipe symbol) to an up arrow as well.
This is pretty bad for Julia folks reading even though copy+paste retains the same text.
*/
font-feature-settings: 'calt' 0;
pre {
font-family: JuliaMono-Light;
};
code {
font-family: JuliaMono-Light;
};
}


/**
* Colors
For inspiration visit
Expand Down
28 changes: 14 additions & 14 deletions docs/src/UserGuide/group_by.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# GroupBy

The next examples will use the `groupby` function to calculate temporal and spatial averages.
The following examples will use the `groupby` function to calculate temporal and spatial averages.

````@example compareXarray
using YAXArrays, DimensionalData
Expand All @@ -27,7 +27,7 @@ nothing # hide

::: warning

The following rebuild should not be necessary in the future, plus is unpractical to use for large data sets. Support for out of memory data is not available yet.
The following rebuild should not be necessary in the future, plus is unpractical to use for large data sets. Out of memory groupby currently is work in progress.
Related to https://github.com/rafaqz/DimensionalData.jl/issues/642

:::
Expand All @@ -45,18 +45,18 @@ nothing # hide

## GroupBy: seasons

::: details function weighted_season(ds) ... end
::: details function weighted_seasons(ds) ... end

````julia
function weighted_season(ds)
function weighted_seasons(ds)
# calculate weights
tempo = dims(ds, :Ti)
month_length = YAXArray((tempo,), daysinmonth.(tempo))
g_tempo = groupby(month_length, Ti => season(; start=December))
g_tempo = groupby(month_length, Ti => seasons(; start=December))
sum_days = sum.(g_tempo, dims=:Ti)
weights = map(./, g_tempo, sum_days)
# unweighted seasons
g_ds = groupby(ds, Ti => season(; start=December))
g_ds = groupby(ds, Ti => seasons(; start=December))
mean_g = mean.(g_ds, dims=:Ti)
mean_g = dropdims.(mean_g, dims=:Ti)
# weighted seasons
Expand All @@ -65,16 +65,16 @@ function weighted_season(ds)
weighted_g = dropdims.(weighted_g, dims=:Ti)
# differences
diff_g = map(.-, weighted_g, mean_g)
seasons = lookup(mean_g, :Ti)
return mean_g, weighted_g, diff_g, seasons
seasons_g = lookup(mean_g, :Ti)
return mean_g, weighted_g, diff_g, seasons_g
end
````
:::

Now, we continue with the `groupby` operations as usual

````@ansi compareXarray
g_ds = groupby(ds, Ti => season(; start=December))
g_ds = groupby(ds, Ti => seasons(; start=December))
````

And the mean per season is calculated as follows
Expand All @@ -96,7 +96,7 @@ mean_g = dropdims.(mean_g, dims=:Ti)
Due to the `groupby` function we will obtain new grouping names, in this case in the time dimension:

````@example compareXarray
seasons = lookup(mean_g, :Ti)
seasons_g = lookup(mean_g, :Ti)
````

Next, we will weight this grouping by days/month in each group.
Expand All @@ -113,7 +113,7 @@ month_length = YAXArray((tempo,), daysinmonth.(tempo))
Now group it by season

````@ansi compareXarray
g_tempo = groupby(month_length, Ti => season(; start=December))
g_tempo = groupby(month_length, Ti => seasons(; start=December))
````

Get the number of days per season
Expand Down Expand Up @@ -159,7 +159,7 @@ diff_g = map(.-, weighted_g, mean_g)
All the previous steps are equivalent to calling the function defined at the top:

````julia
mean_g, weighted_g, diff_g, seasons = weighted_season(ds)
mean_g, weighted_g, diff_g, seasons_g = weighted_seasons(ds)
````

Once all calculations are done we can plot the results with `Makie.jl` as follows:
Expand All @@ -180,7 +180,7 @@ with_theme(theme_ggplot2()) do
# the figure
fig = Figure(; size = (850,500))
axs = [Axis(fig[i,j], aspect=DataAspect()) for i in 1:3, j in 1:4]
for (j, s) in enumerate(seasons)
for (j, s) in enumerate(seasons_g)
hm_o = heatmap!(axs[1,j], mean_g[Ti=At(s)]; colorrange, lowclip, highclip, colormap)
hm_w = heatmap!(axs[2,j], weighted_g[Ti=At(s)]; colorrange, lowclip, highclip, colormap)
hm_d = heatmap!(axs[3,j], diff_g[Ti=At(s)]; colorrange=(-0.1,0.1), lowclip, highclip,
Expand All @@ -190,7 +190,7 @@ with_theme(theme_ggplot2()) do
Colorbar(fig[3,5], hm_d, label="Tair")
hidedecorations!.(axs, grid=false, ticks=false, label=false)
# some labels
[axs[1,j].title = string.(s) for (j,s) in enumerate(seasons)]
[axs[1,j].title = string.(s) for (j,s) in enumerate(seasons_g)]
Label(fig[0,1:5], "Seasonal Surface Air Temperature", fontsize=18, font=:bold)
axs[1,1].ylabel = "Unweighted"
axs[2,1].ylabel = "Weighted"
Expand Down
2 changes: 1 addition & 1 deletion docs/src/components/AsideTrustees.vue
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<template>
<br>
<h1 style="text-align:center;"> <strong> Package Ecosystem </strong> </h1>
<h1 style="text-align:center;"> <strong> <font color="orange"> Package Ecosystem</font> </strong> </h1>
<a
class="enjoyer"
href="https://github.com/meggart/DiskArrays.jl"
Expand Down
2 changes: 1 addition & 1 deletion docs/src/contributors.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ const coreMembers = [
.row img {
border-radius: 50%;
width: 60px;
heigth: 60px;
height: 60px;
}
.row {
display: flex;
Expand Down
2 changes: 1 addition & 1 deletion docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ features:
link: /UserGuide/openZarr
- title: Interoperability
details: Well integrated with Julia's ecosystem, i.e., distributed operations are native. And plotting with <font color="#D27D2D">Makie.jl</font> is well supported.
- title: Named dimensions and GroupBy(in memory. WIP, out of memory)
- title: Named dimensions and GroupBy(in memory)
details: Apply operations over named dimensions, select values by labels and integers as well as efficient split-apply-combine operations with <font color="#D27D2D">groupby</font> via DimensionalData.jl.
link: /UserGuide/group_by
- title: Efficiency
Expand Down

0 comments on commit 149dc8a

Please sign in to comment.