Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev/cohort assess #248

Open
wants to merge 58 commits into
base: develop
Choose a base branch
from
Open

Conversation

emilliman5
Copy link
Collaborator

@emilliman5 emilliman5 commented Apr 12, 2022

A minimal working cohort implementation.

Additions:

  1. cohort_ref class and constructor.
  2. cohort_assess wrapper
  3. cohort_score wrapper
    1. missing summarize_scores
  4. 3 cohort assessments
    1. dependency conflicts (missing or version mismatch)
    2. dependency tree bottlenecking
    3. namespace export conflicts

Many of the helper functions (class coercion, formatters, etc) were copy/pasted from pkg_* versions to get to a working example as well as to enforce/maintain a cohort_* vs pkg_* nomenclature.

Things left outstanding:

 1. `summarize_scores` for `cohort_metrics`
 2. assessment dispatch -- cohort assessments will get execututed against `pkg_refs`
 3. handle cases where underlying `pkg_ref` slot is `NA`/missing when executing assessments
 4. `cohort_score` result formatting
 5. unit testing
 6. vignette -- inprogress
 7. lots of other little things (documentation helpers, class coercion and formatting, etc.)
 8. more assessments
 9. Create better metric for reverse dependency assessment (remove igraph dependency, find metric that better captures dependency bottlenecks)

A brief example:

#Define a cohort of pkgs
pkgs <- c("broom", "cli", "crayon", "dbplyr", "dplyr", "forcats", 
          "ggplot2", "haven", "hms", "httr", "jsonlite", "lubridate",
          "magrittr", "modelr", "pillar", "purrr", "readr", "readxl",
          "reprex", "rlang", "rstudioapi", "rvest", "stringr", "tibble",
          "tidyr", "xml2")

#Create a list of pkg_refs
pr <- lapply(pkgs, pkg_cran)

#Construct a cohort_ref
cohortRef <- cohort_ref(pr, "recommended")

#Run cohort assessments
cohortAssess <- cohortRef %>% cohort_assess()

#Score cohort
cohortScore <- cohortAssess %>% cohort_score()

Eric Milliman and others added 30 commits May 25, 2021 14:18
merge dpenedencies metric into personal dev branch
merge namespace metrics feature into personal dev branch
Merge branch 'master' into dev/cohort

# Conflicts:
#	R/assess_dependencies.R
#	R/assess_exported_namespace.R
#	R/assess_reverse_dependencies.R
#	R/pkg_ref_class.R
#	man/pkg_assess.Rd
#	man/pkg_ref_cache.rev_dependencies.Rd
… to library_ref, and modified its return value to be a list_of_package_ref
…ed cohort_metric for assess_reverse_dependencies cohort_ref
@emilliman5
Copy link
Collaborator Author

A design question to consider.

how to dispatch assessments that are specific for pkg_ref vs cohort_ref. We could modify all_assessments() and get_assessments() to select based on s3 classes. this would also require stricter requirements about assessment method class dispatch. another option would be to use a slightly different assessment naming convention (e.g. instead of assess_has_bug_url, assess_cohort_has_bug_url) however, if an assessment is applicable to both pkg_ref and cohort_ref this would cause some duplication

@codecov-commenter
Copy link

codecov-commenter commented Apr 15, 2022

Codecov Report

Merging #248 (b45ba67) into master (19d429e) will decrease coverage by 5.44%.
The diff coverage is 18.01%.

@@            Coverage Diff             @@
##           master     #248      +/-   ##
==========================================
- Coverage   57.80%   52.36%   -5.45%     
==========================================
  Files          63       68       +5     
  Lines         986     1119     +133     
==========================================
+ Hits          570      586      +16     
- Misses        416      533     +117     
Impacted Files Coverage Δ
R/assess_reverse_dependencies.R 30.76% <0.00%> (-69.24%) ⬇️
R/cohort_assess.R 0.00% <0.00%> (ø)
R/cohort_metric.R 0.00% <0.00%> (ø)
R/cohort_ref_class.R 0.00% <0.00%> (ø)
R/cohort_score.R 0.00% <0.00%> (ø)
R/pkg_metric.R 38.88% <ø> (ø)
R/pkg_ref_class.R 87.40% <0.00%> (-1.40%) ⬇️
R/assess_has_dependency_conflict.R 14.28% <14.28%> (ø)
R/assess_exported_namespace.R 69.23% <20.00%> (-30.77%) ⬇️
R/assess_dependencies.R 67.56% <66.66%> (+3.93%) ⬆️
... and 6 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@elimillera elimillera changed the base branch from master to develop July 17, 2022 14:23
Copy link
Collaborator

@elimillera elimillera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good. I updated the branch to develop to give us a better idea of how this works along with other changes

Comment on lines +17 to +23
if (missing(x) & missing(library) & missing(lib.loc)) {
# return(vctrs::new_list_of(cohort=list_of_pkg_ref(),
# library=list_of_pkg_ref(),
# ptype = list_of_pkg_ref(),
# class = "cohort_ref"))
stop("No packages defined for cohort ref")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove the commented code here. I agree with the option to error out if there isn't anything in the cohort

Copy link
Collaborator

@elimillera elimillera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good. I updated the branch to develop to give us a better idea of how this works along with other changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants