A CLI toolkit for working with dependency data.
Review the following to get up and running quickly.
Ensure JDK 8 or higher is available on your system and java is on your PATH. Download and
explode the release zip. Run the following to start the CLI, adjusting for path differences if
necessary:
java -jar osgeyes-dist-VERSION/osgeyes-VERSION.jar
Once the CLI has loaded, run the following two commands first. Don't forget to adjust for path differences if necessary.
(index-load "osgeyes-dist-VERSION/index-ddf_2_19_5.edn")
(load-file "osgeyes-dist-VERSION/selections.clj")
If you will be scanning new repositories to generate their dependency graphs instead of only leveraging the provided index EDN file, first ensure that all cloned repositories (DDF, Alliance, etc.) are in the same parent directory. Then verify either one of the following:
- You ran
java -jar APPfrom where your cloned repositories live; if DDF lives in~/repos/ddf/then ensure your working directory is~/repos/before you start the app. - Alternatively, you set the
repos.homeproperty when starting the CLI, which looks likejava -Drepos.home=/path/to/cloned/repos -jar APP.
Be sure to account for pathing differences to APP, the path to the OSG-Eyes executable jar. This
also applies for the path arguments to the index-load and load-file commands.
The purpose and value of this tool is to get a rough, high-level view of which components depend on each other. This information can help inform refactoring decisions, component removal, and better separation of concerns across repositories. The two fundamental operations that this tool provides to support those goals are the following:
(list-edges SELECTION OPTIONS)(draw-graph SELECTION)
Run (draw-graph [:node "ddf/.*"]) to generate and open an interactive graph of DDF nodes within
the default browser. There is a lot of information on screen. Filtering the data to get a proper,
bite-sized view of the bigger picture is critical.
Graph nodes directly map to bundle artifacts and are named as follows:
REPO-QUALIFIER/BUNDLE-SYMBOLIC-NAME
The canonical data structure is just a list of edges that claim who depends on what - plus any
additional information that describes the edges. Generating a graph or listing edges is; fundamentally,
just a filter operation on the edge list. You filter by defining a SELECTION. Selections are just
vectors with regex constraints on certain fields of a graph edge, like so:
[CRITERIA REGEX-STRING]
CRITERIA is just a name that represents one or more fields that need to match the compiled
REGEX-STRING. Note that criteria always start with : - no exceptions.
Selecting edges that only concern DDF looks like the following, where a regex
against node names targets the REPO-QUALIFIER part of the name while not imposing constraints
on the BUNDLE-SYMBOLIC-NAME portion:
[:node "ddf/.*"]
Alternatively, to only select edges that are running towards (that depend on) DDF, change the criteria. This ends up widening the selection by a large margin because now any node from any of the scanned repositories will show up if it has a connection to DDF:
[:to "ddf/.*"]
The opposite can be achieved, but it should only return nodes within DDF otherwise we have much bigger problems:
[:from "ddf/.*"]
It follows that a simple wildcard selection could look like [:to ".*"] or [:from ".*"] or
[:node ".*"] - the criteria matters less than the fact that some test has been provided that will
evaluate to true for all edges that are being filtered, thus nothing will get filtered
out. Good luck making any sense of this in the browser, though. It's very noisy and very CPU hungry.
Selecting edges that only concern DDF AND only bundles with "spatial" in the symbolic name looks like this:
[:node "ddf/.*" :node ".*spatial.*"]
OR relationships are not currently supported across different search criteria but they can get baked into the regex for similar effect:
[:node "ddf/.*|alliance/.*" :node ".*spatial.*|.*catalog-core.*"]
The above selection guarantees the resultant edge list only includes edges where both nodes:
(belong to DDF or Alliance) AND (belong to spatial or catalog-core by naming convention only)
There should be enough power here to compose useful-enough selections, even though certain
highly specific selections that involve OR'ing across different CRITERIA cannot be built yet.
You can nest selections.
[[:node "regex1" :node "regex2"][:cause "regex3" :type "regex4"]]
If you find yourself using a selection often, add it to the provided
osgeyes-dist-VERSION/selections.clj file, following the structure already laid out in that file.
(def NAME-OF-SELECTION SELECTION-VALUE)
(def only-ddf [:node "ddf/.*"])
Leveraging selections.clj enables autocomplete and greatly enhanced readability. Most selection
names are prefixed with only- by convention.
[only-ddf only-custom :cause "regex1"]
As long as the final result can be smashed down to a single vector with even size and proper
criteria-regex alternating semantics, it'll work. Note that you'll need to call
(load-file "osgeyes-dist-VERSION/selections.clj") again to see changes take effect.
It is important to note that no existing criteria (as of now) supports filtering the graph by connectivity; that's not how selections work currently. All existing criteria simply filter edges according to regular expression matches against string properties for a particular edge.
Supported criteria with format REPO-QUALIFIER/BUNDLE-SYMBOLIC-NAME:
[:node X] - shorthand for [:from X :to X]
[:from X] - source of the edge must match X
[:to X] - destination of the edge must match X
Supported criteria with either bundle/package or bundle/service:
[:type X] - type of the edge must match X
Supported criteria with either package name or service interface name:
[:cause X] - string that caused the connection must match X
These are all the commands you can call. All caps refers to what the arguments are, not necessarily what a call looks like, similar to UNIX-flavor of cmdline reference.
View the reference in the CLI
(help)
Initialization tasks
(load-file PATH)
(index-load PATH)
(index-repos REPO_PATH_1 REPO_PATH_2 ...)
Analysis tasks
(list-edges SELECTION OPTIONS)
(draw-graph SELECTION)
Convenience
(open-repos-dir)
(open-tmp-dir)
(open-working-dir)
Most users can get by with a pre-built index. This next section exists for those who can't.
Ensure all repositories have the version of the code checked out that intends to be
analyzed, and that the last build ran was against the repositories in that state. It is important
the data in Maven's /target directories accurately reflect the data to be analyzed. Once that
is done, index the desired repositories for analysis:
osgeyes=> (index-repos ddf)
1 repositories indexed:
{:manifests 433}
The index-repos command supports variadic arguments:
osgeyes=> (index-repos ddf alliance)
2 repositories indexed:
{:manifests 461}
There is only one index in application memory at any given time. After indexing or loading an EDN
file, all analysis commands operate on that index and the previous state is lost. To save a
generated index as an EDN file, run (index-dump) then (open-tmp-dir). The index data is saved
to viz-index.edn in the system's temporary directory.
The CLI app is just a Clojure REPL with a particular namespace serving as the application's set of commands - all commands are just Clojure functions.