Skip to content

Commit

Permalink
Args description refined, papers references added
Browse files Browse the repository at this point in the history
  • Loading branch information
luav committed Apr 7, 2017
1 parent 243c10a commit 6edd8f1
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 8 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Extremely fast evaluation of the extrinsic clustering measures: *various F1 measures (including F1-Score) for overlapping multi-resolution clusterings with unequal node base (and optional node base synchronization)* and standard NMI for non-overlapping clustering on a single resolution.
`xmeasures` evaluates F1 and NMI for collections of hundreds thousands clusters withing a dozen seconds on an ordinary laptop using a single CPU core. The computational time is O(N) unlike O(N*C) of the existing state of the art implementations, where N is the number of nodes in the network and C is the number of clusters.
`xmeasures` is one of the utilities designed for the [PyCaBeM](https://github.com/eXascaleInfolab/PyCABeM) clustering benchmark to evaluate clustering of large networks.
A paper about the implemented F1 measures (F1p is much more discriminative than the standard F1-Score, NMI measures and their applicability is being written now and the reference will be specified before Summer, 2017.
A paper about the implemented F1 measures (F1p is much more discriminative than the standard [Average F1-Score](https://cs.stanford.edu/people/jure/pubs/bigclam-wsdm13.pdf)), [NMI measures](www.jmlr.org/papers/volume11/vinh10a/vinh10a.pdf) and their applicability is being written now and the reference will be specified before Summer, 2017.
> Standard NMI is implemented considering overlapping and multi-resolution clustering only to demonstrate non-applicability of the standard NMI for such cases, it yield unfair results. See [GenConvNMI](https://github.com/eXascaleInfolab/GenConvNMI) for the fair generalized NMI evaluation.
Author (c) Artem Lutov <[email protected]>
Expand Down
10 changes: 5 additions & 5 deletions args.ggo
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Configuration file for the automatic generation of the input options parsing

package "xmeasures"
version "3.0.1"
version "3.0.2"

purpose "Extrinsic measures evaluation: F1 (prob, harm and score) for overlapping\
multi-resolution clusterings with possible unequal node base and standard NMI for\
Expand All @@ -10,10 +10,10 @@ purpose "Extrinsic measures evaluation: F1 (prob, harm and score) for overlappin
description "Extrinsic measures are evaluated, i.e. clustering (collection of\
clusters) is compared to another collection, which is typically the ground-truth.
Evaluating measures are:
- F1 - various F1 measures of the Greatest (Max) Match including the standard\
- F1 - various F1 measures of the Greatest (Max) Match including the Average\
F1-Score with optional weighting;
- NMI - Normalized Mutual Information, normalized by either max or also avg\
and min information content denominators.
- NMI - Normalized Mutual Information, normalized by either max or also sqrt,\
avg and min information content denominators.
ATTENTION: this is standard NMI, which should be used ONLY for the HARD\
partitioning evaluation (non-overlapping clustering on a single resolution).\
it penalizes overlapping and multi-resolution structures.
Expand Down Expand Up @@ -70,7 +70,7 @@ args "--default-optional --unamed-opts=clusterings" # Set opional opions, allo


# = Changelog =
# v3.0.+ - NMI_sqrt added
# v3.0.2 - NMI_sqrt added
# v3.0 - Command line interface changed for F1
# - Standard F1-Score added
# - Option for the detailed results output added
Expand Down
2 changes: 1 addition & 1 deletion autogen/cmdline.c
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const char *gengetopt_args_info_usage = "Usage: xmeasures [OPTIONS] clustering1

const char *gengetopt_args_info_versiontext = "";

const char *gengetopt_args_info_description = "Extrinsic measures are evaluated, i.e. clustering (collection of clusters) is\ncompared to another collection, which is typically the ground-truth.\nEvaluating measures are:\n - F1 - various F1 measures of the Greatest (Max) Match including the\nstandard F1-Score with optional weighting;\n - NMI - Normalized Mutual Information, normalized by either max or also avg\nand min information content denominators.\nATTENTION: this is standard NMI, which should be used ONLY for the HARD\npartitioning evaluation (non-overlapping clustering on a single resolution).\nit penalizes overlapping and multi-resolution structures.\nNOTE: unequal node base in the clusterings is allowed, it penalizes the match.\nUse [GenConvNMI](https://github.com/eXascaleInfolab/GenConvNMI) for arbitrary\ncollections evaluation.\n";
const char *gengetopt_args_info_description = "Extrinsic measures are evaluated, i.e. clustering (collection of clusters) is\ncompared to another collection, which is typically the ground-truth.\nEvaluating measures are:\n - F1 - various F1 measures of the Greatest (Max) Match including the Average\nF1-Score with optional weighting;\n - NMI - Normalized Mutual Information, normalized by either max or also\nsqrt, avg and min information content denominators.\nATTENTION: this is standard NMI, which should be used ONLY for the HARD\npartitioning evaluation (non-overlapping clustering on a single resolution).\nit penalizes overlapping and multi-resolution structures.\nNOTE: unequal node base in the clusterings is allowed, it penalizes the match.\nUse [GenConvNMI](https://github.com/eXascaleInfolab/GenConvNMI) for arbitrary\ncollections evaluation.\n";

const char *gengetopt_args_info_help[] = {
" -h, --help Print help and exit",
Expand Down
2 changes: 1 addition & 1 deletion autogen/cmdline.h
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ extern "C" {

#ifndef CMDLINE_PARSER_VERSION
/** @brief the program version */
#define CMDLINE_PARSER_VERSION "3.0.1"
#define CMDLINE_PARSER_VERSION "3.0.2"
#endif

enum enum_f1 { f1__NULL = -1, f1_arg_partprob = 0, f1_arg_harmonic, f1_arg_standard };
Expand Down

0 comments on commit 6edd8f1

Please sign in to comment.