Facing problem with explaining test instances

[sarangs-ntnu]

RuntimeError Traceback (most recent call last)
~/anaconda3/envs/ntnu_meticos/lib/python3.7/site-packages/omnixai/explainers/base.py in explain(self, X, params, run_predict)
283 param = params.get(name, {})
--> 284 explanations[name] = self.explainers[name].explain(X=X, **param)
285 except Exception as e:

~/anaconda3/envs/ntnu_meticos/lib/python3.7/site-packages/omnixai/explainers/tabular/counterfactual/mace/mace.py in explain(self, X, y, max_number_examples, **kwargs)
124 # Get candidate features
--> 125 candidates, indices = self.recall.get_cf_features(x, desired_label)
126

~/anaconda3/envs/ntnu_meticos/lib/python3.7/site-packages/omnixai/explainers/tabular/counterfactual/mace/retrieval.py in get_cf_features(self, instance, desired_label)
186 x = instance.to_pd(copy=False)
--> 187 y, indices = self.get_nn_samples(instance, desired_label)
188 cate_candidates, cont_candidates = {}, {}

~/anaconda3/envs/ntnu_meticos/lib/python3.7/site-packages/omnixai/explainers/tabular/counterfactual/mace/retrieval.py in get_nn_samples(self, instance, desired_label)
173 )
--> 174 indices = self._knn_query(query, desired_label, self.num_neighbors)[0]
175 y = self.subset.iloc(indices).to_pd(copy=False)

~/anaconda3/envs/ntnu_meticos/lib/python3.7/site-packages/omnixai/explainers/tabular/counterfactual/mace/retrieval.py in _knn_query(self, x, label, k)
121 """
--> 122 indices, distances = self.knn_models[label].knn_query(x, k=k)
123 neighbors = [[idx[i] for i in range(len(idx)) if dists[i] > 0] for idx, dists in zip(indices, distances)]

RuntimeError: Cannot return the results in a contigious 2D array. Probably ef or M is too small

During handling of the above exception, another exception occurred:

RuntimeError Traceback (most recent call last)
/tmp/ipykernel_3818713/1430780571.py in
14 # Generate explanations
15 test_instances = test_data[0:7]
---> 16 local_explanations = explainers.explain(X=test_instances)
17 global_explanations = explainers.explain_global(
18 params={"pdp": {"features": []}}

~/anaconda3/envs/ntnu_meticos/lib/python3.7/site-packages/omnixai/explainers/base.py in explain(self, X, params, run_predict)
284 explanations[name] = self.explainers[name].explain(X=X, **param)
285 except Exception as e:
--> 286 raise type(e)(f"Explainer {name} -- {str(e)}")
287 return explanations
288

RuntimeError: Explainer mace -- Cannot return the results in a contigious 2D array. Probably ef or M is too small

[yangwenz]

If the other explainers can generate reasonable results, this error probably comes from that the dataset for initialization doesn't contain the desired class/label (some classes are missing in this dataset). Therefore, when MACE calls KNN search for finding nearest examples in the desired class, KNN cannot find them. So please check if the dataset for initialization contains all the classes, and feel free to contact us if there are still issues.

[rodrigocostacamargos]

Look at this line: indices, distances = self.knn_models[label].knn_query(x, k=k).

If x.size < k, then you get this error.

You should look at your dataset size because x is calculated from there. Alternatively, you can decrease k from 30 to a small number less than x.size

class CFRetrieval:
"""
A KNN-based method for finding the features that may change
the predicted label for a query instance.
"""

def __init__(
    self,
    training_data: Tabular,
    predict_function: Callable,
    ignored_features: List = None,
    feature_column_top_k: int = -1,
    feature_value_top_k: int = 3,
    num_cont_bins: int = 10,
    num_neighbors: int = 5, # CHANGE HERE!
    hnsw_ef_construction: int = 200,
    hnsw_m: int = 30,
    hnsw_ef: int = 50,
    **kwargs
):