Why is reduce_outliers creating new topics when topics with same representations already exists?

[farazk86]

My understanding of reduce_outliers is that it will try to reduce the topics by labelling them to a non-outlier topic, as mentioned here: https://maartengr.github.io/BERTopic/getting_started/outlier_reduction/outlier_reduction.html

I have a dataset that is producing 611 outlier documents. and am using BERTopic as below:

representation_model = KeyBERTInspired()


umap = UMAP(n_neighbors=15,
            n_components=5,
            min_dist=0.0,
            metric='cosine',
            low_memory=False,
            random_state=1337)


# now fit the topic model
topic_model = BERTopic(language="english",
                       representation_model = representation_model,
                       calculate_probabilities=True,
                       n_gram_range=(1, 2), verbose=True,
                       min_topic_size=10,   
                       umap_model=umap,
                       )

topics, probs = topic_model.fit_transform(docs)

I notice that quite a few topics can be merged and visualize these using topic_model.visualize_hierarchy. And since merging can only be done before topic reduction I merge some of my topics and update my model as below:

topics_to_merge = [
    [49, 13, 21],   # accommodation
    [45, 19, 34, 22], # tours, tour talks, guided tours
    [29, 10, 37, 44], # location, nice location, beautiful location 
]

topic_model.merge_topics(docs=docs, topics_to_merge=topics_to_merge)

And then finally I go on to topic reduction using:

new_topics = topic_model.reduce_outliers(docs, topics, strategy="distributions")
# update the model
topic_model.update_topics(docs, topics=new_topics)
# to see the results
freq = topic_model.get_topic_info()
freq

But this process re-labels all of the 611 outliers to current existing topics or create new topics. Problem is that it creates 17 additional topics and most of them are not necessary, I manually viewed the comments of these new topics and they can be easily placed in pre-existing topic labels. For example 3 of the new created topics were directly related to the location topic that I already merged before this. and only a few new topics made sense.

And I cannot merge these topics again as its not possible to merge after reduce_outliers as per the documentation.

I know I can set a threshold so as its not as aggressive i.e. threshold=0.1 leaves 90 outliers while re-labelling the rest, but new topics are still created when similar topics exist. i.e. similar to location 2 new topics were created that discuss accommodation however 2 topics (or 1 topic after the merge) already exists.

Is this how its supposed to happen or am I doing something wrong?

Update:

One thing I noticed is that no matter what strategy I use or which threshold I use, the reduce_topics always brings the number of topics back to what it was before the merge_topics procedure. i.e. the topic_model.fit_transform(docs) would give me 53 total number of topics. After the merge_topics I would come down to 33 topics.

But after the reduce_outluers I would come back to 53 topics, no matter what strategy or threshold I use for reduction. And these unnecessary topics would be created to come back to the original number before merge. Is this a bug needed reporting in issues or am I doing something wrong?

[farazk86]

I believe I figured it out, It was my own mistake :( and the hint was the fact it was giving me the same number of topics that were pre-merge.

I just needed to update my topic list as mentioned here: https://maartengr.github.io/BERTopic/getting_started/topicreduction/topicreduction.html#topic-reduction-after-training

after updating the topics, after merging I am getting the correct results and no more unnecessary topics created :)