Multi-Label Classification

[iR00i]

assuming im working on a Multi-Label classification task, where for each text sample, there are 3+ labels, and for each label there is only two values, in other words you can call it a Multi-Label Binary classification.
For example consider a dataset like this:

id	Text	Label_1	Label_2	Label_3	Label_4
0	Some Arabic text	1	0	0	1
1	Some Arabic text	0	0	0	1
2	Some Arabic text	0	1	0	1
3	Some Arabic text	1	1	0	1

For the task of classification i referred to this colab notebook, i tried to modify the BertDataset class to look like this:

class BERTDataset():
  def __init__(self, text, label_1, label_2, label_3, label_4, model_name, max_len):
    self.text = text

    self.label_1 = label_1
    self.label_2 = label_2
    self.label_3 = label_3
    self.label_4 = label_4

    self.tokenizer_name = model_name
    self.tokenizer = AutoTokenizer.from_pretrained(model_name)
    self.max_len = max_len

  def __len__(self):
    return len(self.text)
  
  def __getitem__(self,item):
    text = str(self.text[item])
    #text =


    input_ids = self.tokenizer.encode(
        text,
        add_special_tokens = True,
        max_length = self.max_len,
        truncation = 'longest_first'
    )

    attention_mask = [1] * len(input_ids)

    #Zero-pad up to the sequence length.
    padding_length = self.max_len - len(input_ids)
    input_ids = input_ids + ([self.tokenizer.pad_token_id] * padding_length)
    attention_mask = attention_mask + ([0] * padding_length)

but when it came to the InputFeatures() function that the method __getitem__ returns, i was unsure how to write it, i referred to the original huggingface doc for InputFeatures(), but there is no mention of it's use for multi-label tasks.

would something like this work?

return InputFeatures( input_ids=input_ids, attention_mask=attention_mask, label = [ label_1, label_2, label_3, label_4] )

and what should be the value of num_labels in:

def model_init():
    return AutoModelForSequenceClassification.from_pretrained(model_name, return_dict=True, num_labels=?)

[WissamAntoun]

Not sure honestly, since this is a multi-label dataset, try 4 and test the output.

You will also need to change the dataset creation script to support multiple labels

[NajlaKSA]

Hello,
Did you manage to find a way to prepare the multi-label dataset for fine-tuning?
I have the same problem and I'm running out of ideas :(

[WissamAntoun]

the labels should be a one-hot encoded vector, and when you create the model you should set problem_type = "multi_label_classification".

def model_init():
    return AutoModelForSequenceClassification.from_pretrained(
        model_name,
        return_dict=True, 
        num_labels=4,
        problem_type = "multi_label_classification"
    )

[ReemAlJunaid]

I did exactly the same but it gave me this error:
TypeError: __init__() got an unexpected keyword argument 'problem_type'

def model_init(): return AutoModelForSequenceClassification.from_pretrained( model_name, return_dict=True, num_labels=7, problem_type="multi_label_classification" )
Any idea how to solve this error?

[ReemAlJunaid]

I think the correct way to add problem_type="multi_label_classification" to the tokenizer and not to the model, it didn't show any error now but I didn't check the results yet.

[AbdullahHasan42]

Greetings;

I have tirelessly looking for any example notebooks on Arabic Multi label Classification to practice on and only found this thread that seems useful. Have you solved this issue and have a sample notebook to study on? @iR00i @ReemAlJunaid @WissamAntoun @NajlaKSA

Thank you.