no result

[da-123-snake]

Hi,
I am trying to run mBERT on my machine but here comes an error,
cmd@zkti:/mnt/disk1/cmd/mBERT$ ./mBERT.sh -in=/mnt/disk1/cmd/defects4j_fixed/Chart/Chart_1_fixed/source/org/jfree/chart/annotations/AbstractAnnotation.java -out=/mnt/disk1/cmd/mBERTm/Chart/Chart1 -N=5 -l=176
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for SnT:mBERT:jar:1.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-compiler-plugin is missing. @ line 12, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] -----------------------------< SnT:mBERT >------------------------------
[INFO] Building mBERT 1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- exec-maven-plugin:3.2.0:java (default-cli) @ mBERT ---
Input: /mnt/disk1/cmd/defects4j_fixed/Chart/Chart_1_fixed/source/org/jfree/chart/annotations/AbstractAnnotation.java
Output: /mnt/disk1/cmd/mBERTm/Chart/Chart1
Settings{CODEBERT_TIMEOUT=60, MAX_NUM_OF_MUTANTS=5}
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Here is the standard error of the command (if any):

Some weights of the model checkpoint at /mnt/disk1/cmd/mBERT/pre-trained/codebert-base-mlm were not used when initializing RobertaForMaskedLM: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']

• This IS expected if you are initializing RobertaForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing RobertaForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Here is the standard error of the command (if any):

Some weights of the model checkpoint at /mnt/disk1/cmd/mBERT/pre-trained/codebert-base-mlm were not used when initializing RobertaForMaskedLM: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']

• This IS expected if you are initializing RobertaForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing RobertaForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Saving mutants...
  Settings{CODEBERT_TIMEOUT=60, MAX_NUM_OF_MUTANTS=5}
  [INFO] ------------------------------------------------------------------------
  [INFO] BUILD SUCCESS
  [INFO] ------------------------------------------------------------------------
  [INFO] Total time: 01:00 min
  [INFO] Finished at: 2024-02-23T15:53:54+08:00
  [INFO] ------------------------------------------------------------------------

BUILD SUCCESS! But there has no result!

[da-123-snake]

Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import torch

print(torch.version)
2.2.1+cu121

import transformers
_)>>>
print(transformers.version)
4.38.1
print(torch.cuda.is_available())
True

cmd@zkti:/mnt/disk1/cmd$ mvn -v
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: /home/cmd/apache-maven-3.8.8
Java version: 1.8.0_391, vendor: Oracle Corporation, runtime: /home/cmd/jdk1.8.0_391/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-52-generic", arch: "amd64", family: "unix"

[michaelkonstantinou]

Hey @da-123-snake ,

I also faced some similar issues and this is what I did to make it work for me:

• I used Java 11
• Python 3.9
• When installing transformers, I found the solution here very helpful as my conda environment couldn't detect the correct transformers version: ImportError: cannot import name 'pipeline' from 'transformers' (unknown location) huggingface/transformers#10277 (comment)

I am not sure whether the problem is in Python or in Java, but you can try this out by running the following command:
python3 run-codebert.py "int <mask> = b;"

If you don't get the following results then most likely this is a problem with python. Otherwise, it might be from Java

    {'score': 0.23396340012550354, 'token': 740, 'token_str': 'c', 'sequence': 'int c= b;'}

    {'score': 0.05450829118490219, 'token': 939, 'token_str': 'i', 'sequence': 'int i= b;'}

    {'score': 0.05004948750138283, 'token': 741, 'token_str': 'b', 'sequence': 'int b= b;'}

    {'score': 0.04164685681462288, 'token': 10, 'token_str': 'a', 'sequence': 'int a= b;'}

    {'score': 0.023635799065232277, 'token': 181, 'token_str': 'p', 'sequence': 'int p= b;'}

Hope it helps. If not, perhaps you can provide more information in case I missed anything

[rdegiovanni]

Hello @da-123-snake,
Sorry for my late reply. Thanks @michaelkonstantinou for your comment.

@da-123-snake I was wondering if you managed to solve the issue.
In my case, I was using Java 11, torch:1.10.1 and transformers:4.15.0.
But if you need to attach to Java 1.8, you can check this issue: #4 (comment)

Please let us know if you have any further questions.

Best,
Renzo