ZOOKEEPER-4880: Generate comments from zookeeper.jute into code.

[luozongle01]

Generate comments from zookeeper.jute into code: https://issues.apache.org/jira/browse/ZOOKEEPER-4880

Brief Description

Support generating end-of-line comments and class comments in zookeeper.jute into code.

How to test this change

1. By comparing the zookeeper-jute/target/generated-sources/java/org/apache/zookeeper and zookeeper-client/zookeeper-client-c/generated directories before and after the modification, there are no other differences except the comments.

2. When the test class has multiple lines of comments, the comments can be generated normally.

3. Testing vector type properties can generate comments normally.

4. Testing class type properties can generate comments normally

5. Testing comments that do not meet expectations will not generate comments and will not affect the generation of normal code.

[luozongle01]

@kezhuw Could you help me take a look? Thanks.

[kezhuw]

Hi @luozongle01, thank you for your contribution!

Given the fact that jute and all targeting languages support both // and multi-line comments, I suggest we output whatever comments captured in origin sharp. With help of beginLine, beginColumn, endLine and endColumn of Token, I think we can reproduce comments in generated code with indentation adjusted.

[kezhuw]

The capture does not shaped well for directly usage. I tasted the following capture, and it captured a complete line comment. This way we can output whatever it captured without post work.

SPECIAL_TOKEN :
{
  <SINGLE_LINE_COMMENT: "//" (~["\n","\r"])* ("\n"|"\r"|"\r\n")>
}

See also https://javacc.github.io/javacc/tutorials/token-manager.html

[luozongle01]

Thank you very much for your comment. I will make some further modifications.😆

[luozongle01]

The capture does not shaped well for directly usage. I tasted the following capture, and it captured a complete line comment. This way we can output whatever it captured without post work.

SPECIAL_TOKEN :
{
  <SINGLE_LINE_COMMENT: "//" (~["\n","\r"])* ("\n"|"\r"|"\r\n")>
}

See also https://javacc.github.io/javacc/tutorials/token-manager.html

It seems like there's no need to change this now, because I get all the comments，then I use and distinguished them by line numbe. 😂😂

[luozongle01]

Hi, @kezhuw, I made some modifications, thank you for your review last time. I feel much better now than last time.
I hope you can help me look again.

Currently, multi-line comments and single-line comments are supported.

I put JField or JRecord into a TreeSet, so that I can get the line or column information of the previous or next element. According to the beginLine, beginColumn, endLine, endColumn information, I determine the range of my comment area, and then format the output in line number order.

Here is my testing process.

1. Verify that the modification only affects the comments and does not affect the normal code.

2. When verifying that each JField is on its own line, multi-line comments and single-line comments are generated normally.

3. If the attributes are on the same line, the comment should belong to the previous Field.

[kezhuw]

@luozongle01 Thank you for your work!

It might be late, I am sorry for this, but I think we probably should align on how comments from jute should be formatted in generated sources ? I see two approaches.

1. Keep it in origin format.
2. Transfer comments from one format to another.

I saw this pr is going on the second approach. I am good to unify the generated comment format. But I think it could be challenge, as we should escape between two formats which could make generated comments strange/inaccurate.

    // /**
    //  * Some dead comments.
    //  */
    //
    // Things changed. We are going ..

Currently, the generated comments blindly strip some special strings. I don't think it is a good thing.

I suggest we keep it simple and don't touch comment content, this is also a safe path. If we want javadoc style comments, I think it is ok to convert them manually.

[kezhuw]

I do think people will want to maintain this kind of code.

[kezhuw]

s/do/don't/

[luozongle01]

I do think people will want to maintain this kind of code.
I'll amend that and I won't touch the comments so they're not needed here.

[kezhuw]

The generated comments has no empty line which is not good.

[luozongle01]

@luozongle01 Thank you for your work!

It might be late, I am sorry for this, but I think we probably should align on how comments from jute should be formatted in generated sources ? I see two approaches.

1. Keep it in origin format.
2. Transfer comments from one format to another.

I saw this pr is going on the second approach. I am good to unify the generated comment format. But I think it could be challenge, as we should escape between two formats which could make generated comments strange/inaccurate.

    // /**
    //  * Some dead comments.
    //  */
    //
    // Things changed. We are going ..

Currently, the generated comments blindly strip some special strings. I don't think it is a good thing.

I suggest we keep it simple and don't touch comment content, this is also a safe path. If we want javadoc style comments, I think it is ok to convert them manually.

@kezhuw Thank you very much for your review. I really didn't consider that 😂😂. I will keep the original format output.

However, for the end-of-line comments, should I continue to keep them at the end of the line or is it better to move them above the code? 🤔

[kezhuw]

for the end-of-line comments, should I continue to keep them at the end of the line or is it better to move them above the code?

I prefer to move them above the code.

[luozongle01]

for the end-of-line comments, should I continue to keep them at the end of the line or is it better to move them above the code?

I prefer to move them above the code.

Thanks, I thought so too, I'll modify the code.

[luozongle01]

Hi @kezhuw , I made some changes, could you take a look at it for me?

I won't touch the annotation content now.

But I still formatted it a little bit here, because for example, field needs a few spaces, class does not need spaces, and the number of spaces in java and c is also different.

  private static String getComments(String space, List<String> commentList) {
      if (commentList == null || commentList.isEmpty()) {
          return "";
      }

      String comments = String.join("", commentList);

      StringBuilder formatBuilder = new StringBuilder();
      for (String s : comments.split("\r?\n")) {
          formatBuilder.append("\n")
                  .append(space)
                  .append(s.replaceAll("^[ \t]+", ""));
      }

      return formatBuilder.append("\n").toString();
  }

Now the comment is like this