Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] Fix unrecognized column name delete handler #32429 #32404

Merged
merged 1 commit into from
Mar 19, 2024

Conversation

gavinchou
Copy link
Collaborator

@gavinchou gavinchou commented Mar 18, 2024

  • doris 2.0.x 版本还有遗留的问题, delete handler 将 predicate 表达式转成一个字
    符串持久化了, 然后在使用的时候通过正则提取字段名 比较符号 以及值来还原
    predicate. 这个序列化和正则匹配的过程是非常tricky并且是不鲁棒的.
  • doris 2.0.x 不允许使用中文列名, 否则有delete 语句可能会导致BE coredump
  • 2.1 引入了 sub_predicate_v2 解决了转成字符串再持久化的问题,
    但是为了保证兼容, master分支最好也修一下(需要单独提PR, 因为不一样了)

master PR #32429

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

}
if (!matched) return false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
if (!matched) return false;
if (!matched) { return false;
}

{R"(@a*<<10086)" , true, gen_cond(R"(@a*)" , "<<", R"(10086)" )}, // column ends with *
{R"(*a=b)" , false, gen_cond(R"(*a)" , "=" , R"(b)" )}, // starts with *
{R"(a*a>>WTF(10086))" , true, gen_cond(R"(a*a)" , ">>", R"(WTF(10086))")}, // function
{R"(a-b IS NULL)" , true, gen_cond(R"(a-b)" , "IS", R"(NULL)" )}, // - in col name and test IS NULL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
{R"(a-b IS NULL)" , true, gen_cond(R"(a-b)" , "IS", R"(NULL)" )}, // - in col name and test IS NULL
<< cond_str; {
}

@wm1581066 wm1581066 added dev/2.0.x usercase Important user case type label labels Mar 18, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

{R"(abc>>b)" , true, gen_cond(R"(abc)" , ">>", R"(b)" )}, // normal case
{R"(abc<<b)" , true, gen_cond(R"(abc)" , "<<", R"(b)" )}, // normal case
{R"(abc!='b')" , true, gen_cond(R"(abc)" , "!=", R"(b)" )}, // value surrounded by '
{R"(@a*<<10086)" , true, gen_cond(R"(@a*)" , "<<", R"(10086)" )}, // column ends with *
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
{R"(@a*<<10086)" , true, gen_cond(R"(@a*)" , "<<", R"(10086)" )}, // column ends with *
exp_succ) { EXPECT_EQ(parsed_cond, exp_cond) << " unexpected result, cond_str: " << cond_str;
}

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

{R"(abc>>b)" , true, gen_cond(R"(abc)" , ">>", R"(b)" )}, // normal case
{R"(abc<<b)" , true, gen_cond(R"(abc)" , "<<", R"(b)" )}, // normal case
{R"(abc!='b')" , true, gen_cond(R"(abc)" , "!=", R"(b)" )}, // value surrounded by '
{R"(abc=)" , true, gen_cond(R"(abc)" , "=" , R"()" )}, // missing value, it means not to be parsed succefully, how every it's a ignorable bug
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
{R"(abc=)" , true, gen_cond(R"(abc)" , "=" , R"()" )}, // missing value, it means not to be parsed succefully, how every it's a ignorable bug
exp_succ) { EXPECT_EQ(parsed_cond, exp_cond) << " unexpected result, cond_str: " << cond_str;
}

@xiaokang
Copy link
Contributor

run buildall

@gavinchou gavinchou changed the title Fix unrecognized column name delete handler [fix] Fix unrecognized column name delete handler #32429 Mar 18, 2024
@gavinchou
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8040/21265)
Line Coverage: 29.47% (65610/222611)
Region Coverage: 28.95% (33764/116637)
Branch Coverage: 24.79% (17331/69900)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2df21c78195a0c10e3840f3f55193fc3f7b3d7a5_2df21c78195a0c10e3840f3f55193fc3f7b3d7a5/report/index.html

@gavinchou
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8040/21265)
Line Coverage: 29.47% (65612/222611)
Region Coverage: 28.95% (33768/116637)
Branch Coverage: 24.80% (17333/69900)
Coverage Report: http://coverage.selectdb-in.cc/coverage/eb82e352cc6c4aff1d551231dd9791931a817e78_eb82e352cc6c4aff1d551231dd9791931a817e78/report/index.html

@wm1581066 wm1581066 removed usercase Important user case type label dev/2.0.x labels Mar 19, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 19, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit d992f08 into apache:branch-2.0 Mar 19, 2024
25 of 28 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. kind/test reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants