Skip to content

[CALCITE-3458] Remove desc in AbstractRelNode #1546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

[CALCITE-3458] Remove desc in AbstractRelNode #1546

wants to merge 3 commits into from

Conversation

hsyuan
Copy link
Member

@hsyuan hsyuan commented Oct 30, 2019

If the query is super large, e.g. contains tens of thousands of nodes or
expressions, the RelNode digest and desc become very large. The content of desc
and digest are almost the same, except that desc consists of id plus digest. So
remove desc, just use id + digest to produce description.
https://issues.apache.org/jira/browse/CALCITE-3458

return desc;
StringBuilder sb = new StringBuilder("rel#");
sb = sb.append(id).append(':').append(digest);
return sb.toString();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is called before recomputeDigest(), we will get a different description string than the original implementation. Is that you want?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. That is expected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we can add a benchmark test for the toString with/without the fix.

If the query is super large, e.g. contains tens of thousands of nodes or
expressions, the RelNode digest and desc become very large. The content of desc
and digest are almost the same, except that desc consists of id plus digest. So
remove desc, just use id + digest to produce description.
// Same with {@link rel#getDescription()}, we don't call it explicitly
// in order to avoid string copy when generating description.
buf.append("rel#").append(rels[i].getId())
.append(':').append(rels[i].getDigest());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we abstract this description as a tool method and reuse that ? Something like RelOptUtil#generatesDescription(StringBuilder builder, int relNodeID, String digest)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I have addressed your comments.

@chunweilei
Copy link
Contributor

LGTM

@hsyuan hsyuan closed this in e2c8e68 Nov 5, 2019
@hsyuan hsyuan deleted the CALCITE-3458 branch November 5, 2019 04:23
XuQianJin-Stars pushed a commit to XuQianJin-Stars/calcite that referenced this pull request Nov 17, 2019
If the query is super large, e.g. contains tens of thousands of nodes or
expressions, the RelNode digest and desc become very large. The content of desc
and digest are almost the same, except that desc consists of id plus digest. So
remove desc, just use id + digest to produce description.

Also deprecated method rel.getDescription(), use rel.toString() instead.

Close apache#1546
XuQianJin-Stars pushed a commit to XuQianJin-Stars/calcite that referenced this pull request Nov 20, 2019
If the query is super large, e.g. contains tens of thousands of nodes or
expressions, the RelNode digest and desc become very large. The content of desc
and digest are almost the same, except that desc consists of id plus digest. So
remove desc, just use id + digest to produce description.

Also deprecated method rel.getDescription(), use rel.toString() instead.

Close apache#1546
XuQianJin-Stars pushed a commit to XuQianJin-Stars/calcite that referenced this pull request Nov 20, 2019
If the query is super large, e.g. contains tens of thousands of nodes or
expressions, the RelNode digest and desc become very large. The content of desc
and digest are almost the same, except that desc consists of id plus digest. So
remove desc, just use id + digest to produce description.

Also deprecated method rel.getDescription(), use rel.toString() instead.

Close apache#1546
wangxlong pushed a commit to wangxlong/calcite that referenced this pull request Feb 13, 2020
If the query is super large, e.g. contains tens of thousands of nodes or
expressions, the RelNode digest and desc become very large. The content of desc
and digest are almost the same, except that desc consists of id plus digest. So
remove desc, just use id + digest to produce description.

Also deprecated method rel.getDescription(), use rel.toString() instead.

Close apache#1546
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants