Log a warning, if the process method of DoFn returns None #28159

timgrein · 2023-08-25T14:01:44Z

Addresses #28061

This PR adds a warning log, if the process method of DoFn returns None by checking the following cases (doesn't check inner functions):

process contains return None (explicit return None)
process contains only return (implicit return None)
process doesn't contain yield neither does it contain return (implicit return None)

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

…cit and implicit cases).

codecov · 2023-08-25T14:29:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6bfb469) 72.31% compared to head (447a7f3) 72.30%.
Report is 1493 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #28159      +/-   ##
==========================================
- Coverage   72.31%   72.30%   -0.02%     
==========================================
  Files         678      678              
  Lines       99802    99815      +13     
==========================================
- Hits        72171    72170       -1     
- Misses      26069    26083      +14     
  Partials     1562     1562

Flag	Coverage Δ
python	`82.85% <100.00%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2023-08-25T15:05:54Z

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

github-actions · 2023-08-25T23:07:19Z

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @AnandInguva for label python.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

AnandInguva · 2023-08-26T07:11:29Z

sdks/python/apache_beam/transforms/core.py

+    return_none_warning = "No iterator is returned by " \
+                          "the process method in DoFn."


Suggested change

return_none_warning = "No iterator is returned by " \

"the process method in DoFn."

return_none_warning = ("No iterator is returned by "

"the process method in DoFn.")

This warning could be meaningful if it can mention the faulty DoFn. Can we add the name of the DoFn?

Added the class name using fn.__self__.__class__ 👍

AnandInguva · 2023-08-28T08:46:31Z

The pydocs are getting this error 04:42:45 WARNING:apache_beam.transforms.core:('No iterator is returned by the process method in %s.', <class 'apache_beam.transforms.core.WindowInto.WindowIntoFn'>) for this DoFn.

beam/sdks/python/apache_beam/transforms/core.py

Line 3519 in e26735d

class WindowIntoFn(DoFn):

Can you check it why since the DoFn is returning yield?

github-actions · 2023-09-04T12:13:55Z

Reminder, please take a look at this pr: @AnandInguva

timgrein · 2023-09-05T09:56:57Z

The pydocs are getting this error 04:42:45 WARNING:apache_beam.transforms.core:('No iterator is returned by the process method in %s.', <class 'apache_beam.transforms.core.WindowInto.WindowIntoFn'>) for this DoFn.

beam/sdks/python/apache_beam/transforms/core.py

Line 3519 in e26735d

class WindowIntoFn(DoFn):

Can you check it why since the DoFn is returning yield?

Taking a look today or tomorrow 👍

github-actions · 2023-09-12T12:14:24Z

Reminder, please take a look at this pr: @AnandInguva

github-actions · 2023-09-15T12:14:00Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @damccorm for label python.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

damccorm · 2023-10-27T14:46:22Z

Hey @timgrein any update on this one?

damccorm · 2023-12-12T13:53:53Z

@timgrein any updates here?

github-actions · 2024-02-11T12:38:05Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions · 2024-04-13T12:37:29Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

codecov-commenter · 2024-04-13T12:37:41Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.30%. Comparing base (6bfb469) to head (447a7f3).
Report is 1959 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #28159      +/-   ##
==========================================
- Coverage   72.31%   72.30%   -0.02%     
==========================================
  Files         678      678              
  Lines       99802    99815      +13     
==========================================
- Hits        72171    72170       -1     
- Misses      26069    26083      +14     
  Partials     1562     1562

Flag	Coverage Δ
python	`82.85% <100.00%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Log warning, if the process method of DoFn returns None (handle expli…

2ac4f39

…cit and implicit cases).

github-actions bot added the python label Aug 25, 2023

Fix formatting

8ed5e04

github-actions bot added the Next Action: Reviewers label Aug 25, 2023

AnandInguva reviewed Aug 26, 2023

View reviewed changes

timgrein added 3 commits August 26, 2023 10:26

Add class name to warning

65ba9cc

Add class name to warning using fn.__self__.__class__

992dbf8

Format

447a7f3

AnandInguva approved these changes Aug 26, 2023

View reviewed changes

github-actions bot added the slow-review label Sep 4, 2023

AnandInguva added Next Action: Author and removed Next Action: Reviewers slow-review labels Sep 4, 2023

github-actions bot added Next Action: Reviewers and removed Next Action: Author labels Sep 5, 2023

github-actions bot added the slow-review label Sep 12, 2023

github-actions bot removed the slow-review label Sep 15, 2023

damccorm added Next Action: Author and removed Next Action: Reviewers labels Sep 15, 2023

damccorm mentioned this pull request Oct 27, 2023

Update core.py [If funnction do not return any thing then warning will be pop up] #28680

Closed

3 tasks

github-actions bot added stale and removed stale labels Feb 11, 2024

github-actions bot added the stale label Apr 13, 2024

github-actions bot removed the stale label Apr 14, 2024

timgrein closed this May 14, 2024

DKER2 mentioned this pull request Sep 24, 2024

Log warning if process function return None #32545

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Log a warning, if the process method of DoFn returns None #28159

Log a warning, if the process method of DoFn returns None #28159

timgrein commented Aug 25, 2023 •

edited

Loading

codecov bot commented Aug 25, 2023 •

edited

Loading

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 25, 2023

AnandInguva Aug 26, 2023

AnandInguva Aug 26, 2023

timgrein Aug 26, 2023 •

edited

Loading

AnandInguva commented Aug 28, 2023

github-actions bot commented Sep 4, 2023

timgrein commented Sep 5, 2023

github-actions bot commented Sep 12, 2023

github-actions bot commented Sep 15, 2023

damccorm commented Oct 27, 2023

damccorm commented Dec 12, 2023

github-actions bot commented Feb 11, 2024

github-actions bot commented Apr 13, 2024

codecov-commenter commented Apr 13, 2024

		return_none_warning = "No iterator is returned by " \
		"the process method in DoFn."

Log a warning, if the process method of DoFn returns None #28159

Log a warning, if the process method of DoFn returns None #28159

Conversation

timgrein commented Aug 25, 2023 • edited Loading

GitHub Actions Tests Status (on master branch)

codecov bot commented Aug 25, 2023 • edited Loading

Codecov Report

github-actions bot commented Aug 25, 2023

github-actions bot commented Aug 25, 2023

AnandInguva Aug 26, 2023

Choose a reason for hiding this comment

AnandInguva Aug 26, 2023

Choose a reason for hiding this comment

timgrein Aug 26, 2023 • edited Loading

Choose a reason for hiding this comment

AnandInguva commented Aug 28, 2023

github-actions bot commented Sep 4, 2023

timgrein commented Sep 5, 2023

github-actions bot commented Sep 12, 2023

github-actions bot commented Sep 15, 2023

damccorm commented Oct 27, 2023

damccorm commented Dec 12, 2023

github-actions bot commented Feb 11, 2024

github-actions bot commented Apr 13, 2024

codecov-commenter commented Apr 13, 2024

Codecov Report

timgrein commented Aug 25, 2023 •

edited

Loading

codecov bot commented Aug 25, 2023 •

edited

Loading

timgrein Aug 26, 2023 •

edited

Loading