Skip to content

[Fortran/gfortran] Disable flaky test random_init_2.f90 #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 22, 2024

Conversation

luporl
Copy link
Contributor

@luporl luporl commented Aug 20, 2024

The random_init_2.f90 test is flaky and may fail sometimes, even
when no Fortran/Flang changes are made:

Looking at the test source, it seems there is no guarantee that 2
distinct random numbers will never collide. This is made worse by
multiplying them by 1e6 and converting them to integer, which could
make distinct but close enough single-precision floating-point
numbers give the same result and fail the test.

The random_init_2.f90 test is flaky and may fail sometimes, even
when no Fortran/Flang changes are made:
- https://lab.llvm.org/buildbot/#/builders/143/builds/1236
- https://lab.llvm.org/buildbot/#/builders/17/builds/2058

Looking at the test source, it seems there is no guarantee that 2
distinct random numbers will never collide. This is made worse by
multiplying them by 1e6 and converting them to integer, which could
make distinct but close enough single-precision floating-point
numbers give the same result and fail the test.
Copy link
Contributor

@tarunprabhu tarunprabhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this to get the buildbots going ...

Do you know why this was never triggered until just now? I have never seen this failure in any of my local testing, and none of the buildbots have complained about this in the past either.

@luporl
Copy link
Contributor Author

luporl commented Aug 20, 2024

Thanks for the review!

Do you know why this was never triggered until just now? I have never seen this failure in any of my local testing, and none of the buildbots have complained about this in the past either.

Actually, this test didn't start failing now. It's quite uncommon, but it fails sometimes.
https://lab.llvm.org/buildbot/#/builders/17 failed today, in a single build, and passed in the next one.
The failure in https://lab.llvm.org/buildbot/#/builders/143 is from 17 days ago, also in a single build, with no changes related to Flang.

I suspect it doesn't happen often because most times the generated random numbers, converted to integers, are different. The failure may also go unnoticed, as it usually disappears in the next build.

@tarunprabhu
Copy link
Contributor

I see. Thanks for the clarification. There are other random_* tests. Do you know if those need to be disabled as well?

@luporl
Copy link
Contributor Author

luporl commented Aug 21, 2024

I see. Thanks for the clarification. There are other random_* tests. Do you know if those need to be disabled as well?

I've checked the random_init* ones and they should be fine:

  • random_init_1.f90 - expects a compile error
  • random_init.f90 - should be fine: generates 5 reals and expects at least 1 to be different than 0.0
  • remaining random_init_*.f90 - already disabled: unimplemented: intrinsic: this_image

I haven't checked the other random_* tests, but I don't remember seeing any failures with those.

@tarunprabhu
Copy link
Contributor

OK. Thanks for checking the other tests, Leandro. We might need to revisit the other random_init_ tests when the unimplemented: issue is fixed but it should be fine to deal with that later.

Feel free to merge this whenever you see fit.

Copy link
Contributor

@DavidSpickett DavidSpickett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work investigating Leandro, thanks!

@DavidSpickett
Copy link
Contributor

(gonna go ahead and merge this myself)

@DavidSpickett DavidSpickett merged commit e93f828 into llvm:main Aug 22, 2024
1 check passed
@luporl luporl deleted the luporl-disable-random-init2 branch August 22, 2024 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants