Skip to content

refactor: Added tracking for disabled CI tests from DLFW integration #738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 24, 2021
Merged

refactor: Added tracking for disabled CI tests from DLFW integration #738

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e34c417
refactor: Added tracking for disabled CI tests from DLFW integration
Nov 23, 2021
296e98a
fix: Added test case names
Nov 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions tests/NOTES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Tests

## Lists of Tests disabled for DLFW CI

Following tests have been disabled due to flaky output issues with DLFW integration CI. These test cases will remain disabled until fixed for integration with DLFW.

```
1. MobileNet_v2 model TestCompile test (test_api.py)

2. ATenGELUConvertsCorrectly test (test_activation.cpp)

3. LSTM tests (test_lstm_cell.cpp)
a. ATenGRUCellConvertsCorrectlyWithBiasCheckHidden
b. ATenGRUCellConvertsCorrectlyWithoutBiasCheckHidden
c. ATenLSTMCellConvertsCorrectlyWithBiasCheckHidden
d. ATenLSTMCellConvertsCorrectlyWithBiasCheckCell
e. ATenLSTMCellConvertsCorrectlyWithoutBiasCheckHidden
f. ATenLSTMCellConvertsCorrectlyWithoutBiasCheckCell

4. Softmax tests (test_softmax.cpp)
a. ATenSoftmax1DConvertsCorrectly
b. ATenSoftmaxNDConvertsCorrectlySub3DIndex
c. ATenSoftmaxNDConvertsCorrectlyAbove3DIndex
d. ATenSoftmaxNDConvertsCorrectlyNegtiveOneIndex
e. ATenSoftmaxNDConvertsCorrectlyNegtiveIndex

5. Partitioning tests (test_fallback_graph_output.cpp)
a. ComputeResNet50FallbackGraphCorrectly
b. ComputeMobileNetFallbackGraphCorrectly
c. ComputeResNet50HalfFallbackGraphCorrectly

6. RuntimeThreadSafety tests (test_runtime_thread_safety.cpp)

7. ModuleAsEngine tests (test_modules_as_engines.cpp)
a. ModuleAsEngineIsClose
b. ModuleToEngineToModuleIsClose

8. ModuleFallback tests (test_module_fallback.cpp)
a. ResNetModuleFallbacksCorrectly
b. MobileNetModuleFallbacksCorrectlyWithOneEngine

9. CanRunMultipleEngines tests (test_multiple_registered_engines.cpp)

10. CompiledModuleIsClose tests (test_compiled_modules.cpp)
```

Note: Most of these tests could be identified with DISABLE_TEST_IN_CI flag in the test suite.