-
Notifications
You must be signed in to change notification settings - Fork 14.3k
[lldb][progress][NFC] Add unit test for progress reports #79533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@llvm/pr-subscribers-lldb Author: Chelsea Cassanova (chelcassanova) ChangesThis test is being added as a way to check the behaviour of how progress events are broadcasted when reports are started and ended with the current implementation of progress reports. Here we're mainly checking and ensuring that the current behaviour is that progress events are broadcasted individually and placed in the event queue in order of their creation. Full diff: https://github.com/llvm/llvm-project/pull/79533.diff 2 Files Affected:
diff --git a/lldb/unittests/Core/CMakeLists.txt b/lldb/unittests/Core/CMakeLists.txt
index b3cddd150635b15..d40c357e3f463be 100644
--- a/lldb/unittests/Core/CMakeLists.txt
+++ b/lldb/unittests/Core/CMakeLists.txt
@@ -7,6 +7,7 @@ add_lldb_unittest(LLDBCoreTests
FormatEntityTest.cpp
MangledTest.cpp
ModuleSpecTest.cpp
+ ProgressReportTest.cpp
RichManglingContextTest.cpp
SourceLocationSpecTest.cpp
SourceManagerTest.cpp
diff --git a/lldb/unittests/Core/ProgressReportTest.cpp b/lldb/unittests/Core/ProgressReportTest.cpp
new file mode 100644
index 000000000000000..bdc168c9e077dd5
--- /dev/null
+++ b/lldb/unittests/Core/ProgressReportTest.cpp
@@ -0,0 +1,105 @@
+#include "Plugins/Platform/MacOSX/PlatformMacOSX.h"
+#include "Plugins/Platform/MacOSX/PlatformRemoteMacOSX.h"
+#include "lldb/Core/Debugger.h"
+#include "lldb/Core/Progress.h"
+#include "lldb/Host/FileSystem.h"
+#include "lldb/Host/HostInfo.h"
+#include "lldb/Utility/Listener.h"
+#include "gtest/gtest.h"
+#include <thread>
+
+using namespace lldb;
+using namespace lldb_private;
+
+namespace {
+class ProgressReportTest : public ::testing::Test {
+public:
+ void SetUp() override {
+ FileSystem::Initialize();
+ HostInfo::Initialize();
+ PlatformMacOSX::Initialize();
+ Debugger::Initialize(nullptr);
+ }
+ void TearDown() override {
+ Debugger::Terminate();
+ PlatformMacOSX::Terminate();
+ HostInfo::Terminate();
+ FileSystem::Terminate();
+ }
+};
+} // namespace
+TEST_F(ProgressReportTest, TestReportCreation) {
+ std::chrono::milliseconds timeout(100);
+
+ // Set up the debugger, make sure that was done properly
+ ArchSpec arch("x86_64-apple-macosx-");
+ Platform::SetHostPlatform(PlatformRemoteMacOSX::CreateInstance(true, &arch));
+
+ DebuggerSP debugger_sp = Debugger::CreateInstance();
+ ASSERT_TRUE(debugger_sp);
+
+ // Get the debugger's broadcaster
+ Broadcaster &broadcaster = debugger_sp->GetBroadcaster();
+
+ // Create a listener, make sure it can receive events and that it's
+ // listening to the correct broadcast bit
+ ListenerSP listener_sp = Listener::MakeListener("progress-listener");
+
+ listener_sp->StartListeningForEvents(&broadcaster,
+ Debugger::eBroadcastBitProgress);
+ EXPECT_TRUE(
+ broadcaster.EventTypeHasListeners(Debugger::eBroadcastBitProgress));
+
+ EventSP event_sp;
+ const ProgressEventData *data;
+
+ // Scope this for RAII on the progress objects
+ // Create progress reports and check that their respective events for having
+ // started are broadcasted
+ {
+ Progress progress1("Progress report 1", "Starting report 1");
+ EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout));
+
+ data = ProgressEventData::GetEventDataFromEvent(event_sp.get());
+ ASSERT_EQ(data->GetDetails(), "Starting report 1");
+
+ Progress progress2("Progress report 2", "Starting report 2");
+ EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout));
+
+ data = ProgressEventData::GetEventDataFromEvent(event_sp.get());
+ ASSERT_EQ(data->GetDetails(), "Starting report 2");
+
+ Progress progress3("Progress report 3", "Starting report 3");
+ EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout));
+ ASSERT_TRUE(event_sp);
+
+ data = ProgressEventData::GetEventDataFromEvent(event_sp.get());
+ ASSERT_EQ(data->GetDetails(), "Starting report 3");
+
+ std::this_thread::sleep_for(timeout);
+ }
+
+ // Progress report objects should be destroyed at this point so
+ // get each report from the queue and check that they've been
+ // destroyed in reverse order
+ std::this_thread::sleep_for(timeout);
+ EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout));
+ data = ProgressEventData::GetEventDataFromEvent(event_sp.get());
+
+ ASSERT_EQ(data->GetTitle(), "Progress report 3");
+ ASSERT_TRUE(data->GetCompleted());
+
+ std::this_thread::sleep_for(timeout);
+ EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout));
+ data = ProgressEventData::GetEventDataFromEvent(event_sp.get());
+
+ ASSERT_EQ(data->GetTitle(), "Progress report 2");
+ ASSERT_TRUE(data->GetCompleted());
+
+ std::this_thread::sleep_for(timeout);
+ EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout));
+ data = ProgressEventData::GetEventDataFromEvent(event_sp.get());
+
+ ASSERT_EQ(data->GetTitle(), "Progress report 1");
+ ASSERT_TRUE(data->GetCompleted());
+}
|
void SetUp() override { | ||
FileSystem::Initialize(); | ||
HostInfo::Initialize(); | ||
PlatformMacOSX::Initialize(); | ||
Debugger::Initialize(nullptr); | ||
} | ||
void TearDown() override { | ||
Debugger::Terminate(); | ||
PlatformMacOSX::Terminate(); | ||
HostInfo::Terminate(); | ||
FileSystem::Terminate(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a utility that simplifies this called SubsystemRAII
. Here's how you use that:
#include "TestingSupport/SubsystemRAII.h"
class ProgressReportTest : public ::testing::Test {
SubsystemRAII<FileSystem, HostInfo, PlatformMacOSX, Debugger> subsystems;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lovely, I'll add this in here
} | ||
}; | ||
} // namespace | ||
TEST_F(ProgressReportTest, TestReportCreation) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: missing newline
// Progress report objects should be destroyed at this point so | ||
// get each report from the queue and check that they've been | ||
// destroyed in reverse order | ||
std::this_thread::sleep_for(timeout); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This shouldn't be necessary: you've done the progress reporting from the same thread that you're getting the event from so since this is linear code, all events should be queued up in the broadcaster.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I thought I had to give a tiny bit of time for the objects to be destroyed, I'll remove the timeouts here then.
This is only testing the |
@@ -0,0 +1,105 @@ | |||
#include "Plugins/Platform/MacOSX/PlatformMacOSX.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't forget to include the license header in this file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do! I told myself I'd add it at the end then I fully forgot to do that 🙃
While it would be nice to have full test coverage, the goal was to cover the existing behavior which we'll modify/extend to coalesce events as discussed in the RFC. Increments are orthogonal to that so I think it's fine to keep that for a separate PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is great to have unit tests for this. I remember with your previous patch we caught an issue where the completed event might not be delivered, or it would be delivered without the values being set correctly, so this will be great to be able to catch this kind of stuff when we make changes.
Progress progress1("Progress report 1", "Starting report 1"); | ||
EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout)); | ||
|
||
data = ProgressEventData::GetEventDataFromEvent(event_sp.get()); | ||
ASSERT_EQ(data->GetDetails(), "Starting report 1"); | ||
|
||
Progress progress2("Progress report 2", "Starting report 2"); | ||
EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout)); | ||
|
||
data = ProgressEventData::GetEventDataFromEvent(event_sp.get()); | ||
ASSERT_EQ(data->GetDetails(), "Starting report 2"); | ||
|
||
Progress progress3("Progress report 3", "Starting report 3"); | ||
EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout)); | ||
ASSERT_TRUE(event_sp); | ||
|
||
data = ProgressEventData::GetEventDataFromEvent(event_sp.get()); | ||
ASSERT_EQ(data->GetDetails(), "Starting report 3"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we are looking to verify things are delivered in the order that they were supplied, do you want to do all of the event checking outside of this loop? Otherwise the only thing on the event queue here is the data we are looking for so this doesn't help us to verify that we got things in the order that they were pushed onto the queue. We should be able to do:
{
Progress progress1("Progress report 1", "Starting report 1");
Progress progress2("Progress report 2", "Starting report 2");
Progress progress3("Progress report 3", "Starting report 3");
}
// Now check that we got all events in the right order for both the start and completed
// check progress started for "Progress report 1"
// check progress started for "Progress report 2"
// check progress started for "Progress report 3"
// check progress completed for "Progress report 1"
// check progress completed for "Progress report 2"
// check progress completed for "Progress report 3"
This will help us verify the order of things.
Might be a good idea to do a check for progress reports with a valid total as well, and test that if we increment too many times that we don't deliver multiple completed events
{
Progress progress1("Progress report 1", "Starting report 1");
Progress progress2("Progress report 2", "Starting report 2");
Progress progress3("Progress report 3", "Starting report 3");
Progress progress4("Progress report 4", "Starting report 4", 2);
progress4.Increment(1, "progress detail 1");
// This should cause the progress to complete
progress4.Increment(1, "progress detail 2");
// The progress is already completed, I would expect to not see any more events for this progress
progress4.Increment(1, "too many, we shouldn't see this!");
}
And when verifying the events for progress4
:
ASSERT_TRUE(data->IsFinite());
ASSERT_FALSE|ASSERT_TRUE(data->GetCompleted()); // Make sure this is correct for each event
ASSERT_EQ(data->GetTotal(), 2);
ASSERT_EQ(data->GetMessage(), "..."); // Make sure this is correct for each increment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can check the events being delivered outside of the event queue as well as checking the other details from ProgressEventData
. Per Jonas' comment though since we're only testing the current behaviour maybe testing Increment
could be done in another PR?
EXPECT_TRUE(listener_sp->GetEvent(event_sp, timeout)); | ||
|
||
data = ProgressEventData::GetEventDataFromEvent(event_sp.get()); | ||
ASSERT_EQ(data->GetDetails(), "Starting report 1"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be good to verify a few more things here for each start event:
ASSERT_FALSE(data->IsFinite());
ASSERT_FALSE(data->GetCompleted());
ASSERT_EQ(data->GetTotal(), 1);
ASSERT_EQ(data->GetMessage(), "Progress report 1: Starting report 1");
data = ProgressEventData::GetEventDataFromEvent(event_sp.get()); | ||
|
||
ASSERT_EQ(data->GetTitle(), "Progress report 3"); | ||
ASSERT_TRUE(data->GetCompleted()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a few more tests for each complete event
ASSERT_FALSE(data->IsFinite());
ASSERT_EQ(data->GetMessage(), "...");
@@ -39,7 +39,7 @@ class ProgressEventData : public EventData { | |||
GetAsStructuredData(const Event *event_ptr); | |||
|
|||
uint64_t GetID() const { return m_id; } | |||
bool IsFinite() const { return m_total != UINT64_MAX; } | |||
bool IsFinite() const { return m_total != 1; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have this change here but let me know if it's better to have this in its own PR.
✅ With the latest revision this PR passed the C/C++ code formatter. |
|
||
TEST_F(ProgressReportTest, TestReportCreation) { | ||
std::chrono::milliseconds timeout(100); | ||
const unsigned long long NO_TOTAL = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should define this in Progress class:
class Progress {
constexpr uint64_t NO_TOTAL = UINT64_MAX;
UINT64_MAX is better than 1 because someone might actually create a valid finite progress with only 1 item.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can use the changes from #79912 in here for checking the total progress.
ASSERT_EQ(data->GetDetails(), "Starting report 1"); | ||
ASSERT_FALSE(data->IsFinite()); | ||
ASSERT_FALSE(data->GetCompleted()); | ||
ASSERT_EQ(data->GetTotal(), NO_TOTAL); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ASSERT_EQ(data->GetTotal(), Progress::NO_TOTAL);
@@ -39,7 +39,7 @@ class ProgressEventData : public EventData { | |||
GetAsStructuredData(const Event *event_ptr); | |||
|
|||
uint64_t GetID() const { return m_id; } | |||
bool IsFinite() const { return m_total != UINT64_MAX; } | |||
bool IsFinite() const { return m_total != 1; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mention creating a variable in the Progress class in an inline comment below:
bool IsFinite() const { return m_total != Progress::NO_TOTAL; }
This test is being added as a way to check the behaviour of how progress events are broadcasted when reports are started and ended with the current implementation of progress reports. Here we're mainly checking and ensuring that the current behaviour is that progress events are broadcasted individually and placed in the event queue in order of their creation.
Uses `SubsystemRAII` to initialize the necessary platform and host information. Since the debugger cannot be initialized with no arguments (which is how `SubsystemRAII` will initialize everything), the debugger will be initialized the usual way. Also removes the thread timeouts and adds the LLVM license header.
Checks that the progress events were received in order of report creation and deletion outside of the scope that the events were created in, also checks for more information from the progress event data. `IsFinite()` from DebuggerEvents would return true if the total was not UINT64_MAX. Since we no longer use this to specify that an event has no total this has been changed to check that the total is not 1.
5b74fb9
to
06522a6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a handful of comments that are missing a period. Otherwise this LGTM.
Use the changes implemented in llvm#79912 to use a static variable from `Progress` to ensure that the report was created with a non-deterministic total.
06522a6
to
f3f2d84
Compare
)" This reverts commit 51e0d1b. That commit breaks a unit test: ``` Failed Tests (1): lldb-unit :: Core/./LLDBCoreTests/4/8 ```
)" This reverts commit 209fe1f. The original commit failed to due an assertion failure in the unit test `ProgressReportTest` that the commit added. The Debugger::Initialize() function was called more than once which triggered the assertion, so this commit calls that function under a `std::call_once`.
…m#79533)" This reverts commit 209fe1f. The original commit failed to due an assertion failure in the unit test `ProgressReportTest` that the commit added. The Debugger::Initialize() function was called more than once which triggered the assertion, so this commit calls that function under a `std::call_once`.
…rts (llvm#79533)"" This reverts commit a5a8cbb. The test being added by that commit still fails on the assertion that Debugger::Initialize has been called more than once.
I tried adding a new unit test to the core test suite (llvm#79533) but it broke the test suite on AArch64 Linux due to hitting an assertion for calling `Debugger::Initialize` more than once. When the unit test suite is invoked as a standalone binary the test suite state is shared, and `Debugger::Initialize` gets called in `DiagnosticEventTest.cpp` before being called in `ProgressReportTest.cpp`. `DiagnosticEventTest.cpp` uses a call_once flag to initialize the debugger but it's local to that test. This commit adds a once_flag to `TestUtilities` so that `Debugger::Initialize` can be called once by the tests that use it.
I tried adding a new unit test to the core test suite (llvm#79533) but it broke the test suite on AArch64 Linux due to hitting an assertion for calling `Debugger::Initialize` more than once. When the unit test suite is invoked as a standalone binary the test suite state is shared, and `Debugger::Initialize` gets called in `DiagnosticEventTest.cpp` before being called in `ProgressReportTest.cpp`. `DiagnosticEventTest.cpp` uses a call_once flag to initialize the debugger but it's local to that test. This commit adds a once_flag to `TestUtilities` so that `Debugger::Initialize` can be called once by the tests that use it.
This file was previously approved and merged from this PR: llvm#79533 but caused a test failure on the Linux AArch64 bots due to hitting an assertion that `Debugger::Initialize` was already called. To fix this, this commit uses the changes made here: llvm#80786 to use a shared call_once flag to initialize the debugger.
I tried adding a new unit test to the core test suite (llvm#79533) but it broke the test suite on AArch64 Linux due to hitting an assertion for calling `Debugger::Initialize` more than once. When the unit test suite is invoked as a standalone binary the test suite state is shared, and `Debugger::Initialize` gets called in `DiagnosticEventTest.cpp` before being called in `ProgressReportTest.cpp`. `DiagnosticEventTest.cpp` uses a call_once flag to initialize the debugger but it's local to that test. This commit adds a once_flag to `TestUtilities` so that `Debugger::Initialize` can be called once by the tests that use it.
I tried adding a new unit test to the core test suite (#79533) but it broke the test suite on AArch64 Linux due to hitting an assertion for calling `Debugger::Initialize` more than once. When the unit test suite is invoked as a standalone binary the test suite state is shared, and `Debugger::Initialize` gets called in `DiagnosticEventTest.cpp` before being called in `ProgressReportTest.cpp`. `DiagnosticEventTest.cpp` uses a call_once flag to initialize the debugger but it's local to that test. This commit adds a once_flag to `TestUtilities` so that `Debugger::Initialize` can be called once by the tests that use it.
This file was previously approved and merged from this PR: llvm#79533 but caused a test failure on the Linux AArch64 bots due to hitting an assertion that `Debugger::Initialize` was already called. To fix this, this commit uses the changes made here: llvm#80786 to use a shared call_once flag to initialize the debugger.
This file was previously approved and merged from this PR: llvm#79533 but caused a test failure on the Linux AArch64 bots due to hitting an assertion that `Debugger::Initialize` was already called. To fix this, this commit uses the changes made here: llvm#80786 to use a shared call_once flag to initialize the debugger.
…0791) This file was previously approved and merged from this PR: #79533 but caused a test failure on the Linux AArch64 bots due to hitting an assertion that `Debugger::Initialize` was already called. To fix this, this commit uses the changes made here: #80786 to use a shared call_once flag to initialize the debugger.
I tried adding a new unit test to the core test suite (llvm#79533) but it broke the test suite on AArch64 Linux due to hitting an assertion for calling `Debugger::Initialize` more than once. When the unit test suite is invoked as a standalone binary the test suite state is shared, and `Debugger::Initialize` gets called in `DiagnosticEventTest.cpp` before being called in `ProgressReportTest.cpp`. `DiagnosticEventTest.cpp` uses a call_once flag to initialize the debugger but it's local to that test. This commit adds a once_flag to `TestUtilities` so that `Debugger::Initialize` can be called once by the tests that use it. (cherry picked from commit 3885483)
…vm#80791) This file was previously approved and merged from this PR: llvm#79533 but caused a test failure on the Linux AArch64 bots due to hitting an assertion that `Debugger::Initialize` was already called. To fix this, this commit uses the changes made here: llvm#80786 to use a shared call_once flag to initialize the debugger. (cherry picked from commit a8ab830)
…ing support (#8141) * [lldb][unittest] Add call_once flag to initialize debugger (llvm#80786) I tried adding a new unit test to the core test suite (llvm#79533) but it broke the test suite on AArch64 Linux due to hitting an assertion for calling `Debugger::Initialize` more than once. When the unit test suite is invoked as a standalone binary the test suite state is shared, and `Debugger::Initialize` gets called in `DiagnosticEventTest.cpp` before being called in `ProgressReportTest.cpp`. `DiagnosticEventTest.cpp` uses a call_once flag to initialize the debugger but it's local to that test. This commit adds a once_flag to `TestUtilities` so that `Debugger::Initialize` can be called once by the tests that use it. (cherry picked from commit 3885483) * [lldb][unittest] Use shared once_flag in DiagnosticEventTest (llvm#80788) Incorporates the changes from llvm#80786 to use a once_flag from `TestUtilities` instead of a local flag in order to prevent hitting an assertion that the debugger was initialized again in another test. (cherry picked from commit 5690027)
…vm#80791) (#8142) This file was previously approved and merged from this PR: llvm#79533 but caused a test failure on the Linux AArch64 bots due to hitting an assertion that `Debugger::Initialize` was already called. To fix this, this commit uses the changes made here: llvm#80786 to use a shared call_once flag to initialize the debugger. (cherry picked from commit a8ab830)
This test is being added as a way to check the behaviour of how progress events are broadcasted when reports are started and ended with the current implementation of progress reports. Here we're mainly checking and ensuring that the current behaviour is that progress events are broadcasted individually and placed in the event queue in order of their creation.