Skip to content

[lldb] Register reflection images with LLDBMemoryReader #3800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 1, 2022
Merged

[lldb] Register reflection images with LLDBMemoryReader #3800

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lldb/include/lldb/Target/Target.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,8 @@ class TargetProperties : public Properties {

bool GetSwiftCreateModuleContextsInParallel() const;

bool GetSwiftReadMetadataFromFileCache() const;

bool GetEnableAutoImportClangModules() const;

bool GetUseAllCompilerFlags() const;
Expand Down
154 changes: 145 additions & 9 deletions lldb/source/Plugins/LanguageRuntime/Swift/LLDBMemoryReader.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
#include "lldb/Utility/Log.h"
#include "lldb/Utility/Logging.h"

#include "llvm/Support/MathExtras.h"

using namespace lldb;
using namespace lldb_private;

Expand Down Expand Up @@ -134,15 +136,26 @@ bool LLDBMemoryReader::readBytes(swift::remote::RemoteAddress address,
LLDB_LOGV(log, "[MemoryReader] asked to read {0} bytes at address {1:x}",
size, address.getAddressData());

llvm::Optional<Address> maybeAddr =
resolveRemoteAddress(address.getAddressData());
if (!maybeAddr) {
LLDB_LOGV(log, "[MemoryReader] could not resolve address {1:x}",
address.getAddressData());
return false;
}
auto addr = *maybeAddr;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good!


if (size > m_max_read_amount) {
LLDB_LOGV(log, "[MemoryReader] memory read exceeds maximum allowed size");
return false;
}

Target &target(m_process.GetTarget());
Address addr(address.getAddressData());
Status error;
if (size > target.ReadMemory(addr, dest, size, error, true)) {
// We only want to allow the file-cache optimization if we resolved the
// address to section + offset.
const bool force_live_memory =
!readMetadataFromFileCacheEnabled() || !addr.IsSectionOffset();
if (size > target.ReadMemory(addr, dest, size, error, force_live_memory)) {
LLDB_LOGV(log,
"[MemoryReader] memory read returned fewer bytes than asked for");
return false;
Expand Down Expand Up @@ -171,11 +184,19 @@ bool LLDBMemoryReader::readString(swift::remote::RemoteAddress address,
std::string &dest) {
Log *log(GetLogIfAllCategoriesSet(LIBLLDB_LOG_TYPES));

LLDB_LOGV(log, "[MemoryReader] asked to read string data at address {0:x}",
LLDB_LOGV(log, "[MemoryReader] asked to read string data at address {0x}",
address.getAddressData());

llvm::Optional<Address> maybeAddr =
resolveRemoteAddress(address.getAddressData());
if (!maybeAddr) {
LLDB_LOGV(log, "[MemoryReader] could not resolve address {1:x}",
address.getAddressData());
return false;
}
auto addr = *maybeAddr;

Target &target(m_process.GetTarget());
Address addr(address.getAddressData());
Status error;
target.ReadCStringFromMemory(addr, dest, error);
if (error.Success()) {
Expand All @@ -194,11 +215,10 @@ bool LLDBMemoryReader::readString(swift::remote::RemoteAddress address,
LLDB_LOGV(log, "[MemoryReader] memory read returned data: \"{0}\"",
format_string(dest));
return true;
} else {
LLDB_LOGV(log, "[MemoryReader] memory read returned error: {0}",
error.AsCString());
return false;
}
LLDB_LOGV(log, "[MemoryReader] memory read returned error: {0}",
error.AsCString());
return false;
}

void LLDBMemoryReader::pushLocalBuffer(uint64_t local_buffer,
Expand All @@ -214,4 +234,120 @@ void LLDBMemoryReader::popLocalBuffer() {
m_local_buffer_size = 0;
}

llvm::Optional<std::pair<uint64_t, uint64_t>>
LLDBMemoryReader::addModuleToAddressMap(ModuleSP module) {
if (!readMetadataFromFileCacheEnabled())
return {};

// The first available address is the mask, since subsequent images are mapped
// in ascending order, all of them will contain this mask.
uint64_t module_start_address = LLDB_FILE_ADDRESS_BIT;
if (!m_range_module_map.empty())
// We map the images contiguously one after the other, all with the tag bit
// set.
// The address that maps the last module is exactly the address the new
// module should start at.
module_start_address = m_range_module_map.back().first;

#ifndef NDEBUG
static std::initializer_list<uint64_t> objc_bits = {
SWIFT_ABI_ARM_IS_OBJC_BIT,
SWIFT_ABI_X86_64_IS_OBJC_BIT,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this SWIFT_ABI_X86_64_IS_OBJC_BIT included in SWIFT_ABI_X86_64_SWIFT_SPARE_BITS_MASK?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, should we remove it?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only need a static_assert that our tag is inside of SWIFT_ABI_*_SWIFT_SPARE_BITS_MASK, to check that our tag will not collide with real addresses in the process. And then a second check to make sure that our tag does not collide with the OBJC bit to make sure our addresses don't look like OBjC tagged pointers to ReflectionContext.

SWIFT_ABI_ARM64_IS_OBJC_BIT};

for (auto objc_bit : objc_bits)
assert((module_start_address & objc_bit) != objc_bit &&
"LLDB file address bit clashes with an obj-c bit!");
#endif

SectionList *section_list = module->GetObjectFile()->GetSectionList();

auto section_list_size = section_list->GetSize();
if (section_list_size == 0)
return {};

auto last_section =
section_list->GetSectionAtIndex(section_list->GetSize() - 1);
// The virtual file address + the size of last section gives us the total size
// of this image in memory.
uint64_t size = last_section->GetFileAddress() + last_section->GetByteSize();
auto module_end_address = module_start_address + size;

// The address for the next image is the next pointer aligned address
// available after the end of the current image.
uint64_t next_module_start_address = llvm::alignTo(module_end_address, 8);
m_range_module_map.emplace_back(next_module_start_address, module);
return {{module_start_address, module_end_address}};
}

llvm::Optional<Address>
LLDBMemoryReader::resolveRemoteAddress(uint64_t address) const {
Log *log(GetLogIfAllCategoriesSet(LIBLLDB_LOG_TYPES));

if (!m_process.GetTarget().GetSwiftReadMetadataFromFileCache())
return Address(address);

// If the address contains our mask, this is an image we registered.
if (!(address & LLDB_FILE_ADDRESS_BIT))
return Address(address);

// Dummy pair with the address we're looking for.
auto comparison_pair = std::make_pair(address, ModuleSP());

// Explicitly compare only the addresses, never the modules in the pairs.
auto pair_iterator = std::lower_bound(
m_range_module_map.begin(), m_range_module_map.end(), comparison_pair,
[](auto &a, auto &b) { return a.first < b.first; });

// If the address is larger than anything we have mapped the address is out
if (pair_iterator == m_range_module_map.end()) {
LLDB_LOG(log,
"[MemoryReader] Address {1:x} is larger than the upper bound "
"address of the mapped in modules",
address);
return {};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be logged (with LOGV)?

}

ModuleSP module = pair_iterator->second;
uint64_t file_address;
if (pair_iterator == m_range_module_map.begin())
// Since this is the first registered module,
// clearing the tag bit will give the virtual file address.
file_address = address & ~LLDB_FILE_ADDRESS_BIT;
else
// The end of the previous section is the start of the current one.
file_address = address - std::prev(pair_iterator)->first;

LLDB_LOGV(log,
"[MemoryReader] Successfully resolved mapped address {1:x} "
"into file address {1:x}",
address, file_address);
auto *object_file = module->GetObjectFile();
if (!object_file)
return {};

Address resolved(file_address, object_file->GetSectionList());
if (!resolved.IsValid()) {
LLDB_LOG(log,
"[MemoryReader] Could not make a real address out of file "
"address {1:x} and object file {}",
file_address, object_file->GetFileSpec().GetFilename());
return {};
}

LLDB_LOGV(log,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is dead code.

"[MemoryReader] Unsuccessfully resolved mapped address {1:x} "
"into file address {1:x}",
address, address);
return resolved;
}

bool LLDBMemoryReader::readMetadataFromFileCacheEnabled() const {
auto &triple = m_process.GetTarget().GetArchitecture().GetTriple();

// 32 doesn't have a flag bit we can reliably use, so reading from filecache
// is disabled on it.
return m_process.GetTarget().GetSwiftReadMetadataFromFileCache() &&
triple.isArch64Bit();
}
} // namespace lldb_private
43 changes: 41 additions & 2 deletions lldb/source/Plugins/LanguageRuntime/Swift/LLDBMemoryReader.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ namespace lldb_private {
class LLDBMemoryReader : public swift::remote::MemoryReader {
public:
LLDBMemoryReader(Process &p, size_t max_read_amount = INT32_MAX)
: m_process(p) {
: m_process(p), m_range_module_map() {
m_max_read_amount = max_read_amount;
}

Expand All @@ -31,14 +31,53 @@ class LLDBMemoryReader : public swift::remote::MemoryReader {

void pushLocalBuffer(uint64_t local_buffer, uint64_t local_buffer_size);

void popLocalBuffer();
void popLocalBuffer();

/// Adds the module to the list of modules we're tracking using tagged
/// addresses, so we can read memory from the file cache whenever possible.
/// \return a pair of addresses indicating the start and end of this image in
/// the tagged address space. None on failure.
llvm::Optional<std::pair<uint64_t, uint64_t>>
addModuleToAddressMap(lldb::ModuleSP module);

/// Returns whether the filecache optimization is enabled or not.
bool readMetadataFromFileCacheEnabled() const;

private:
/// Resolves the address by either mapping a tagged address back to an LLDB
/// Address with section + offset, or, in case the address is not tagged,
/// constructing an LLDB address with just the offset.
/// \return an Address with Section + offset if we succesfully converted a tagged
/// address back, an Address with just an offset if the address was not tagged,
/// and None if the address was tagged but we couldn't convert it back to an
/// Address.
llvm::Optional<Address> resolveRemoteAddress(uint64_t address) const;

private:
Process &m_process;
size_t m_max_read_amount;

llvm::Optional<uint64_t> m_local_buffer;
uint64_t m_local_buffer_size = 0;

/// LLDBMemoryReader prefers to read reflection metadata from the
/// binary on disk, which is faster than reading it out of process
/// memory, especially when debugging remotely. To achieve this LLDB
/// registers virtual addresses starting at (0x0 &
/// LLDB_VIRTUAL_ADDRESS_BIT) with ReflectionContext. Sorted by
/// virtual address, m_lldb_virtual_address_map stores each
/// lldb::Module and the first virtual address after the end of that
/// module's virtual address space.
std::vector<std::pair<uint64_t, lldb::ModuleSP>> m_range_module_map;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about?

// LLDBMemoryReader prefers to read reflection metadata from the
// binary on disk, which is faster than reading it out of process
// memory, especially when debugging remotely.  To achieve this LLDB
// registers virtual addresses starting at (0x0 &
// LLDB_VIRTUAL_ADDRESS_BIT) with ReflectionContext.  Sorted by
// virtual address, m_lldb_virtual_address_map stores each
// lldb::Module and the first virtual address after the end of that
// module's virtual address space.
std::vector<std::pair<uint64_t, lldb::ModuleSP>> m_lldb_virtual_address_map;


/// The bit used to tag LLDB's virtual addresses as such. See \c
/// m_range_module_map.
const static uint64_t LLDB_FILE_ADDRESS_BIT = 0x2000000000000000;
static_assert(LLDB_FILE_ADDRESS_BIT & SWIFT_ABI_X86_64_SWIFT_SPARE_BITS_MASK,
"LLDB file address bit not in spare bits mask!");
static_assert(LLDB_FILE_ADDRESS_BIT & SWIFT_ABI_ARM64_SWIFT_SPARE_BITS_MASK,
"LLDB file address bit not in spare bits mask!");

};
} // namespace lldb_private
#endif
Loading