Skip to content

Fix tasty hash #4865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Aug 7, 2018
Merged

Fix tasty hash #4865

Changes from 5 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
bd41bee
Assemble buffer before computing the hash
nicolasstucki Jul 29, 2018
c905ff2
Add missing if in pjwHash64 implementation
nicolasstucki Jul 29, 2018
3ddfa3c
Fix input of pjwHash64
nicolasstucki Jul 29, 2018
05ab4cf
Fix pjwHash64 to generate 64-bit Longs
nicolasstucki Jul 29, 2018
d6898b9
Improve entropy of low bits of the Tasty UUID
nicolasstucki Jul 29, 2018
26679a4
Remove branching in while loop
nicolasstucki Jul 29, 2018
2bf8696
Add pjwHash64 regression tests
nicolasstucki Aug 6, 2018
38a6cf9
Use an *unsigned* right shift
Blaisorblade Aug 6, 2018
88ca356
Make tasty hash test a JUnit test
nicolasstucki Aug 7, 2018
ac7647f
Use JUnit assert
Blaisorblade Aug 7, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 19 additions & 11 deletions compiler/src/dotty/tools/dotc/core/tasty/TastyPickler.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,19 @@ class TastyPickler(val rootCls: ClassSymbol) {
sections += ((nameBuffer.nameIndex(name.toTermName), buf))

def assembleParts(): Array[Byte] = {
def lengthWithLength(buf: TastyBuffer) = {
buf.assemble()
def lengthWithLength(buf: TastyBuffer) =
buf.length + natSize(buf.length)
}

val uuidLow: Long = pjwHash64(nameBuffer.bytes)
val uuidHi: Long = sections.iterator.map(x => pjwHash64(x._2.bytes)).fold(0L)(_ ^ _)
nameBuffer.assemble()
sections.foreach(_._2.assemble())

val nameBufferHash = pjwHash64(nameBuffer.bytes)
val treeSectionHash +: otherSectionHashes = sections.map(x => pjwHash64(x._2.bytes))

// Hash of name table and tree
val uuidLow: Long = nameBufferHash ^ treeSectionHash
// Hash of positions, comments and any additional section
val uuidHi: Long = otherSectionHashes.fold(0L)(_ ^ _)

val headerBuffer = {
val buf = new TastyBuffer(header.length + 24)
Expand Down Expand Up @@ -74,17 +80,19 @@ class TastyPickler(val rootCls: ClassSymbol) {

/** Returns a non-cryptographic 64-bit hash of the array.
*
* from https://en.wikipedia.org/wiki/PJW_hash_function#Implementation
* from https://en.wikipedia.org/wiki/PJW_hash_function#Algorithm
*/
private def pjwHash64(data: Array[Byte]): Long = {
var h = 0L
var high = 0L
var i = 0
while (i < data.length) {
h = (h << 4) + data(i)
high = h & 0xF0000000L
h ^= high >> 24
h &= ~high
val d = data(i) & 0xFFL // Interpret byte as unsigned byte
h = (h << 8) + d
val high = h & 0xFF00000000000000L
if (high != 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the if, it's not necessary and branches should be avoided in tight loops.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

h ^= high >> 48L
h &= ~high
}
i += 1
}
h
Expand Down