MurmurHash

MurmurHash is a non-cryptographic hash function suitable for general hash-based lookup.^[1][2][3] It was created by Austin Appleby in 2008^[4] and is currently hosted on Github along with its test suite named 'SMHasher'. It also exists in a number of variants,^[5] all of which have been released into the public domain. The name comes from two basic operations, multiply (MU) and rotate (R), used in its inner loop.

Unlike cryptographic hash functions, it is not specifically designed to be difficult to reverse by an adversary, making it unsuitable for cryptographic purposes.

Variants

Implementations

The canonical implementation is in C++, but there are efficient ports for a variety of popular languages, including Python,^[9] C,^[10] Go,^[11] C#,^[7][12] D,^[13],Lua Perl,^[14] Ruby,^[15] Rust,^[16] PHP,^[17] Common Lisp,^[18] Haskell,^[19] Clojure,^[20] Scala,^[21] Java,^[22][23] Erlang,^[24] and JavaScript,^[25][26] together with an online version.^[27]

It has been adopted into a number of open-source projects, most notably libstdc++ (ver 4.6), nginx (ver 1.0.1),^[28] Rubinius,^[29] libmemcached (the C driver for Memcached),^[30] npm (nodejs package manager),^[31] maatkit,^[32] Hadoop,^[1] Kyoto Cabinet,^[33] RaptorDB,^[34] OlegDB,^[35] Cassandra,^[36] Solr,^[37] vowpal wabbit,^[38]Elasticsearch,^[39], Guava^[40] and RedHat Virtual Data Optimizer (VDO)^[[41]]

Vulnerabilities

Hash functions can be vulnerable to attack if a user can choose input data in such as way to intentionally cause hash collisions. Jean-Philippe Aumasson and Daniel J. Bernstein were able to show that even implementations of MurmurHash using a randomized seed are vulnerable to so-called HashDoS attacks.^[42] With the use of differential cryptanalysis they were able to generate inputs that would lead to a hash collision. The authors of the attack recommend to use their own SipHash instead.

Algorithm

Murmur3_32(key, len, seed)
    // Note: In this version, all integer arithmetic is performed with unsigned 32 bit integers.
    //       In the case of overflow, the result is constrained by the application of modulo 
 arithmetic.
    
    c1 ← 0xcc9e2d51
    c2 ← 0x1b873593
    r1 ← 15
    r2 ← 13
    m ← 5
    n ← 0xe6546b64
 
    hash ← seed

    for each fourByteChunk of key
        k ← fourByteChunk

        k ← k × c1
        k ← (k ROL r1)
        k ← k × c2

        hash ← hash XOR k
        hash ← (hash ROL r2)
        hash ← hash × m + n

    with any remainingBytesInKey
        remainingBytes ← SwapToLittleEndian(remainingBytesInKey)
        // Note: Endian swapping is only necessary on big-endian machines.
        //       The purpose is to place the meaningful digits towards the low end of the value,
        //       so that these digits have the greatest potential to affect the low range digits
        //       in the subsequent multiplication.  Consider that locating the meaningful digits
        //       in the high range would produce a greater effect upon the high digits of the
        //       multiplication, and notably, that such high digits are likely to be discarded
        //       by the modulo arithmetic under overflow.  We don't want that.
        
        remainingBytes ← remainingBytes × c1
        remainingBytes ← (remainingBytes ROL r1)
        remainingBytes ← remainingBytes × c2

        hash ← hash XOR remainingBytes
 
    hash ← hash XOR len

    hash ← hash XOR (hash >> 16)
    hash ← hash × 0x85ebca6b
    hash ← hash XOR (hash >> 13)
    hash ← hash × 0xc2b2ae35
    hash ← hash XOR (hash >> 16)

• A sample C implementation follows (for little-endian CPUs)

  uint32_t murmur3_32(const uint8_t* key, size_t len, uint32_t seed) {
    uint32_t h = seed;
    if (len > 3) {
      const uint32_t* key_x4 = (const uint32_t*) key;
      size_t i = len >> 2;
      do {
        uint32_t k = *key_x4++;
        k *= 0xcc9e2d51;
        k = (k << 15) | (k >> 17);
        k *= 0x1b873593;
        h ^= k;
        h = (h << 13) | (h >> 19);
        h = (h * 5) + 0xe6546b64;
      } while (--i);
      key = (const uint8_t*) key_x4;
    }
    if (len & 3) {
      size_t i = len & 3;
      uint32_t k = 0;
      key = &key[i - 1];
      do {
        k <<= 8;
        k |= *key--;
      } while (--i);
      k *= 0xcc9e2d51;
      k = (k << 15) | (k >> 17);
      k *= 0x1b873593;
      h ^= k;
    }
    h ^= len;
    h ^= h >> 16;
    h *= 0x85ebca6b;
    h ^= h >> 13;
    h *= 0xc2b2ae35;
    h ^= h >> 16;
    return h;
  }
  

See also

• Non-cryptographic hash functions

References

External links

• Official SMHasher site