Hardware acceleration

In computing, hardware acceleration is the use of computer hardware specially made to perform some functions more efficiently than is possible in software running on a general-purpose CPU. Any transformation of data or routine that can be computed, can be calculated purely in software run on a generic CPU, purely in custom-made hardware, or in some mix of both. An operation can be computed faster in application-specific hardware designed or programmed to compute the operation than specified in software and performed on a general-purpose computer processor. Each approach has advantages and disadvantages. The implementation of computing tasks in hardware to decrease latency and increase throughput is known as hardware acceleration.

Typical advantages of software include more rapid development (leading to faster times to market), lower non-recurring engineering costs, heightened portability, and ease of updating features or patching bugs, at the cost of overhead to compute general operations. Advantages of hardware include speedup, reduced power consumption,[1] lower latency, increased parallelism[2] and bandwidth, and better utilization of area and functional components available on an integrated circuit; at the cost of lower ability to update designs once etched onto silicon and higher costs of functional verification and times to market. In the hierarchy of general-purpose processors such as CPUs, more specialized processors such as GPUs, fixed-function implemented on field-programmable gate arrays (FPGAs), and fixed-function implemented on application-specific integrated circuit (ASICs); there is a tradeoff between flexibility and efficiency, with efficiency increasing by orders of magnitude when any given application is implemented higher up that hierarchy.[3][4]

Hardware acceleration is advantageous for performance, and practical when the functions are fixed so updates are not as needed as in software solutions. With the advent of reprogrammable logic devices such as FPGAs, the restriction of hardware acceleration to fully fixed algorithms is easing in the 2010s decade, allowing hardware acceleration to be applied to problem domains requiring modification to algorithms and processing control flow.[5][6][7]

Overview

Integrated circuits can be created to perform arbitrary operations on analog and digital signals. Most often in computing, signals are digital and can be interpreted as binary number data. Computer hardware and software operate on information in binary representation to perform computing; this is accomplished by calculating boolean functions on the bits of input and outputting the result to some output device downstream for storage or further processing. Either software or hardware can compute any computable function. Custom hardware offers higher performance per watt for the same functions that can be specified in software. Hardware description languages (HDLs) such as Verilog and VHDL can model the same semantics as software and synthesize the design into a netlist that can be programmed to an FPGA or composed into logic gates of an application-specific integrated circuit.

The vast majority of software-based computing occurs on machines implementing the von Neumann architecture, collectively known as stored-program computers. Computer programs are stored as data and executed by processors, typically one or more CPU cores. Such processors must fetch and decode instructions as well as data operands from memory as part of the instruction cycle to execute the instructions constituting the software program. Relying on a common cache for code and data leads to the von Neumann bottleneck, a fundamental limitation on the throughput of software on processors implementing the von Neumann architecture. Even in the modified Harvard architecture, where instructions and data have separate caches in the memory hierarchy, there is overhead to decoding instruction opcodes and multiplexing available execution units on a microprocessor or microcontroller, leading to low circuit utilization. Intel's hyper-threading technology provides simultaneous multithreading by exploiting underutilization of available processor functional units and instruction level parallelism between different hardware threads.

Hardware execution units do not in general rely on the von Neumann or modified Harvard architectures and do not need to perform the instruction fetch and decode steps of an instruction cycle and incur their overhead. If needed calculations are specified in a register transfer level hardware design, the time and circuit area costs of the instruction fetch and decoding stages can be reclaimed. This saves time, power and circuit area, allowing the reclaimed resources to be used for increased parallelism, possibly for multiple functions, as well as increased input/output capabilities; at the opportunity cost of less general-purpose utility. Additionally, greater RTL customization of hardware designs allows emerging architectures such as in-memory computing, transport triggered architectures (TTA) and networks-on-chip (NoC) to further benefit from increased locality of data to execution context, thereby reducing computing and communication latency between modules and functional units.

Custom hardware is limited in parallel processing capability only by the area and logic blocks available on the integrated circuit die.[8] Therefore, hardware is much more free to offer massive parallelism than software on general-purpose processors, offering a possibility of implementing the parallel random-access machine (PRAM) model. It is common to build multicore and manycore processing units out of microprocessor IP core schematics on a single FPGA or ASIC.[9][10][11][12][13] Similarly, specialized functional units can be composed in parallel as in digital signal processing without being embedded in a processor IP core. Therefore, hardware acceleration is often employed for repetitive, fixed tasks involving little conditional branching, especially on large amounts of data. This is how Nvidia's CUDA line of GPUs are implemented.

Example tasks accelerated

Summing one million integers

Suppose we wish to compute the sum of integers. Assuming large integers are available as bignum large enough to hold the sum, this can be done in software by specifying

constexpr int N = 20;
constexpr int two_to_the_N = 1 << N;

bignum array_sum(const std::array<int, two_to_the_N>& ints) {
    bignum result = 0;
    for (std::size_t i = 0; i < two_to_the_N; i++) {
        result += ints[i];
    }
    return result;
}

This algorithm is linear time, in Big O notation. In hardware, with sufficient area on chip, calculation can be parallelized to take only 20 time steps using the prefix sum algorithm.[14] The algorithm requires only logarithmic time, , and space as an in-place algorithm.

parameter int N = 20;
parameter int two_to_the_N = 1 << N;

function int array_sum;
    input int array[two_to_the_N];
    begin
        for (genvar i = 0; i < N; i++) begin
            for (genvar j = 0; j < two_to_the_N; j++) begin
                if (j >= (1 << i)) begin
                    array[j] = array[j] + array[j - (1 << i)];
                end
            end
        end
        return array[two_to_the_N - 1];
    end
endfunction

This example takes advantage of the greater parallel resources available in application-specific hardware than most software and general-purpose computing paradigms and architectures.

Stream processing

Hardware acceleration can be applied to stream processing.

Applications

Examples of hardware acceleration include bit blit acceleration functionality in graphics processing units (GPUs), use of memristors for accelerating neural networks[15] and regular expression hardware acceleration for spam control in the server industry, intended to prevent regular expression denial of service (ReDoS) attacks.[16] The hardware that performs the acceleration may be part of a general-purpose CPU, or a separate unit. In the second case, it is referred to as a hardware accelerator, or often more specifically as a 3D accelerator, cryptographic accelerator, etc.

Traditionally, processors were sequential (instructions are executed one by one), and were designed to run general purpose algorithms controlled by instruction fetch (for example moving temporary results to and from a register file). Hardware accelerators improve the execution of a specific algorithm by allowing greater concurrency, having specific datapaths for its temporary variables, and reducing the overhead of instruction control in the fetch-decode-execute cycle. Modern processors are multi-core and often feature parallel SIMD units; however hardware acceleration still yields benefits. Hardware acceleration is suitable for any computation-intensive algorithm which is executed frequently. Depending upon the granularity, hardware acceleration can vary from a small functional unit, to a large functional block (like motion estimation in MPEG-2).

Hardware acceleration units by application

Application Hardware accelerator Acronym
Computer graphics Graphics processing unit GPU
  • GPGPU
  • CUDA
  • N/A
Digital signal processing Digital signal processor DSP
Analog signal processing Field-programmable analog array FPAA
  • FPRF
Sound processing Sound card and sound card mixer N/A
Computer networking Network processor and network interface controller NPU and NIC
  • NoC
  • TCPOE or TOE
  • I/OAT or IOAT
Cryptography Cryptographic accelerator and secure cryptoprocessor N/A
Artificial intelligence AI accelerator N/A
  • VPU
  • PNN
  • N/A
Multilinear algebra Tensor processing unit TPU
Physics simulation Physics processing unit PPU
Regular expressions[16] Regular expression coprocessor N/A
Data compression[17] Data compression accelerator N/A
In-memory processing Network on a chip and Systolic array NoC; N/A
Any computing task Computer hardware HW (sometimes)
  • FPGA
  • ASIC
  • CPLD
  • SoC
    • MPSoC
    • PSoC

See also

References

  1. "Microsoft Supercharges Bing Search With Programmable Chips". WIRED. 16 June 2014.
  2. "Archived copy". Archived from the original on 2007-10-08. Retrieved 2012-08-18. "FPGA Architectures from 'A' to 'Z'" by Clive Maxfield 2006
  3. "Mining hardware comparison - Bitcoin". Retrieved 17 July 2014.
  4. "Non-specialized hardware comparison - Bitcoin". Retrieved 25 February 2014.
  5. "A Survey of FPGA-based Accelerators for Convolutional Neural Networks", S. Mittal, NCAA, 2018
  6. Morgan, Timothy Pricket (2014-09-03). "How Microsoft Is Using FPGAs To Speed Up Bing Search". Enterprise Tech. Retrieved 2018-09-18.
  7. "Project Catapult". Microsoft Research.
  8. MicroBlaze Soft Processor: Frequently Asked Questions Archived 2011-10-27 at the Wayback Machine.
  9. István Vassányi. "Implementing processor arrays on FPGAs". 1998.
  10. Zhoukun WANG and Omar HAMMAMI. "A 24 Processors System on Chip FPGA Design with Network on Chip".
  11. John Kent. "Micro16 Array - A Simple CPU Array"
  12. Kit Eaton. "1,000 Core CPU Achieved: Your Future Desktop Will Be a Supercomputer". 2011.
  13. "Scientists Squeeze Over 1,000 Cores onto One Chip". 2011.
  14. Hillis, W. Daniel; Steele, Jr., Guy L. (December 1986). "Data parallel algorithms". Communications of the ACM. 29 (12): 1170–1183. doi:10.1145/7902.7903.
  15. "A Survey of ReRAM-based Architectures for Processing-in-memory and Neural Networks", S. Mittal, Machine Learning and Knowledge Extraction, 2018
  16. 1 2 "Regular Expressions in hardware". Retrieved 17 July 2014.
  17. "Compression Accelerators - Microsoft Research". Microsoft Research. Retrieved 2017-10-07.
  18. 1 2 Farabet, Clément, et al. "Hardware accelerated convolutional neural networks for synthetic vision systems." Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on. IEEE, 2010.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.