A highly scalable Restricted Boltzmann Machine FPGA implementation
- 1 August 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2523 (1946147X) , 367-372
- https://doi.org/10.1109/fpl.2009.5272262
Abstract
Restricted Boltzmann machines (RBMs)- the building block for newly popular deep belief networks (DBNs) - are a promising new tool for machine learning practitioners. However, future research in applications of DBNs is hampered by the considerable computation that training requires. In this paper, we describe a novel architecture and FPGA implementation that accelerates the training of general RBMs in a scalable manner, with the goal of producing a system that machine learning researchers can use to investigate ever-larger networks. Our design uses a highly efficient, fully-pipelined architecture based on 16-bit arithmetic for performing RBM training on an FPGA. We show that only 16-bit arithmetic precision is necessary, and we consequently use embedded hardware multiply-and-add (MADD) units. We present performance results to show that a speedup of 25-30X can be achieved over an optimized software implementation on a high-end CPU.Keywords
This publication has 9 references indexed in Scilit:
- A high-performance FPGA architecture for restricted boltzmann machinesPublished by Association for Computing Machinery (ACM) ,2009
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- FPGA Implementations of Neural Networks – A Survey of a Decade of ProgressPublished by Springer Nature ,2003
- A Hardware Random Number GeneratorPublished by Springer Nature ,2003
- Back propagation simulations using limited precision calculationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Piecewise linear approximation applied to nonlinear function of a neural networkIEE Proceedings - Circuits, Devices and Systems, 1997
- Fuzzy logic, neural networks, and soft computingCommunications of the ACM, 1994
- Artificial neural network implementation on a fine-grained FPGAPublished by Springer Nature ,1994
- GANGLION-a fast field-programmable gate array implementation of a connectionist classifierIEEE Journal of Solid-State Circuits, 1992