FPGA Implementations of Neural Networks

1. Auflage 2006
ISBN: 978-0-387-28487-3
Verlag: Springer-Verlag
Format: PDF
Kopierschutz: Wasserzeichen (»Systemvoraussetzungen)

E-Book, Englisch, 360 Seiten

During the 1980s and early 1990s there was signi?cant work in the design and implementation of hardware neurocomputers. Nevertheless, most of these efforts may be judged to have been unsuccessful: at no time have have ha- ware neurocomputers been in wide use. This lack of success may be largely attributed to the fact that earlier work was almost entirely aimed at developing custom neurocomputers, based on ASIC technology, but for such niche - eas this technology was never suf?ciently developed or competitive enough to justify large-scale adoption. On the other hand, gate-arrays of the period m- tioned were never large enough nor fast enough for serious arti?cial-neur- network (ANN) applications. But technology has now improved: the capacity and performance of current FPGAs are such that they present a much more realistic alternative. Consequently neurocomputers based on FPGAs are now a much more practical proposition than they have been in the past. This book summarizes some work towards this goal and consists of 12 papers that were selected, after review, from a number of submissions. The book is nominally divided into three parts: Chapters 1 through 4 deal with foundational issues; Chapters 5 through 11 deal with a variety of implementations; and Chapter 12 looks at the lessons learned from a large-scale project and also reconsiders design issues in light of current and future technology.

Autoren/Hrsg.

Inhaltsverzeichnis

1;Contents;5
2;Preface;9
3;FPGA NEUROCOMPUTERS;13
3.1;1.1 Introduction;13
3.2;1.2 Review of neural-network basics;15
3.3;1.3 ASIC vs. FPGA neurocomputers;21
3.4;1.4 Parallelism in neural networks;24
3.5;1.5 Xilinx Virtex-4 FPGA;25
3.6;1.6 Arithmetic;27
3.7;1.7 Activation-function implementation: unipolar sigmoid;33
3.8;1.8 Performance evaluation;44
3.9;1.9 Conclusions;46
3.10;References;46
4;ON THE ARITHMETIC PRECISION FOR IMPLEMENTING BACK- PROPAGATION NETWORKS ON FPGA: A CASE STUDY;49
4.1;2.1 Introduction;49
4.2;2.2 Background;51
4.3;2.3 Architecture design and implementation;55
4.4;2.4 Experiments using logical-XOR problem;60
4.5;2.5 Results and discussion;62
4.6;2.6 Conclusions;67
4.7;References;68
5;FPNA: CONCEPTS AND PROPERTIES;74
5.1;3.1 Introduction;74
5.2;3.2 Choosing FPGAs;76
5.3;3.3 FPNAs, FPNNs;82
5.4;3.4 Correctness;97
5.5;3.5 Underparameterized convolutions by FPNNs;99
5.6;3.6 Conclusions;107
5.7;References;108
6;FPNA: APPLICATIONS AND IMPLEMENTATIONS;113
6.1;Introduction;113
6.2;4.1 Summary of Chapter 3;114
6.3;4.2 Towards simplified architectures: symmetric boolean functions by FPNAs;115
6.4;4.3 Benchmark applications;119
6.5;4.4 Other applications;123
6.6;4.5 General FPGA implementation;126
6.7;4.6 Synchronous FPNNs;130
6.8;4.7 Implementations of synchronous FPNNs;134
6.9;4.8 Implementation performances;140
6.10;4.9 Conclusions;143
6.11;References;144
7;BACK-PROPAGATION ALGORITHM ACHIEVING 5 GOPS ON THE VIRTEX-E;147
7.1;5.1 Introduction;148
7.2;5.2 Problem specification;149
7.3;5.3 Systolic implementation of matrix-vector multiply;151
7.4;5.4 Pipelined back-propagation architecture;152
7.5;5.5 Implementation;154
7.6;5.6 MMAlpha design environment;157
7.7;5.7 Architecture derivation;159
7.8;5.8 Hardware generation;165
7.9;5.9 Performance evaluation;167
7.10;5.10 Related work;169
7.11;5.11 Conclusion;170
7.12;Appendix;171
7.13;References;173
8;FPGA IMPLEMENTATION OF VERY LARGE ASSOCIATIVE MEMORIES;176
8.1;6.1 Introduction;176
8.2;6.2 Associative memory;177
8.3;6.3 PC Performance Evaluation;188
8.4;6.4 FPGA Implementation;193
8.5;6.5 Performance comparisons;199
8.6;6.6 Summary and conclusions;201
8.7;References;202
9;FPGA IMPLEMENTATIONS OF NEOCOGNITRONS;205
9.1;7.1 Introduction;205
9.2;7.2 Neocognitron;206
9.3;7.3 Alternative neocognitron;209
9.4;7.4 Reconfigurable computer;213
9.5;7.5 Reconfigurable orthogonal memory multiprocessor;214
9.6;7.6 Alternative neocognitron hardware implementation;217
9.7;7.7 Performance analysis;223
9.8;7.8 Applications;226
9.9;7.9 Conclusions;229
9.10;References;230
10;SELF ORGANIZING FEATURE MAP FOR COLOR QUANTIZATION ON FPGA;233
10.1;8.1 Introduction;233
10.2;8.2 Algorithmic adjustment;236
10.3;8.3 Architecture;239
10.4;8.4 Implementation;243
10.5;8.5 Experimental results;247
10.6;8.6 Conclusions;250
10.7;References;250
11;IMPLEMENTATION OF SELF-ORGANIZING FEATURE MAPS IN RECONFIGURABLE HARDWARE;254
11.1;9.1 Introduction;254
11.2;9.2 Using reconfigurable hardware for neural networks;255
11.3;9.3 The dynamically reconfigurable rapid prototyping system RAPTOR2000;257
11.4;9.4 Implementing self-organizing feature maps on RAPTOR2000;259
11.5;9.5 Conclusions;274
11.6;References;274
12;FPGA IMPLEMENTATION OF A FULLY AND PARTIALLY CONNECTED MLP;277
12.1;10.1 Introduction;277
12.2;10.2 MLP/XMLP and speech recognition;279
12.3;10.3 Activation functions and discretization problem;282
12.4;10.4 Hardware implementations of MLP;290
12.5;10.5 Hardware implementations of XMLP;297
12.6;10.6 Conclusions;299
12.7;Acknowledgments;300
12.8;References;301
13;FPGA IMPLEMENTATION OF NON-LINEAR PREDICTORS;303
13.1;11.1 Introduction;304
13.2;11.2 Pipeline and back-propagation algorithm ;305
13.3;11.3 Synthesis and FPGAs ;310
13.4;11.4 Implementation on FPGA;319
13.5;11.5 Conclusions;325
13.6;References;327
14;THE REMAP RECONFIGURABLE ARCHITECTURE: A RETROSPECTIVE;330
14.1;12.1 Introduction;331
14.2;12.2 Target Application Area;332
14.3;12.3 REMAP-ß – design and implementation;340
14.4;12.4 Neural networks mapped on REMAP-ß;351
14.5;12.5 REMAP- . architecture;358
14.6;12.6 Discussion;359
14.7;12.7 Conclusions;362
14.8;Acknowledgments;362
14.9;References;362