Eckmiller Advanced Neural Computers

1. Auflage 2014
ISBN: 978-1-4832-9427-8
Verlag: Elsevier Science & Techn.
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 464 Seiten, Web PDF

This book is the outcome of the International Symposium on Neural Networks for Sensory and Motor Systems (NSMS) held in March 1990 in the FRG. The NSMS symposium assembled 45 invited experts from Europe, America and Japan representing the fields of Neuroinformatics, Computer Science, Computational Neuroscience, and Neuroscience.As a rapidly-published report on the state of the art in Neural Computing it forms a reference book for future research in this highly interdisciplinary field and should prove useful in the endeavor to transfer concepts of brain function and structure to novel neural computers with adaptive, dynamical neural net topologies.A feature of the book is the completeness of the references provided. An alphabetical list of all references quoted in the papers is given, as well as a separate list of general references to help newcomers to the field. A subject index and author index also facilitate access to various details.

Autoren/Hrsg.

Eckmiller, R.

Inhaltsverzeichnis

1;Front Cover;1
2;Advanced Neural Computers;4
3;Copyright Page;5
4;Table of Contents;8
5;PREFACE;6
6;ACKNOWLEDGEMENT OF SPONSORSHIP;7
7;Section 1: General Introduction;12
7.1;Chapter 1. Prolegomena to an Analysis of Form and Structure;14
7.1.1;I. INTRODUCTION;14
7.1.2;II.
Preliminary Remarks;16
7.1.3;Ill.
Information as Kullback-Leibler Entroou;17
7.1.4;References;20
7.2;Chapter 2. The Truck Backer-Upper: An Example of Self-Learning in Neural Networks;22
7.2.1;1 Introduction;22
7.2.2;2 Training;23
7.2.3;3 Summary and Results;26
7.2.4;References;27
7.3;Chapter 3. DIE LERNMATRIX - THE BEGINNING OF ASSOCIATIVE MEMORIES;32
7.3.1;1. INTRODUCTION;32
7.3.2;2. BINARY LEARNING MATRIX;33
7.3.3;3. NON-BINARY LEARNING MATRIX;35
7.3.4;4. CIRCUITS WITH SEVERAL LEARNING MATRICES;37
7.3.5;5. APPLICATIONS;39
7.3.6;References;40
8;Section 2: Biological Sensory and Motor Systems;42
8.1;Chapter 4. MODEL OF VISUO-MOTOR TRANSFORMATIONS PERFORMED BY THE CEREBRAL CORTEX TO COMMAND ARM MOVEMENTS AT VISUAL TARGETS IN THE 3-D SPACE;44
8.1.1;I. BIOLOGICAL CONSTRAINTS;44
8.1.2;II. INVARIANT PROPERTIES OF MOTOR CORTICAL NEURONS;45
8.1.3;III. SENSORIMOTOR COMBINATIONS IN CORTICAL AREAS;46
8.1.4;IV. PROCESSING UNITS: MODEL OF THE CORTICAL COLUMN;47
8.1.5;V. CORTICAL REPRESENTATIONS OF MUSCLES AND 3-D SPACE;48
8.1.6;VI. MOVEMENT DEPENDENT MATCHING OF SENSORY INPUTS;49
8.1.7;VII. GENERALIZATION OF LEARNING;50
8.1.8;VIII. PREDICTIONS OF THE MODEL : INVARIANT PROPERTIES;51
8.2;Chapter 5.
NEURAL NETWORK MODELS OF THE PRIMATE MOTOR SYSTEM;54
8.2.1;1.INTRODUCTON: PHYSIOLOGICAL NETWORKS;54
8.2.2;2. DYNAMIC NETWORK MODELS;55
8.2.3;3. MANIPULATON OF HIDDEN UNITS;59
8.2.4;4. CONCLUDING COMMENTS;60
8.2.5;ACKNOWLEDGEMENTS;60
8.2.6;REFERENCES;60
8.3;Chapter 6. Neural Representation of Motor Synergies;62
8.3.1;1. Introduction;62
8.3.2;2. Motor Relaxation and Hopfield Networks;62
8.3.3;3. Behaviour of the Network;64
8.3.4;4. Simulation Results;66
8.3.5;References;67
8.3.6;Acknowledgments;67
8.3.7;Appendix: M-net of the finger;70
8.4;Chapter 7. Vestibular Head- Eye Coordination: a Geometrical Sensorimotor Neurocomputer Paradigm;72
8.4.1;1. Introduction;72
8.4.2;2. Vestlbulo-Cerebellar Coordination as a Sensorlnrwtor Neurocomputing Paradigm;73
8.4.3;3.Skeletomuscular Systems' Modeling: Btomechanics of Intrinsk: Coordinates;77
8.4.4;4. Development of Neural Network Theory & Experimentation;77
8.4.5;5. Advanced Experimental Paradigms: Linear Vestibulo-Ocular Reflex; Gaussian Testing;77
8.4.6;6. References;78
8.5;Chapter 8. THE REPRESENTATION OF INFORMATION IN THE TEMPORAL LOBE VISUAL CORTICAL AREAS OF MACAQUES;80
8.5.1;Ensemble encoding of object identity;81
8.5.2;The development of specificity of the neuronal responses;81
8.5.3;A Neuronal Representation Showing Invariance;82
8.5.4;An Object-Centered Representation of Visual Information;83
8.5.5;The functions of backprojections in neuronal networks in the neocortex in the storage and recall of visual memories;83
8.5.6;REFERENCES;87
8.6;Chapter 9. Exploration of a natural environment;90
8.6.1;1 Introduction;90
8.6.2;2 The system architecure;91
8.6.3;3 Modules of the exploration system;92
8.6.4;4 Conclusion;97
8.6.5;References;97
8.7;Chapter 10. MODELING VISUAL CORTEX: HIDDEN ANISOTROPIES IN AN ISOTROPIC INHIBITORY CONNECTION SCHEME;98
8.7.1;1. Introduction;98
8.7.2;2.1. A Detailed Model of the Visual Cortex - Methods;98
8.7.3;2.2. A Detailed Model of the Visual Cortex - Results;100
8.7.4;3.2. Circular inhibition in a real cortex;102
8.7.5;4. Directional bias;103
8.7.6;5. Discussion;104
8.7.7;References;104
9;Section 3: Theory of
Artificial Neural Networks;106
9.1;Chapter 11. THE LOGIG OF NEURAL COGNITION;108
9.1.1;1. INTRODUCTION;108
9.1.2;2. A COGNITIVE ARCHITECTURE;108
9.1.3;3. THE SENSORY NEURAL LEVEL;109
9.1.4;4. THE COGNITIVE PROCESSING LEVEL;110
9.1.5;5. IMPLEMENTATION ISSUES AND CONCLUSIONS;113
9.1.6;REFERENCES;113
9.2;Chapter 12.
Non-Lipschitzian Neural Dynamics;114
9.2.1;ABSTRACT;114
9.2.2;1. Introduction;114
9.2.3;2. Unpredictable Neural Nets;116
9.2.4;3. Application: Modeling Brain Activity;120
9.2.5;Acknowledgement;123
9.2.6;References;123
9.3;Chapter 13.
GEMINI: A PERCEPTRON-LIKE NEURAL NETWORK WITH BIOLOGICAL SIMILARITIES;124
9.3.1;1. INTRODUCTION;124
9.3.2;2. CLASSICAL CONDITIONING;125
9.3.3;3. THE NEW TYPE OF NETWORK;126
9.3.4;4. SPONTANEOUS RECOVERY OF DISCRIMINATION;128
9.3.5;5. CONCLUSIONS;130
9.3.6;REFERENCES;130
9.4;Chapter 14. ON THE HEBB RULE AND UNLIMITED PRECISION ARITHMETIC IN A Mc CULLOCH AND PITTS NETWORK;132
9.4.1;1. INTRODUCTION;132
9.4.2;2. THE FAULT TOLERANT NETWORK AND ITS ASSOCIATED LANGUAGE;133
9.4.3;3. ON THE HEBB RULE AND THE INHIBITORY CONNECTIONS;135
9.4.4;4. ON UNLIMITED PRECISION ARITHMETIC;136
9.4.5;ACKNOWLEDGEMENTS;139
9.4.6;REFERENCES;139
9.5;Chapter 15. ON THE ALGEBRAIC STRUCTURE OF FEEDFORWARD NETWORK WEIGHT SPACES;140
9.5.1;Abstract;140
9.5.2;1 Introduction;140
9.5.3;2 The Algebra of Weight Equivalence;141
9.5.4;3 Weight Space Symmetries;143
9.5.5;4 Discussion;145
9.5.6;References;146
9.6;Chapter 16. STATISTICAL PATTERN RECOGNITION REVISITED;148
9.6.1;1. Introduction;148
9.6.2;2. Initialization of the Codebook Vectors;149
9.6.3;3. The First Version of Learning Vector Quantization (LVQl);150
9.6.4;4. TheLVQ2;151
9.6.5;5. Instabilities in the Basic LVQ2;151
9.6.6;6. TheLVQS;152
9.6.7;7. Taking More "Runners-up" into Account;153
9.6.8;8. Experiments with Speech Data;153
9.6.9;9. Conclusions;154
9.6.10;References;155
9.7;Chapter 17. LOCAL LEARNING RULES AND SPARSE CODING IN NEURAL NETWORKS;156
9.7.1;1. INTRODUCTION;156
9.7.2;2. HETERO-ASSOCIATION;157
9.7.3;3. HIGH FIDELITY;158
9.7.4;4. AUTO-ASSOCIATION;160
9.7.5;5. CONCLUSION: SPARSE CODING IS NECESSARY;160
9.7.6;ACKNOWLEDGEMENTS;161
9.7.7;REFERENCES;161
9.8;Chapter 18.
Speeding up Backpropagation;162
9.8.1;ABSTRACT;162
9.8.2;1. INTRODUCTION;162
9.8.3;2. RESCALING AND RAVINES;163
9.8.4;3. STEP SIZE ADAPTATION;164
9.8.5;4. SOME IMPLEMENTATION DETAILS;165
9.8.6;5. EXPERIMENTAL RESULTS;166
9.8.7;6. DISCUSSION AND CONCLUSIONS;169
9.8.8;REFERENCES;169
10;Section 4:
Neural Network Simulators;170
10.1;Chapter 19. VLSI IMPLEMENTATION OF SENSORY PROCESSING SYSTEMS;172
10.1.1;1. INTRODUCTION;172
10.1.2;2. A LOCAL TRAINING RULE FOR DYNAMIC SCENE ANALYSIS;172
10.1.3;3. IGORS - A VLSI SMART SENSOR;174
10.1.4;4. CONCLUSION;175
10.1.5;ACKNOWLEDGEMENTS;175
10.1.6;REFERENCES;175
10.2;Chapter 20. THE PYGMALION NEURAL NETWORK PROGRAMMING ENVIRONMENT;178
10.2.1;1. INTRODUCTON;178
10.2.2;2. THE NEURAL NETWORK PROGRAMMING SYSTEM (NNPS);178
10.2.3;3. HARDWARE INTEGRATON;184
10.2.4;4. CONCLUSION;186
10.3;Chapter 21. SIMULATORS FOR NEURAL NETWORKS;188
10.3.1;1. INTRODUCTION;188
10.3.2;2. NEURAL NETWORK REQUIREMENTS;188
10.3.3;3. ARCHITECTURE OF A NEURAL NETWORK SIMULATOR;189
10.3.4;4. THE NEXT STAGE;190
10.3.5;5. OUTLINE OF AVAILABLE SYSTEMS;193
10.3.6;6. CASE STUDY - LOCATION OF EYES IN A FACIAL IMAGE;193
10.3.7;7. CONCLUSION;193
10.3.8;REFERENCES;194
10.4;Chapter 22. Dynamics and VLSI Implementation of Self–Organizing Networks;196
10.4.1;1 Introduction;196
10.4.2;2 Scaling and Feature Discovery;197
10.4.3;3 Self-Organizing Biological Networks and VLSI;200
10.4.4;References;203
10.5;Chapter 23.
HARDWARE FOR NEURAL·NET OVTICAL CHARACTER RECOGNTTION;204
10.5.1;1. INTRODUCnON;204
10.5.2;2. THE CHARACTER RECOGNTHON TASK;205
10.5.3;3. FEATURE EXTRACHON;205
10.5.4;4. A PRE-PROGRAMMED FEATURE EXTRACTION SYSTEM;206
10.5.5;5. A DIGIT RECOGNIZER FOR LEARNED FEATURE EXTRACTION;207
10.5.6;6. COMMON CHARACTERISTOS OF THE RECOGNIZER NETWORKS;209
10.5.7;7. AN ADVANCED NEURAL·NET CHIP FOR MACHINE VISION;209
10.5.8;8. CONCLUSIONS;211
10.5.9;ACKNOWLEDGEMENT;211
10.5.10;REFERENCES;211
10.6;Chapter 24. THE ROLE OF TINE IN NATURAL INTELLIGENCE: IMPLICATIONS FOR NEURAL NETWORK AND ARTIFICIAL INTELLIGENCE RESEARCH;212
10.6.1;1. INTRODUCTION;212
10.6.2;2. REAL-TIME LEARNING MECHANISM MODELS;213
10.6.3;3. EXPERIMENTAL TESTS;215
10.6.4;4. CONCLUSIONS;216
10.6.5;REFERENCES;216
10.7;Chapter 25. HARDWARE CONCEPTS FOR NEURAL NETWORKS;220
10.7.1;1.
INTRODUCTION;220
10.7.2;2. SINGLE-CHIP INTEGRATION OF NEURAL NETS;221
10.7.3;3.
CHIP INTEGRATION OF COMPUTE-BOUND NEURAL ALGORITHMS;223
10.7.4;4.
CONCLUSIONS;227
10.7.5;ACKNOWLEDGEMENTS;228
10.7.6;REFERENCES;228
10.8;Chapter 26. Rapid Prototyping for Neural Networks;230
10.8.1;Abstract;230
10.8.2;1 Introduction;230
10.8.3;2 TInMANN Algorithm and Architecture;231
10.8.4;3 Rapid Prototyping of TInMANN;234
10.8.5;References;236
11;Section 5: Pattem Recognition with Neural Networks;238
11.1;Chapter 27. ANIMATE VISION USES OBJECTT-CENTERED REFERENCE FRAMES;240
11.1.1;1. VISION AS BEHAVIOR;240
11.1.2;2. USING THE FIXATION FRAME;241
11.1.3;3. RELATIV. VISION;242
11.1.4;4. LEARNING COORDINATED BEHAVIORS;243
11.1.5;5. CONCLUSIONS;245
11.1.6;Acknowledgements;246
11.1.7;References;246
11.2;Chapter 28. A Performance Evaluation of ALIAS for the Detection of Geometric Anomalies on Fractal Images;248
11.2.1;1. PROJECT ALIAS;248
11.2.2;2. EXPERIMENT PROCEDURE AND PERFORMANCE MEASURES;250
11.2.3;3. GEOMETRIC ANOMALIES EXPERIMENT;252
11.2.4;4. CONCLUSIONS;256
11.2.5;REFERENCES;257
11.3;Chapter 29. HOW CONNECTIONIST MODELS COULD IMPROVE MARKOV MODELS FOR SPEECH RECOGNITION;258
11.3.1;Abstract;258
11.3.2;1. Introduction;258
11.3.3;2. Hidden Markov Models;259
11.3.4;3. Connectionist models and time sequential inputs;261
11.3.5;4. Hybrid approaches: HMM + MLP;262
11.3.6;5. Another hybrid approach;263
11.3.7;6. Conclusions;264
11.3.8;References;265
11.4;Chapter 30. AN ALGEBRAIC APPROACH TO BOOLEAN FEEDFORWARD NETWORKS;266
11.4.1;ABSTRACT;266
11.4.2;1 INTRODUCTION;266
11.4.3;2 AN ALGEBRAIC FRAMEWORK FOR BOOLEAN FEEDFORWARD NETWORKS;268
11.4.4;3 A GEOMETRICAL SEMANTICS OF LAYERED NETWORK ACTIVITY;270
11.4.5;4 LEARNING IN BOOLEAN FEEDFORWARD NETWORKS;271
11.4.6;5 CONCLUSIONS;273
11.4.7;6 REFERENCES;273
11.4.8;7 ACKNOWLEDGEMENTS;273
11.5;Chapter 31. ALPHANUMERIC CHARACTER RECOGNITION BY THE NEOCOGNITRON;274
11.5.1;1. INTRODUCTION;274
11.5.2;2. THE STRUCTURE OF THE NETWORK;275
11.5.3;3. TRAINING THE NETWORK TO RECOGNIZE HANDWRITTEN ALPHANUMERIC CHARACTERS;276
11.5.4;4. DISCUSSION;281
11.5.5;REFERENCES;281
11.6;Chapter 32. STORING AND PROCESSING INFORMATION IN CONNECTIONIST SYSTEMS;282
11.6.1;1.0 Introduction;282
11.6.2;2.0 The model;284
11.6.3;3.0 Conclusion;288
11.6.4;REFERENCES;288
11.7;Chapter 33. THE CLOSED LOOP ANTAGONISTIC NETWORK (CLAN);290
11.7.1;1. INTRODUCTION;290
11.7.2;2. A QUALITY MEASURE FOR PATTERN MATCHING;290
11.7.3;3. EVALUATION OF THE QUALITY MEASURE BY A NEURON;291
11.7.4;4. THE CLOSED LOOP ANTAGONISTIC NETWORK (CLAN);293
11.7.5;5· LEARNING AND ASSOCIATIVE RECALL IN A CLAN;295
11.7.6;6. CONCLUSION;296
11.7.7;REFERENCES;296
11.8;Chapter 34. Adaptive Resonance Structures in Hierarciiical Receptive Fieid Pattern Recognition Machines;298
11.8.1;Abstract;298
11.8.2;Three ART Architectures;298
11.8.3;Hierarchical Receptive Field Architectures;300
11.8.4;ART and Receptive Fields;301
11.8.5;Selective Attention Neocognitron (SAN);301
11.8.6;The SAN and Hierarchical ART;302
11.8.7;Conclusions;305
11.8.8;References;305
11.9;Chapter 35.
RECEPTIVE FIELD TAXONOMY;306
11.9.1;1 INTRODUCTION;306
11.9.2;2 BASIC ASSUMPTIONS;306
11.9.3;3 RECEPTIVE FIELD STRUCTURE;307
11.9.4;4 SPECIFIC FORMS;308
11.9.5;5 TRANSFORMATIONS AND COMBINATIONS;311
11.9.6;6 BRAIN CIRCUITRY;311
11.9.7;Acknowledgements;311
11.9.8;References;312
11.10;Chapter 36. CONSIDERATIONS FOR A VISUAL ARCHITECTURE;314
11.10.1;THE VISUAL PROCESS;314
11.10.2;THE ARCHITECTURE APPROACH;317
11.10.3;DATA STRUCTURE;317
11.10.4;ORGANIZATION;321
11.10.5;References;322
11.11;Chapter 37. NEURAL MODELLING OF VISION AND OLFACTION;324
11.11.1;1. INTRODUCTION;324
11.11.2;2. THE RETINA;325
11.11.3;3. THE PRIMARY VISUAL CORTEX;327
11.11.4;4. OLFACTION;328
11.11.5;5. HIPPOCAMPUS;330
11.11.6;6. CONCLUSIONS;332
11.11.7;7. ACKNOWLEDGEMENTS;332
11.11.8;8. REFERENCES;332
11.12;Chapter 38. Neural Computers for Foveating Vision Systems;334
11.12.1;1. Introduction;334
11.12.2;2. Foveating Vision Systems;334
11.12.3;3. A Foveating Sensor;336
11.12.4;4. A Foveating Image Acquisition System;336
11.12.5;5. Example of a Foveated Image;337
11.12.6;6. Image Processing of Foveated Images;338
11.12.7;7. Viewing Foveated Images;339
11.12.8;8. Temporal Foveation Control Method;340
11.12.9;References;341
11.13;Chapter 39.
On the Neural Computations Underiying Curve Detection;342
11.13.1;1. The Dilemma of Curve Detection;342
11.13.2;2. Two Stages of Curve Detection;342
11.13.3;3. The Model of Curve Detection;343
11.13.4;4. References;347
12;Section 6: Motor Control with Neural Networks;350
12.1;Chapter 40.
Network Based Autonomous Robot Motor Control: from Hormones to Leaming;352
12.1.1;Abstract;352
12.1.2;1 Introduction;352
12.1.3;2 Hormonal Control;353
12.1.4;3 A Robot Implementation;354
12.1.5;4 Critique;357
12.1.6;5 Grounding Out in Motor Control;357
12.1.7;6 References;358
12.2;Chapter 41. SPINAL NETWORK COMPUTATIONS ENABLE INDEPENDENT CONTROL OF MUSCLE LENGTH AND JOINT COMPLIANCE;360
12.2.1;1. AN OPPONENT NEUROMUSCULAR DESIGN FOR INDEPENDENT CONTROL OF MUSCLE LENGTH AND MUSCLE TENSION;360
12.2.2;2. WIDE FORCE RANGE AT EACH MUSCLE LENGTH REQUIRES SIZE PRINCIPLE;360
12.2.3;3. SIZE PRINCIPLE WITH CO-CONTRACTION THREATEN POSITION-CODE INVARIANCE;362
12.2.4;4. AUTOMATIC COMPENSATION BY THE RENSHAW-IalN PATHWAY FOR UNEQUAL AMPLIFICATIONS OF CO-CONTRACTIVE SIGNALS;362
12.2.5;5. LEARNED AND REACTIVE COMPENSATIONS FOR VARIABLE MOMENT-ARMS USING SPINDLE ORGAN ERROR SIGNALS;364
12.2.6;6. CO-CONTRACTIVE AND STRETCH FEEDBACK CONTROL OF LOAD COMPENSATION;366
12.2.7;REFERENCES;367
12.3;Chapter 42. Neural Computers for Motor Control;368
12.3.1;Biological Motor Control;368
12.3.2;Neural Networks for Control of a Redundant Robot Arm;371
12.3.3;References;374
12.4;Chapter 43. Feedback-Error-Learning Neural Network for Supervised Motor Learning;376
12.4.1;Abstract;376
12.4.2;1 Introduction;376
12.4.3;2 Computational schemes to convert trajectory error into motor command error;377
12.4.4;3 Feedback-error-learning as a Newton-like method in functional space;379
12.4.5;4 Stability of feedback-error-learning scheme;380
12.4.6;5 Feedback-error-learning neural network as models of different parts of cerebellum;382
12.4.7;Ackuowledgment;382
12.4.8;References;382
12.5;Chapter 44. DAS LERNFAHRZEUG NEURAL NETWORK APPLICATION FOR AUTONOMOUS MOBILE ROBOTS;384
12.5.1;SUMMARY;384
12.5.2;TARGET;384
12.5.3;SYSTEM ARCHITECTURE;385
12.5.4;PROCESS OF NEURAL APPLICATION SIMULATION;386
12.5.5;LERNFAHRZEUG I;387
12.5.6;LERNFAHRZEUG II;388
12.5.7;ADVANCED CONCEPTS FOR NEURAL NETWORKS;389
12.6;Chapter 45. Motor Learning by "Charge'' Placement with Self-organizing Maps;392
12.6.1;1 Introduction;392
12.6.2;2 The "Method of Charges";393
12.6.3;3 Learning to Recognize Stable Grasp Points;394
12.6.4;4 Conclusion;398
12.6.5;References;398
13;List of General References;400
14;REFERENCES FROM ALL CONTRIBUTIONS;402
15;List of Contributors;446
16;AUTHOR INDEX;454
17;SUBJECT INDEX;462