C++ Neural Networks and Fuzzy Logic:Backpropagation
Click Here! function GetCookie (name) { var arg = name + "="; var alen = arg.length; var clen = document.cookie.length; var i = 0; while (i < clen) { var j = i + alen; if (document.cookie.substring(i, j) == arg) { var end = document.cookie.indexOf (";", j); if (end == -1) end = document.cookie.length; return unescape(document.cookie.substring(j, end)); } i = document.cookie.indexOf(" ", i) + 1; if (i == 0) break; } return null; } var m1=''; var gifstr=GetCookie("UsrType"); if((gifstr!=0 ) && (gifstr!=null)) { m2=gifstr; } document.write(m1+m2+m3);           Keyword Title Author ISBN Publisher Imprint Brief Full  Advanced      Search  Search Tips Please Select ----------- Components Content Mgt Certification Databases Enterprise Mgt Fun/Games Groupware Hardware Intranet Dev Middleware Multimedia Networks OS Prod Apps Programming Security UI Web Services Webmaster Y2K ----------- New Titles ----------- Free Archive




To access the contents, click the chapter and section titles.


C++ Neural Networks and Fuzzy Logic


(Publisher: IDG Books Worldwide, Inc.)

Author(s): Valluru B. Rao

ISBN: 1558515526

Publication Date: 06/01/95










Search this book:
 





















Previous
Table of Contents
Next




The backprop.cpp file implements the simulator controls. First, data is accepted from the user for network parameters. Assuming Training mode is used, the training file is opened and data is read from the file to fill the IO buffer. Then the main loop is executed where the network processes pattern by pattern to complete a cycle, which is one pass through the entire training data set. (The IO buffer is refilled as required during this process.) After executing one cycle, the file pointer is reset to the beginning of the file and another cycle begins. The simulator continues with cycles until one of the two fundamental criteria is met:

1.  The maximum cycle count specified by the user is exceeded.
2.  The average error per pattern for the latest cycle is less than the error tolerance specified by the user.

When either of these occurs, the simulator stops and reports out the error achieved, and saves weights in the weights.dat file and one output vector in the output.dat file.

In Test mode, exactly one cycle is processed by the network and outputs are written to the output.dat file. At the beginning of the simulation in Test mode, the network is set up with weights from the weights.dat file. To simplify the program, the user is requested to enter the number of layers and size of layers, although you could have the program figure this out from the weights file.
Compiling and Running the Backpropagation Simulator
Compiling the backprop.cpp file will compile the simulator since layer.cpp is included in backprop.cpp. To run the simulator, once you have created an executable (using 80X87 floating point hardware if available), you type in backprop and see the following screen (user input in italic):


C++ Neural Networks and Fuzzy Logic
Backpropagation simulator
version 1
Please enter 1 for TRAINING on, or 0 for off:

Use training to change weights according to your
expected outputs. Your training.dat file should contain
a set of inputs and expected outputs. The number of
inputs determines the size of the first (input) layer
while the number of outputs determines the size of the
last (output) layer :

1

-> Training mode is *ON*. weights will be saved
in the file weights.dat at the end of the
current set of input (training) data
Please enter in the error_tolerance
-- between 0.001 to 100.0, try 0.1 to start --

and the learning_parameter, beta
-- between 0.01 to 1.0, try 0.5 to start --

separate entries by a space
example: 0.1 0.5 sets defaults mentioned :

0.2 0.25

Please enter the maximum cycles for the simulation
A cycle is one pass through the data set.
Try a value of 10 to start with
Please enter in the number of layers for your network.
You can have a minimum of three to a maximum of five.
three implies one hidden layer; five implies three hidden layers:

3

Enter in the layer sizes separated by spaces.
For a network with three neurons in the input layer,
two neurons in a hidden layer, and four neurons in the
output layer, you would enter: 3 2 4.
You can have up to three hidden layers for five maximum entries :

2 2 1

1 0.353248
2 0.352684
3 0.352113
4 0.351536
5 0.350954
...
299 0.0582381
300 0.0577085
------------------------
done: results in file output.dat
training: last vector only
not training: full cycle

weights saved in file weights.dat
-->average error per cycle = 0.20268 <--
-->error last cycle = 0.0577085 <--
->error last cycle per pattern= 0.0577085 <--
------>total cycles = 300 <--
------>total patterns = 300 <--



The cycle number and the average error per pattern is displayed as the simulation progresses (not all values shown). You can monitor this to make sure the simulator is converging on a solution. If the error does not seem to decrease beyond a certain point, but instead drifts or blows up, then you should start the simulator again with a new starting point defined by the random weights initializer. Also, you could try decreasing the size of the learning rate parameter. Learning may be slower, but this may allow a better minimum to be found.

This example shows just one pattern in the training set with two inputs and one output. The results along with the (one) last pattern are shown as follows from the file output.dat:



for input vector:
0.400000 -0.400000
output vector is:
0.842291
expected output vector is:
0.900000


The match is pretty good, as can be expected, since the optimization is easy for the network; there is only one pattern to worry about. Let’s look at the final set of weights for this simulation in weights.dat. These weights were obtained by updating the weights for 300 cycles with the learning law:



1 0.175039 0.435039
1 -1.319244 -0.559244
2 0.358281
2 2.421172


We’ll leave the backpropagation simulator for now and return to it in a later chapter for further exploration. You can experiment a number of different ways with the simulator:


•  Try a different number of layers and layer sizes for a given problem.
•  Try different learning rate parameters and see its effect on convergence and training time.
•  Try a very large learning rate parameter (should be between 0 and 1); try a number over 1 and note the result.





Previous
Table of Contents
Next






Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home Use of this site is subject to certain Terms & Conditions, Copyright © 1996-1999 EarthWeb Inc. All rights reserved. Reproduction whole or in part in any form or medium without express written permision of EarthWeb is prohibited.