backpropagation with multiple outputs

backpropagation with multiple outputs

Post by davidpierc » Wed, 23 Apr 2008 11:29:08


Hi All,
I am having some difficulty finding information about using more than
one output neuron with backpropagation. I am NOT a math guy and am
only a programming hobbyist. I have successfully implemented a NN
with one output using code based on Paras Chopra code. My training
function is in VB.NET below.

Public Function SupervisedTrain(ByVal inputdata() As Integer, ByVal
outputdata As Integer) As Integer '0=unsuccessful and 1 = sucessful
Dim x As Integer ' Loop through Nurons starting with 1
Dim y As Integer 'loop through Dendrites starting with 0 as
our Weighted bias
L is the number of neurons in the hidden Layer
L is the number of neurons in the Input Layer

Results = Think(inputdata) ' This sets initail values & output

'Calculate Delta of Output layer
OutputLayer.Delta = OutputLayer.Output * (1 -
OutputLayer.Output) * (outputdata - OutputLayer.Output)
'Calculate Delta of hidden layer
For x = 1 To HL
HiddenLayer1(x).Delta = HiddenLayer1(x).Output * (1 -
HiddenLayer1(x).Output) * OutputLayer.DendriteWeight(x) *
OutputLayer.Delta
Next


'Calculate new Weights
For y = 0 To HL 'calc weights for output dendrites
If y = 0 Then
OutputLayer.DendriteWeight(y) += (LearningRate * 1 *
OutputLayer.Delta) 'Bias Dendrite
Else
OutputLayer.DendriteWeight(y) += (LearningRate *
HiddenLayer1(y).Output * OutputLayer.Delta)
End If
Next

For x = 1 To HL ' loop through hidden layer Neurons
For y = 0 To IL 'Calc new weights for our hidden layer
dendrites
If y = 0 Then
HiddenLayer1(x).DendriteWeight(y) += (LearningRate
* 1 * HiddenLayer1(x).Delta) 'Bias Dendrite
Else
HiddenLayer1(x).DendriteWeight(y) += (LearningRate
* InputLayer(y).Output * HiddenLayer1(x).Delta)
End If
Next
Next

End Function

Now. The above works if I have 1 Output neuron ( Ie done the XOR). I
would like to implament multiple output. But don realy understand
the math. And most of the examples out there use only one output
neuron (and I looked extensivly before posting). I found
generation5.org usfull as it dumbed down things enough for me. Can
someone Point me in the right direction to some documentaion that is
in PLAIN english. Ie looked on this newsgroup and on the web and
have not found anything that has helped.
Thank you in advance