Backpropagation-Netzwerk (DAN+WI Übung 4 Aufgabe 4)

Konvergiert nicht bei mir, hat aber möglicherweise noch Fehler. Ausserdem ist es noch limitiert auf einen OutputNode und nur einen HiddenLayer...

Code (auch für SingleLayer) ist jetzt auf github: https://github.com/Berdir/danwi

Falls ihr Fehler findet, bitte forken und korrigieren :)

Erstellen des Netzwerks:

InputNode x1 = new InputNode("x1", 1);
InputNode x2 = new InputNode("x2", 1);
Node h1 = new Node("h1");
Node h2 = new Node("h2");
OutputNode y = new OutputNode("y");
h1.link(1.5);
h2.link(0.5);
h1.link(x1, -1);
h1.link(x2, -1);
h2.link(x1, -1);
h2.link(x2, -1);
y.link(-0.5);
y.link(h1, 1);
y.link(h2, -1);
bool converged = false;
int iterations = 0;
while (!converged && iterations < 10) {
Console.WriteLine("======== ITERATION {0} =========", ++iterations);
y.forwardCalculate();
converged = y.backwardSweep(1, 0.3, 0);
}


Ausgabe:

Connected 1 with h1 (Weight: 1.5)
Connected 1 with h2 (Weight: 0.5)
Connected x1 with h1 (Weight: -1)
Connected x2 with h1 (Weight: -1)
Connected x1 with h2 (Weight: -1)
Connected x2 with h2 (Weight: -1)
Connected 1 with y (Weight: -0.5)
Connected h1 with y (Weight: 1)
Connected h2 with y (Weight: -1)
======== ITERATION 1 =========
Calculating h1 = 1 (1) * 1.5 + 1 (x1) * -1 + 1 (x2) * -1 = -0.5 -> 0
Calculating h2 = 1 (1) * 0.5 + 1 (x1) * -1 + 1 (x2) * -1 = -1.5 -> 0
Calculating y = 1 (1) * -0.5 + 0 (h1) * 1 + 0 (h2) * -1 = -0.5 -> 0
Expected 1, got -0.5, difference 1.5 for y
Delta = Result * (1 - Result) * (expectedValue - Result) = -0.5 * (1 - -0.5) * (1 - -0.5) = -1.125
Calculating new weight for link to 1 weight -0.5 += learningRate * Result * leftNodeResult = 0.3 * -0.5 * 1 = -0.15 => -0.65
Calculating new weight for link to h1 weight 1 += learningRate * Result * leftNodeResult = 0.3 * -0.5 * -0.5 = 0.075 => 1.075
Calculating new weight for link to h2 weight -1 += learningRate * Result * leftNodeResult = 0.3 * -0.5 * -1.5 = 0.225 => -0.775
Delta = Result * (1 - Result) * oldWeight * rightResult = 1 * (1 - 1) * 0 * -0.5 = 0
Delta = Result * (1 - Result) * oldWeight * rightResult = -0.5 * (1 - -0.5) * 0 * -0.5 = 0
Calculating new weight for link to 1 weight 1.5 += learningRate * Result * leftNodeResult = 0.3 * -0.5 * 1 = -0.15 => 1.35
Calculating new weight for link to x1 weight -1 += learningRate * Result * leftNodeResult = 0.3 * -0.5 * 1 = -0.15 => -1.15
Calculating new weight for link to x2 weight -1 += learningRate * Result * leftNodeResult = 0.3 * -0.5 * 1 = -0.15 => -1.15
Delta = Result * (1 - Result) * oldWeight * rightResult = -1.5 * (1 - -1.5) * 0 * -0.5 = 0
Calculating new weight for link to 1 weight 0.5 += learningRate * Result * leftNodeResult = 0.3 * -1.5 * 1 = -0.45 => 0.05
Calculating new weight for link to x1 weight -1 += learningRate * Result * leftNodeResult = 0.3 * -1.5 * 1 = -0.45 => -1.45
Calculating new weight for link to x2 weight -1 += learningRate * Result * leftNodeResult = 0.3 * -1.5 * 1 = -0.45 => -1.45
======== ITERATION 2 =========
Calculating h1 = 1 (1) * 1.35 + 1 (x1) * -1.15 + 1 (x2) * -1.15 = -0.95 -> 0
Calculating h2 = 1 (1) * 0.05 + 1 (x1) * -1.45 + 1 (x2) * -1.45 = -2.85 -> 0
Calculating y = 1 (1) * -0.65 + 0 (h1) * 1.075 + 0 (h2) * -0.775 = -0.65 -> 0
Expected 1, got -0.65, difference 1.65 for y
Delta = Result * (1 - Result) * (expectedValue - Result) = -0.65 * (1 - -0.65) * (1 - -0.65) = -1.769625
Calculating new weight for link to 1 weight -0.65 += learningRate * Result * leftNodeResult = 0.3 * -0.65 * 1 = -0.195 => -0.845
Calculating new weight for link to h1 weight 1.075 += learningRate * Result * leftNodeResult = 0.3 * -0.65 * -0.95 = 0.18525 => 1.26025
Calculating new weight for link to h2 weight -0.775 += learningRate * Result * leftNodeResult = 0.3 * -0.65 * -2.85 = 0.55575 => -0.21925
Delta = Result * (1 - Result) * oldWeight * rightResult = 1 * (1 - 1) * 0 * -0.65 = 0
Delta = Result * (1 - Result) * oldWeight * rightResult = -0.95 * (1 - -0.95) * 0 * -0.65 = 0
Calculating new weight for link to 1 weight 1.35 += learningRate * Result * leftNodeResult = 0.3 * -0.95 * 1 = -0.285 => 1.065
Calculating new weight for link to x1 weight -1.15 += learningRate * Result * leftNodeResult = 0.3 * -0.95 * 1 = -0.285 => -1.435
Calculating new weight for link to x2 weight -1.15 += learningRate * Result * leftNodeResult = 0.3 * -0.95 * 1 = -0.285 => -1.435
Delta = Result * (1 - Result) * oldWeight * rightResult = -2.85 * (1 - -2.85) * 0 * -0.65 = 0
Calculating new weight for link to 1 weight 0.05 += learningRate * Result * leftNodeResult = 0.3 * -2.85 * 1 = -0.855 => -0.805
Calculating new weight for link to x1 weight -1.45 += learningRate * Result * leftNodeResult = 0.3 * -2.85 * 1 = -0.855 => -2.305
Calculating new weight for link to x2 weight -1.45 += learningRate * Result * leftNodeResult = 0.3 * -2.85 * 1 = -0.855 => -2.305