/*
1. PREPARATION STEPS
*/
Random, Weight1, -4.0, 4.0 ; We start by initializing random numbers into the weight variables (this simulates a first hipotesis of a solution and allows the beggining of the training).
Random, Weight2, -4.0, 4.0
Random, Weight3, -4.0, 4.0
WEIGHTS := Array([Weight1],[Weight2],[Weight3]) ; And than organize them into a matrix.
TRAINING_INPUTS := Array([0,0,1],[1,1,1],[1,0,1],[0,1,1]) ; We will also feed the net creator code with the values of the inputs in the training samples (all organized in a matrix too).
EXPECTED_OUTPUTS := Array([0],[1],[1],[0]) ; And we will also provide the net creator with the expected answers to our training samples so that the net creator can properly train the net.
/*
2 . ACTUAL TRAINING
*/
Loop 10000 ; And now we do the net creator code (which is the training code). It will perform 10.000 training cycles.
{
Loop 4 ; For each training cycle, this net creator code will train the network of weights using the four training samples.
{
ACQUIRED_OUTPUT := 1 / (1 + exp(-1 * MATRIX_ROW_TIMES_COLUMN_MULTIPLY(TRAINING_INPUTS, WEIGHTS, A_Index))) ; First, the net is set to calculate some possible results using the weights we currently have. (At the first iteration of the loop these weights are absolutely random, but don't forget they will be recalculated every time). We use a sigmoid function here to set any results (from -infinite to +infinite) to a value between 0 and 1.
SIGMOID_GRADIENT := ACQUIRED_OUTPUT * (1 - ACQUIRED_OUTPUT) ; But since the sigmoid function has a curve like shape, the distance between values is highly distorted depending on the position they occupy in the S-shaped curve, so we will also use the sigmoids gradient function to correctly account for that (This is to find better pictures of the actual distances between values while still keeping the results between 0 and 1).
WEIGHTS[1,1] += TRAINING_INPUTS[A_Index, 1] * ((EXPECTED_OUTPUTS[A_Index, 1] - ACQUIRED_OUTPUT) * SIGMOID_GRADIENT) ; Than, each weight is recalculated using the available knowledge in the samples and also the current calculated results.
WEIGHTS[2,1] += TRAINING_INPUTS[A_Index, 2] * ((EXPECTED_OUTPUTS[A_Index, 1] - ACQUIRED_OUTPUT) * SIGMOID_GRADIENT)
WEIGHTS[3,1] += TRAINING_INPUTS[A_Index, 3] * ((EXPECTED_OUTPUTS[A_Index, 1] - ACQUIRED_OUTPUT) * SIGMOID_GRADIENT)
; Breaking the formula above: Weight is adjusted (we use +=, not :=) by getting the value of the input byte and multiplying it by the difference between calculated input (sigmoidally treated) and expected input, after this difference is adjusted by the gradient of the sigmoid (removing the sigmoidal distortions).
}
}
/*
3. FINAL RESULTS
*/
; VALIDATION CASE [1,0,0]:
Input1 := 1, Input2 := 0, Input3 := 0
; After recalculating the weights in 10.000 iterations of training, we apply them by multiplying these weights to inputs that resembles a new case
; (this new case is a validation sample, not one of the training ones: [1, 0 ,0])
MSGBOX % "VALIDATION CASE: `n" . Input1 . "`, " . Input2 . "`, " . Input3 . "`n`nFINAL WEIGHTS: `nWEIGHT1: " . WEIGHTS[1,1] . "`nWEIGHT2: " . WEIGHTS[2,1] . "`nWEIGHT3: " . WEIGHTS[3,1] . "`n`nWEIGHTED SOLUTION: `n" Input1 * WEIGHTS[1,1] + Input2 * WEIGHTS[2,1] + Input3 * WEIGHTS[3,1] . "`n`nFINAL SOLUTION: `n" . (1 / (1 + EXP(-1 * (Input1 * WEIGHTS[1,1] + Input2 * WEIGHTS[2,1] + Input3 * WEIGHTS[3,1])))) . "`n`nComments: `nA FINAL SOLUTION between 0.5 and 1.0 means the final network thinks the solution is 1. How close the value is to 1 means how certain the net is of that. `nA FINAL SOLUTION between 0 and 0.5 means the final network thinks the solution is 0. How close the value is of 0 means how certain the net is of that."
; Breaking the output numbers:
; WEIGHTED_SOLUTION: If this is positive, the net believes the answer is 1 (If zero or negative, it belives the answer is 0). The higher a positive value is,
; the more certain the net is of its answer being 1. The lower a negative value is, the more certain the net is of its answer being 0.
; FINAL SOLUTION: A sigmoidally treated weighted_solution. If this is above 0.50, the net believes the answer to be 1. The closer to 1, the more certain
; the net is about that. If this is 0.50 or below it, the net believes the answer to be 0. The closer to 0, the more certain the net is about that.
Return
; The function below is just a single step in multiplying matrices (this is repeated many times to multiply an entire matrix). It is used because the input_data, weights and expected results were set into matrices for organization purposes.
MATRIX_ROW_TIMES_COLUMN_MULTIPLY(A,B,RowOfA)
{
If (A[RowOfA].MaxIndex() != B.MaxIndex())
{
msgbox, 0x10, Error, Number of Columns in the first matrix must be equal to the number of rows in the second matrix.
Return
}
Result := 0
Loop % A[RowOfA].MaxIndex()
{
Result += A[RowOfA, A_index] * B[A_Index, 1]
}
Return Result
}