Lua Neural Networks

Posts: 477

Reputation Power: 0

Re: Lua Neural Networks

Soulkiller, I'm attempting to write an interface to access this from freebasic. So far I've figured out how to call the create and backwardPropagate functions from FB, but I'm having trouble retrieving the table returned by the forwardPropagate function. I know you probably don't know much about FB, but just from the lua api calls can you tell me what I'm doing wrong here?

Code:

sub neural_net.forewardPropagate(inputs() as single, outputs() as single)
lua_settop(L,0) ' clear the stack

' retreives the neural network table and pushes it onto the stack
lua_getglobal(L, lua_name)

if lua_type(L, -1) <> LUA_TTABLE then
Print "error: lua_name isn't a table"
sleep
system
endif

' get the forewardPropagate method
lua_pushstring(L, "forewardPropagate")
lua_gettable(L, -2)

' get the neural network table and push it on the stack again
' this is needed because forewardPropagate is a method
' of the neural network table
lua_getglobal(L, lua_name)

' convert the inputs array into a lua table,
' this will end up as arg[1] in the
' forewardPropagate method
lua_newtable(L)
for i as integer = lbound(inputs) to ubound(inputs)
lua_pushnumber(L, (i+1))
lua_pushnumber(L, inputs(i))
lua_settable(L, -3)
next i

' call the function with the arguments that have been pushed onto the stack
If lua_pcall(L, 2, LUA_MULTRET, 0) <> 0 Then
print "ERROR:"
Print *lua_tostring(L, 1)
Sleep
system
End If

' check if the return value from the function is a table
if lua_type(L, -1) <> LUA_TTABLE then
' retrieve the table returned by the function
' and put the contents into the outputs array
for i as integer = 0 to ubound(outputs)
' push the table key onto the stack
lua_pushnumber(L, (i+1))
' pop the table key from the stack and replace it with the value stored at that key
lua_gettable(L, -2)
' put the value into the outputs array
outputs(i) = lua_tonumber(L,-1)
lua_pop(L,-1)
next i
endif

end sub

March 13, 2008, 09:24:58 PM

#34

Soulkiller

Soulkiller

Computer Scientist & Electronics Engineer

Posts: 1,098

Awards:

Reputation Power: 0

Re: Lua Neural Networks

I honestly don't know, I'm not gona liue and say I"m proficent in Lua api calls so I can only help you so much :P what kidna results are you getting (if you are gettinga ny at all)

MQ:BTG Programmer
Previous work: FRAG an overhead shooter

March 14, 2008, 03:32:31 AM

#35

Merick

Merick

Posts: 477

Reputation Power: 0

Re: Lua Neural Networks

oops, it looks like I had an <> where I should have had an =

I am getting the results back now, I don't really know much about how this is works but are the numbers I get back supposed to be this random?

*edit*

finished!

http://freefile.kristopherw.us/uploads/merick/nnet.zip

« Last Edit: March 14, 2008, 07:18:53 PM by Merick »

March 15, 2008, 12:34:50 AM

#36

Soulkiller

Soulkiller

Computer Scientist & Electronics Engineer

Posts: 1,098

Awards:

Reputation Power: 0

Re: Lua Neural Networks

Glad you fixed it. I don't ahve my windows box so I can't test it.

Depends what you did. When you create the NN it starts off with random weights (which would yeild random results) Its after the training (backwardPropagate) that the results start turning into things that arn't random and possibly very useful.

MQ:BTG Programmer
Previous work: FRAG an overhead shooter

March 15, 2008, 03:25:37 AM

#37

Merick

Merick

Posts: 477

Reputation Power: 0

Re: Lua Neural Networks

My test app is a direct translation of your example script form the first post. When i say random, what I mean is that even with plugging in the same numbers I will get different results from one run to the next. For instance, one run would give me this (I have my test app truncating the results to 4 decimals when printing to the screen): 0.0041, 0.9965, 0.9965, 0.0026, but then with the next run I would get something like this: 0.0014, 0.0000, 0.9986, 0.0000

March 15, 2008, 11:41:53 AM

#38

Soulkiller

Soulkiller

Computer Scientist & Electronics Engineer

Posts: 1,098

Awards:

Reputation Power: 0

Re: Lua Neural Networks

You need to give me inputs and outputs to properly anaylze it. If you start using inputs that it hasn't trained for, thats when it starts to show its intelligence by "guessing" at the anwser with its previous knowledge. If you know that the NN is going to receive a certain input set and you want it to yeild a certain output then train it to do so. Otherwise don't expect it to know what you wanted it to do. A NN is only as smart as you train it to be.

I have been working on this here and there and my version as two new functions NerualNetowrk:train() which helps the porgrammer train the network at a much faster pace by using TestSet scripts, and NerualNetwork:test() which tests the NN by comparing its results to taht of a TraingSet script and display usful info about all of it inculding the accuracy of the NN which also greatly speeds up training.

« Last Edit: March 15, 2008, 11:45:42 AM by Soulkiller »

MQ:BTG Programmer
Previous work: FRAG an overhead shooter

March 15, 2008, 02:32:53 PM

#39

Merick

Merick

Posts: 477

Reputation Power: 0

Re: Lua Neural Networks

The app that I've been using to test it is a direct port to FB code of the example "solve for xor" script in your first post, and I've been using the "try" values from the input statements. If I load a previously saved network I get the same results, however, if I start a fresh run with a newly created network, I can get wildly different results even when using those same recommended values for creating the network

March 15, 2008, 02:45:15 PM

#40

Soulkiller

Soulkiller

Computer Scientist & Electronics Engineer

Posts: 1,098

Awards:

Reputation Power: 0

Re: Lua Neural Networks

Oh I see why you think its random. an XOR gate is a binary gate that performs the following actions

Code:

input output
0 0 0
1 0 1
0 1 1
1 1 0

your looking at the outputs as decmils where in realaity they would be clamped (look at how close they are, thats what truly matters, the results will never be exact with a sgimoid, but if you changed the activation function so its a jsut a step funciton then yes they would be clear cut answers)

MQ:BTG Programmer
Previous work: FRAG an overhead shooter

March 15, 2008, 02:50:38 PM

#41

Merick

Merick

Posts: 477

Reputation Power: 0

Re: Lua Neural Networks

I've already realized that the results won't be exact, but (1 0) giving me 0.9986 in one run and 0.0014 in another?

March 15, 2008, 02:56:53 PM

#42

Soulkiller

Soulkiller

Computer Scientist & Electronics Engineer

Posts: 1,098

Awards:

Reputation Power: 0

Re: Lua Neural Networks

after training? then You would call that an unsucesful traing sequence. It has still not learned its training set.

Here are the two functions that will aid you in training

Code:

function NeuralNetwork:train( trainingSet, attempts)
while attempts > 0 do
for i = 1,table.getn(trainingSet) do
self:backwardPropagate(trainingSet[i].input,trainingSet[i].output)
end
attempts = attempts - 1
end
end

function NeuralNetwork:test( trainingSet, extraInputs)
local testResults = "Training Test For Nerual Network:\n"
--print("Training Test For Nerual Network:")
local overallPer = 0
for i = 1,table.getn(trainingSet) do
testResults = testResults.." Set #"..i.."\n"
--print(" Set #"..i)
local results = self:forewardPropagate(trainingSet[i].input)
testResults = testResults.." Input:\n"
--print(" Input:")
for j = 1,table.getn(trainingSet[i].input) do
testResults = testResults.." "..trainingSet[i].input[j].."\n"
--print(" "..trainingSet[i].input[j])
end
testResults = testResults.." Output|Desired Results:\n"
--print(" Output|Desired Results:")
local percision = 0
for j = 1,table.getn(results) do
testResults = testResults.." "..results[j].." | "..trainingSet[i].output[j].."\n"
--print(" "..results[j].." | "..trainingSet[i].output[j])
percision = percision + math.abs(results[j] - trainingSet[i].output[j])
end
percision = percision / table.getn(results)
overallPer = overallPer + percision
testResults = testResults.." Average Output Percision: "..percision.."\n Average Percentage of Accuracy: "..100-math.ceil(percision*100).."%\n"
--print(" Average Output Percision: "..percision)
--print(" Average Percentage of Accuracy: "..100-math.ceil(percision*100).."%")
end
if extraInputs then
for i = 1,table.getn(extraInputs) do
testResults = testResults.." Unprepared Set #"..i.."\n"
--print(" Unprepared Set #"..i)
local results = self:forewardPropagate(extraInputs[i])
testResults = testResults.." Input:\n"
--print(" Input:")
for j = 1,table.getn(extraInputs[i]) do
testResults = testResults.." "..extraInputs[i][j].."\n"
--print(" "..extraInputs[i][j])
end
testResults = testResults.." Output:\n"
--print(" Output:")
for j = 1,table.getn(results) do
testResults = testResults.." "..results[j].."\n"
--print(" "..results[j])
end
end
end
overallPer = overallPer / table.getn(trainingSet)
testResults = testResults.." Overall Average Percision of Trained Sets: "..overallPer.."\n Overall Average Percentage of Accuracy: "..100-math.ceil(overallPer*100).."%\n"
--print(" Overall Average Percision of Trained Sets: "..overallPer)
--print(" Overall Average Percentage of Accuracy: "..100-math.ceil(overallPer*100).."%")
return testResults
end

Here's a sample Training script for the xor test

Code:

TrainingSet = {
{
input = {0,0},
output = {0}
},
{
input = {1,0},
output = {1}
},
{
input = {0,1},
output = {1}
},
{
input = {1,1},
output = {1}
},
}

and finally here's the new test script

Code:

math.randomseed(os.time())

dofile("NeuralNetwork.lua")

dofile("trainingScript.lua")

print("Enter the amount of hidden layers in Neural Network (recommended|1) : ")

hiddenLayers = tonumber(io.stdin:read())

print("Enter the amount of Neurons per hidden layer(recommended|4) : ")

neuronsPerLayer = tonumber(io.stdin:read())

print("Enter the learing rate (0-100) (recommened|50) : ")

learningRate = tonumber(io.stdin:read()/100)

network = NeuralNetwork.create(table.getn(TrainingSet[1].input),table.getn(TrainingSet[1].output),hiddenLayers,neuronsPerLayer,learningRate)

print("Enter the number of attempts at Teaching the Neural Network (Try 4000): ")

attempts = tonumber(io.stdin:read())

print("Teaching...")

network:train( TrainingSet, attempts)

print(network:test( TrainingSet))

print("Saving Neural Network...")

NN = io.open("neuralNet.txt","w")

NN:write(network:save())

NN:flush()

NN:close()

print("Loading Neural Network...")

NN = io.open("neuralNet.txt","r")

network = NeuralNetwork.load(NN:read())

results = network:test( TrainingSet)
NNresults = io.open("neuralNetTestResults.txt","a")
NNresults:write(results)
NNresults:flush()
NNresults:close()
print(results)

MQ:BTG Programmer
Previous work: FRAG an overhead shooter

March 15, 2008, 03:07:48 PM

#43

Merick

Merick