Neural Network basics - Artificial Intelligence using AutoHotkey!

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :| :mrgreen: :geek: :ugeek: :arrow: :angel: :clap: :crazy: :eh: :lolno: :problem: :shh: :shifty: :sick: :silent: :think: :thumbup: :thumbdown: :salute: :wave: :wtf: :yawn: :facepalm: :bravo: :dance: :beard: :morebeard: :xmas: :HeHe: :trollface: :cookie: :rainbow: :monkeysee: :monkeysay: :happybday: :headwall: :offtopic: :superhappy: :terms: :beer:
View more smilies

BBCode is ON
[img] is OFF
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: Neural Network basics - Artificial Intelligence using AutoHotkey!

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 18 Jun 2018, 13:09

Well it's not hard to get ideas for optimisation from the current implementation - there is a lot of room for it.

I also plan to create different types of neural networks such as Convolutional neural nets and LSTM for RNN.
I might want to look into GANs but right now thats far into the future since finals are coming up and I have a lot of projects coming from my uni.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by Gio » 18 Jun 2018, 12:47

nnnik wrote:While the script was still at 0% progress after letting it for a night it was already down to 16.6% total error.


This is pretty awesome news actually. 83.4% accuracy is very exciting for a first shot. Random guessing is only 10% accurate, which means you managed to get the AHK coded network to learn a lot about the images. Regarding the processing times, it reminds me of a barcode reader i once developed. At first, it was taking like 2-3 minutes for the code to scan an image. After changing the method of retrieving the pixel colors though, it started to scan the whole images in like 3-5 seconds. Optimization can make huge speed improvements when dealing with bulk data.

Also, maybe we can use Michael Nielsens python code as a rough estimate for how long a well optimized code will take to run the Mnist Database. After running a translation of the code to Python 3.x i saw it achieve 94% accuracy in under 5 minutes. Curisously though, a second run of the same Python code only got to 83,53% accuracy after 10 minutes :crazy:

Michael probably did a lot of testing untill he got that specific code though (simplicity of code often hides the hardwork of optimizing it).

First run:
PYTHON RESULTS.png
(55.81 KiB) Not downloaded yet


Second run:
PYTHON RESULTS2.png
(34.55 KiB) Not downloaded yet


The Python version i ran the code on was 3.6.4 and the files can be downloaded here. To run the code, open the file Nielsen Code 3x by Unknown.Py inside the TEST CASE folder on Python 3.6.4 IDLE. Numpy must be installed aswell.

I think I might update the entire library to use a Matrix class which uses binary data storage and MCODE to do the calculations.
Additionally the splitting between the input and output inside the train method itself is unneccessary. I think I will just accept 2 parameters and possibly also allow for binary data there.


Those are some very good ideas :thumbup:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 18 Jun 2018, 11:22

There is a bug in it though.
I tried training a deeper neural network using a part of the MNIST database: (28x28 Inputs, 16 Neurons each layer, 3 hidden layers, 10 outputs )
http://yann.lecun.com/exdb/mnist/
However the performance is abyssmal. I think I might update the entire library to use a Matrix class which uses binary data storage and MCODE to do the calculations.
Additionally the splitting between the input and output inside the train method itself is unneccessary. I think I will just accept 2 parameters and possibly also allow for binary data there.
While the script was still at 0% progress after letting it for a night it was already down to 16.6% total error.

If anybody is interested:
The script expects 2 files from the mnist database in its folder (see my link above)
The training images should be called "images" with no extension.
The training labels should be called "labels" with no extension.

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by Gio » 18 Jun 2018, 10:08

nnnik wrote:I worked a little on Speedmasters example grid for section 2 of your tutorial.
Here is the updated version:
-On clicking the VALIDATION CASE the resulting values will be updated automatically
-extracted the network creation and training code and put it into a class
-added a second output
-clicking calculate ANN wont reset the neurons
-added bias to each neuron


Excellent Nnnik, the new class style is very nice :thumbup:

Will link your version in the tutorial aswell.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 16 Jun 2018, 15:04

I worked a little on Speedmasters example grid for section 2 of your tutorial.
Here is the updated version:
-On clicking the VALIDATION CASE the resulting values will be updated automatically
-extracted the network creation and training code and put it into a class
-added a second output
-clicking calculate ANN wont reset the neurons
-added bias to each neuron

Code: [Select all] [Expand]GeSHi © Codebox Plus

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by Joe Glines » 01 Jun 2018, 04:37

@Gio- Thanks for sharing that video! Nice to know humans are safe from extinction for a while... lol

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by Gio » 29 May 2018, 12:33

Just watched an excellent Youtube video by SciShow that deals with the hardships of developing self-driving cars. This is a perfect example on how overly complicated abstract models will still require a lot of programmers work for the years to come. I guess it is pretty safe to say this is a great opportunity :thumbup:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by blue83 » 17 May 2018, 03:18

Hi Gio,

Thank you for your answer and clarification.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by Gio » 16 May 2018, 15:40

blue83 wrote:how can we use AI if we use some conditions to recieve something back.

(..) my question is can be done something about prediction of our moves, clicks etc.

I mean that UI can learn how we use Windows and if something happens that we dont have in our script, that UI can learn and act accordingly to that new situation.


In my opinion something like that is certainly possible, although it will most likely take the form of a project and scope planning must be considered. In this video a Neural Network was trained to play a certain game (Mario Kart) based on the inputs of a player related to the position of the objects on screen. The key point in there is that the network does not even know what winning a race is, but was able to win an entire cup by attempting to predict and mimic the players movements in each new situation based on previous data.

To this end, it is important to realise that the scope of the project is very important and must be very well planned. A network to predict any possible human action in a computer will certainly be too costly for any practical implementation, but a network that decides when a pop-up window is probably going to be immediately closed by the user, and than closes it for the user is something much more feasible to do. Also, it is impotant to realise that some tasks are more efficiently done by ordinary programming. Opening certain programs on boot based on how likely the user is to open that program as soon as he boots the PC is something that can be achieved with more simple statistic controls than a Neural Network. Also, prediction accuracy demands are VERY important. Neural Networks are unlikely to be 100% precise in their judgements. If an accuracy between 95-98% is too troublesome (i.e., there being a certain pop-up window that should never ever be closed, such as an online test that considers each session an attempt), i would suggest considering the project too complicated unless previous experience tells you otherwise or some form of "easy undo" routine is present (i.e, let's suppose that the flagged pop-ups are not really immediately closed, but rather hidden for some seconds before closing and the user can unhide during this time).

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by blue83 » 15 May 2018, 01:32

Hi I have one more question.

Here are examples how can we use AI if we use some conditions to recieve something back.

Because AHK is script languague for automation of tasks in windows, my question is can be done something about prediction of our moves, clicks etc.

I mean that UI can learn how we use Windows and if something happens that we dont have in our script, that UI can learn and act accordingly to that new situation.

Also there is an issue with unstructured data.

Thanks

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 05 May 2018, 01:39

Well if we really wanted to program Neural Networks in AHK we probably need to implement a framework like iseahound mentioned.
And that seems like a lot of work for a single person.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by blue83 » 05 May 2018, 00:15

Nice Gio :)

I hope that also the others experienced programers in ahk will join you here and start developing something with you what is really a future for ahk and for all of us.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by Gio » 04 May 2018, 11:53

iseahound wrote:(...) if you took a look at the tutorials that Gio posted, he was writing neural networks from scratch. In this scenario he would get a higher speed boost by using CUDA, or at least C. There's a difference between writing a neural network from scratch and training a model, i.e. writing proof of concept code vs. using an existing framework.


nnnik wrote:(...) which language to choose when it comes to wanting to learn about Machine Learning.
The answer is Python.


You both have valid arguments. Python and other languages (C, C++, CUDA) have some very good tools to implement Neural Network code. In my opinion, this is to be expected, since Code Reuse is a thing and these languages have been used as first-choices by ANN developers for a while now. That being said, let me take the opportunity to explain the advantages of choosing AHK over these languages (yes, there are some advantages for using AutoHotkey too)

:arrow: Why i have choosen to write this tutorial in AutoHotkey:

  • 1. If you are a seasoned AutoHotkey programmer, choosing another language to design a system for this specific task means having to learn additional languages. There is a learning curve to be considered here. If you are an AHK programmer with years of experience, you are well aware that after 3 years of coding in a language, you are usually much better than when you have been coding for just 1 year. Facing the learning curve of a new language just to accomodate a single type of task can be too costly in terms of development time and quality of code. Also, if you wish to use Python alone to make a whole system, it is important to keep in mind that even if Python is more suited for ANNs at the moment, is it equaly suited for easy Automation? GUIs? etc? And even if it is, we should not forget that learning how to use Python for such tasks is also a learning process of its own.
  • 2. For those who wish to implement Neural Network code into well estabilished AHK systems, implementing new Python code as an aid could pose some problems too. An example of this is the chore of having to search for and implement reliable means of communication between two different processes. If any process terminates abruptedly or stops answering, the synchrony of the Control Flow can be in danger if nothing is done to prevent it (and doing something to prevent it costs time and effort or money).
  • 3. Continuing on the hardships of implementing multi-language systems, if the software in question is worked on by a team, finding new suitable teammates to work with can be much harder. This could mean having to spend time (and money) to allow a fellow AutoHotkey programmer to learn Python OR a Python programmer to learn AutoHotkey.
  • 4. On the other hand, if we choose to undergo the process of developing ANN aiding code for AHK (functions, libraries or perhaps even the means of connecting and operating existing frameworks using AHK Code, etc) we may be pioneers in our own community. This does come with some merit. Also, it allows fellow programmers to build on to our work, and may ultimately allow AHK to be one of the first choice languages for ANNs in the future.

For these reasons and some others, i am keeping an open mind about the possibilities of implementing ANN code using mostly AutoHotkey :thumbup:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 29 Apr 2018, 23:45

No I was just about to start learning Neural Networks in Python and your comment irritated me as I might have chosen the wrong language.
I cannot deny that i enjoy being right though.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by iseahound » 29 Apr 2018, 17:59

nnnik, if you took a look at the tutorials that Gio posted, he was writing neural networks from scratch. In this scenario he would get a higher speed boost by using CUDA, or at least C. There's a difference between writing a neural network from scratch and training a model, i.e. writing proof of concept code vs. using an existing framework.

nnnik wrote:I was neither asking for your opinion nor did I want to learn anything.


It's plainly obvious to anyone else reading this: you enjoy being right. How vapid and trite.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 29 Apr 2018, 07:57

Yeah well I guess we wont get along after all. I was neither asking for your opinion nor did I want to learn anything.
Your point was entirely pointless and added nothing to the discussion at all - at most it could make people confused as to which language to choose when it comes to wanting to learn about Machine Learning.
The answer is Python.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by iseahound » 29 Apr 2018, 05:55

nnnik, I was adding some depth to the discussion. Purely optional, yet interesting paper. You seem to be very confused, or are deliberately trying to provoke me. If you are confused, I highly suggest learning neural networks on your own. After all the best work comes with effort. There's no point in asking for my opinion on anything other than my preference. My agreement won't change any facts. If I'm wrong or mistaken, you might want to present your evidence. (Although the only opinion I expressed was that I admire d3 and ggplot2.)

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by nnnik » 29 Apr 2018, 04:16

Well Im not quite sure why I would have to read a 34 page overview for something you are simply wrong about.
Please stop wasting my time and provide proper proof of your statement in a way thats easily treaceable.

So in the end you do agree that development with Machine Learning is happening in Python after all?
And just part of the underlying libraries are written in other languages to provide a speed boost?

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by iseahound » 29 Apr 2018, 03:47

If there's still confusion, nearly every library is written in C, C++, ASM, or GPU specific code like CUDA. To take an example from Scipy: @ Python 51.9% @ Fortran 25.9% @C 19.7% @ C++ 2.3% @ TeX 0.2% @ Matlab 0.0%. Here's Numpy @C 53.9% @ Python 44.8% @ C++ 1.1% JavaScript 0.1% @ Fortran 0.1% @ Shell 0.09%.

There's a HUGE difference in writing the underlying logic for a neural network, which is basically advanced statistics (nothing neural or remotely intelligent) that preforms convolutions and other matrix operations. This has to be done on the GPU, which happens to be very good at multiplying numbers repeatedly. That's completely different from making a model and training it - all of this can be written in pure Python. The data that is output by each line of Tensorflow API is never converted into a Python representation, it's directly passed into the next line of Tensorflow code so that speed is maintained. As far as I am aware of, nothing happens in C++ because python is cleaner and easier to use. You could use R or Javascript. I like their Data visualization libraries, ggplot and d3 respectively.

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

Post by iseahound » 29 Apr 2018, 03:31

Isn't that obvious? There's scipy, numpy, pandas, pillow, nltk, and IPython + jupyter notebooks (which makes it feel like Wolfram Mathematica), and lots more. There's already lots of good code in Python, lots of good libraries, big userbase, and importantly many programmers feel comfortable with Python. Also I sent an "in-depth look at the fundamental trade-off between code safety and speed". Not sure how anyone can mistake a 34 page overview of a proof as introductory LMAO

Top