Neural Network basics - Artificial Intelligence using AutoHotkey!

Helpful script writing tricks and HowTo's
User avatar
Gio
Posts: 1247
Joined: 30 Sep 2013, 10:54
Location: Brazil

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

21 Mar 2018, 17:35

Had a great time presenting the webinar, thanks for allowing me the opportunity Joe :thumbup:

The webinar covered most of the section I of this tutorial and it constitutes awesome material to complement it. Thanks for sharing the link.
User avatar
SpeedMaster
Posts: 494
Joined: 12 Nov 2016, 16:09

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

21 Apr 2018, 17:15

Hello,
Very intresting subject ! :shock:
Thanks for sharing :thumbup:

I made 2 interactive grids for the section 1 and 2 of the tutorial :D
:!: You can click a cell to change its value

section 1:
Spoiler
section 2:
Spoiler
Cheers
User avatar
Gio
Posts: 1247
Joined: 30 Sep 2013, 10:54
Location: Brazil

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

23 Apr 2018, 10:28

Awesome, thanks for sharing the code SpeedMaster. It is certainly much more intuitive that way. I'll link your post in the tutorial. :thumbup:

Also, glad you found this subject interesting. I am currently moving on to create code for handwritten digit recognition (trying to get some time to do it, actually). Might aswell make an addition to this tutorial once i get to something useful. If you succesfully create Neural Network code for doing any tasks, feel free to share or discuss it here (or create a new topic in any of the forums). This subject is really of great use :D
blue83
Posts: 157
Joined: 11 Apr 2018, 06:38

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

26 Apr 2018, 01:59

Hi Gio,

You said "I am currently moving on to create code for handwritten digit recognition".

Is it really possible? I mean extract data from pdf that has handwriting? Or even recognize digital numbers from pdf better than with OCR?
User avatar
Gio
Posts: 1247
Joined: 30 Sep 2013, 10:54
Location: Brazil

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

26 Apr 2018, 08:56

Hello Blue83.

Welcome to the AutoHotkey Community Forums.

Yes, it is possible. In fact, coding a neural network to achieve some form of handwritten text recognition is considered by some as the "hello world" task for neural networks. There is a well known free online book by Michael Nielsen that dives deep into the subject, providing a step-by-step into an implementation. You can find it in this link.

Regarding OCR and Neural Networks, they are not exclusive to each other. OCR is considered an umbrella term and most of the OCR algorithms nowadays were devised using machine learning algorithms, whose main tool is now Neural Networks.

There is also what we specifically call ICR (Intelligent Character Recognition). You can have a quick read it about in this wikipedia page.
blue83
Posts: 157
Joined: 11 Apr 2018, 06:38

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

27 Apr 2018, 02:29

Hi Gio,

Thank you :)

Ok, I now I understand and thank you for a link.

If this is so easy for NN, can you transfer that code from PY to AHK? :)

Anyway this your topics about NN and machine learning is something that need to be more frequently discussed on this forum.
User avatar
Gio
Posts: 1247
Joined: 30 Sep 2013, 10:54
Location: Brazil

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

27 Apr 2018, 11:32

blue83 wrote:If this is so easy for NN, can you transfer that code from PY to AHK? :)
I wouldn't call developing a fully function ANN to recognize handwritten digits easy. In fact, the development of better ANNs for the task is an ongoing effort. That being said, if you are looking for a choice first-problem to study the actual application of Neural Networks, this could be a good choice.

Here is why: i think coding a network to achieve 90% accuracy using the MNIST DataBase is somewhat completely doable for a Neural Network beginner, but getting 98%+ or perhaps 99%+ accuracy is a whole different story. It may well be possible to achieve such high rates (and the table in the link above shows it has been achieved for the MNIST Database already) but these high rates will probably require an exponential ammount of effort in studying and correcting your code and testing it again and again and again. Thats why the accuracy rate has been improving over the years (not days).

Neural Networks are about abstraction power. There is no known or feasibly conceivable algorithm to understand each and every possible human handwriting. But if your code can somehow absorb some of the undescribable basic underlying rules of handwriten digits, it may be able to do some good predictions for unlabeled samples.

Regarding translating the code in that tutorial to AHK, it can be done, and i may well dwelve into the task when i finish studying the ebook, but for the moment, i unfortanetly am lacking some spare time to do it. If however, you wish to take on the task and require assintance, feel free to drop in your questions :thumbup:
Anyway this your topics about NN and machine learning is something that need to be more frequently discussed on this forum.
I agree. This subject is popping up everywhere when it comes to technology advancement news. This decade and the next one are probably THE Neural Network and Machine Learning decades, as it is going to be when it will change our lives the most in the same way ordinary programming changed our lives in the 90s and 2000s.
iseahound
Posts: 1427
Joined: 13 Aug 2016, 21:04
Contact:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

28 Apr 2018, 05:35

To clear up the confusion: Python and AHK are scripting languages. Neural networks are written in C, C++, and CUDA, then compiled into executable code. The executable code can be wrapped in an API, which is then exposed in a higher level scripting language like Python. In other words, it's not that Python is magically capable of being used for advanced data processing. Someone simply decided that they enjoyed the syntax of Python, and made it easy for Python to access their awesome C/C++/CUDA functions.
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

28 Apr 2018, 11:05

Do you have any proof or source for your claims?
Recommends AHK Studio
iseahound
Posts: 1427
Joined: 13 Aug 2016, 21:04
Contact:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

28 Apr 2018, 18:33

https://stackoverflow.com/questions/356 ... e/35678837

For an in-depth look at the fundamental trade-off between code safety (higher level programming languages) and speed (low level resource management) try this: https://people.mpi-sws.org/~dreyer/pape ... /paper.pdf

Here's an example of Tensorflow bindings for Go. https://github.com/danieldk/tensorflow It's really a question of who wants to write Tensorflow bindings for AHK.
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 00:28

Im aware of code safety and speed. The link you send thats supposed to be an introduction to code safety stuff sucks.
This just seems to be one frame work? Is it the only way to use neural Networks in Python? Are there other frame works that use different languages?
You make it seem like most of the neural network development is happening in C++ rather than in Python that only has one of the FrameWorks copied there.
Towards me it seemed like Python is the language where the neural network stuff is happening if you program it in your free time.

Also quoting the original TensorFlow website:
The following guides explain how to install TensorFlow libraries for use in other programming languages. These APIs are aimed at deploying TensorFlow models in applications and are not as extensive as the Python APIs.
  • Installing TensorFlow for Java
  • Installing TensorFlow for C
  • Installing TensorFlow for Go
It seems towards me that the Stackoverflow link you posted barely touches the topic and actually fails to provide any other information than: "Speed is not an issue on Python due to using other languages"
Why they actually used Python seems to be left out completely.
Recommends AHK Studio
iseahound
Posts: 1427
Joined: 13 Aug 2016, 21:04
Contact:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 03:31

Isn't that obvious? There's scipy, numpy, pandas, pillow, nltk, and IPython + jupyter notebooks (which makes it feel like Wolfram Mathematica), and lots more. There's already lots of good code in Python, lots of good libraries, big userbase, and importantly many programmers feel comfortable with Python. Also I sent an "in-depth look at the fundamental trade-off between code safety and speed". Not sure how anyone can mistake a 34 page overview of a proof as introductory LMAO
iseahound
Posts: 1427
Joined: 13 Aug 2016, 21:04
Contact:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 03:47

If there's still confusion, nearly every library is written in C, C++, ASM, or GPU specific code like CUDA. To take an example from Scipy: @ Python 51.9% @ Fortran 25.9% @C 19.7% @ C++ 2.3% @ TeX 0.2% @ Matlab 0.0%. Here's Numpy @C 53.9% @ Python 44.8% @ C++ 1.1% JavaScript 0.1% @ Fortran 0.1% @ Shell 0.09%.

There's a HUGE difference in writing the underlying logic for a neural network, which is basically advanced statistics (nothing neural or remotely intelligent) that preforms convolutions and other matrix operations. This has to be done on the GPU, which happens to be very good at multiplying numbers repeatedly. That's completely different from making a model and training it - all of this can be written in pure Python. The data that is output by each line of Tensorflow API is never converted into a Python representation, it's directly passed into the next line of Tensorflow code so that speed is maintained. As far as I am aware of, nothing happens in C++ because python is cleaner and easier to use. You could use R or Javascript. I like their Data visualization libraries, ggplot and d3 respectively.
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 04:16

Well Im not quite sure why I would have to read a 34 page overview for something you are simply wrong about.
Please stop wasting my time and provide proper proof of your statement in a way thats easily treaceable.

So in the end you do agree that development with Machine Learning is happening in Python after all?
And just part of the underlying libraries are written in other languages to provide a speed boost?
Recommends AHK Studio
iseahound
Posts: 1427
Joined: 13 Aug 2016, 21:04
Contact:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 05:55

nnnik, I was adding some depth to the discussion. Purely optional, yet interesting paper. You seem to be very confused, or are deliberately trying to provoke me. If you are confused, I highly suggest learning neural networks on your own. After all the best work comes with effort. There's no point in asking for my opinion on anything other than my preference. My agreement won't change any facts. If I'm wrong or mistaken, you might want to present your evidence. (Although the only opinion I expressed was that I admire d3 and ggplot2.)
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 07:57

Yeah well I guess we wont get along after all. I was neither asking for your opinion nor did I want to learn anything.
Your point was entirely pointless and added nothing to the discussion at all - at most it could make people confused as to which language to choose when it comes to wanting to learn about Machine Learning.
The answer is Python.
Recommends AHK Studio
iseahound
Posts: 1427
Joined: 13 Aug 2016, 21:04
Contact:

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 17:59

nnnik, if you took a look at the tutorials that Gio posted, he was writing neural networks from scratch. In this scenario he would get a higher speed boost by using CUDA, or at least C. There's a difference between writing a neural network from scratch and training a model, i.e. writing proof of concept code vs. using an existing framework.
nnnik wrote:I was neither asking for your opinion nor did I want to learn anything.
It's plainly obvious to anyone else reading this: you enjoy being right. How vapid and trite.
User avatar
nnnik
Posts: 4500
Joined: 30 Sep 2013, 01:01
Location: Germany

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

29 Apr 2018, 23:45

No I was just about to start learning Neural Networks in Python and your comment irritated me as I might have chosen the wrong language.
I cannot deny that i enjoy being right though.
Recommends AHK Studio
User avatar
Gio
Posts: 1247
Joined: 30 Sep 2013, 10:54
Location: Brazil

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

04 May 2018, 11:53

iseahound wrote:(...) if you took a look at the tutorials that Gio posted, he was writing neural networks from scratch. In this scenario he would get a higher speed boost by using CUDA, or at least C. There's a difference between writing a neural network from scratch and training a model, i.e. writing proof of concept code vs. using an existing framework.
nnnik wrote:(...) which language to choose when it comes to wanting to learn about Machine Learning.
The answer is Python.
You both have valid arguments. Python and other languages (C, C++, CUDA) have some very good tools to implement Neural Network code. In my opinion, this is to be expected, since Code Reuse is a thing and these languages have been used as first-choices by ANN developers for a while now. That being said, let me take the opportunity to explain the advantages of choosing AHK over these languages (yes, there are some advantages for using AutoHotkey too)

:arrow: Why i have choosen to write this tutorial in AutoHotkey:
  • 1. If you are a seasoned AutoHotkey programmer, choosing another language to design a system for this specific task means having to learn additional languages. There is a learning curve to be considered here. If you are an AHK programmer with years of experience, you are well aware that after 3 years of coding in a language, you are usually much better than when you have been coding for just 1 year. Facing the learning curve of a new language just to accomodate a single type of task can be too costly in terms of development time and quality of code. Also, if you wish to use Python alone to make a whole system, it is important to keep in mind that even if Python is more suited for ANNs at the moment, is it equaly suited for easy Automation? GUIs? etc? And even if it is, we should not forget that learning how to use Python for such tasks is also a learning process of its own.
  • 2. For those who wish to implement Neural Network code into well estabilished AHK systems, implementing new Python code as an aid could pose some problems too. An example of this is the chore of having to search for and implement reliable means of communication between two different processes. If any process terminates abruptedly or stops answering, the synchrony of the Control Flow can be in danger if nothing is done to prevent it (and doing something to prevent it costs time and effort or money).
  • 3. Continuing on the hardships of implementing multi-language systems, if the software in question is worked on by a team, finding new suitable teammates to work with can be much harder. This could mean having to spend time (and money) to allow a fellow AutoHotkey programmer to learn Python OR a Python programmer to learn AutoHotkey.
  • 4. On the other hand, if we choose to undergo the process of developing ANN aiding code for AHK (functions, libraries or perhaps even the means of connecting and operating existing frameworks using AHK Code, etc) we may be pioneers in our own community. This does come with some merit. Also, it allows fellow programmers to build on to our work, and may ultimately allow AHK to be one of the first choice languages for ANNs in the future.
For these reasons and some others, i am keeping an open mind about the possibilities of implementing ANN code using mostly AutoHotkey :thumbup:
blue83
Posts: 157
Joined: 11 Apr 2018, 06:38

Re: Neural Network basics - Artificial Intelligence using AutoHotkey!

05 May 2018, 00:15

Nice Gio :)

I hope that also the others experienced programers in ahk will join you here and start developing something with you what is really a future for ahk and for all of us.

Return to “Tutorials (v1)”

Who is online

Users browsing this forum: No registered users and 30 guests