Combining AI with RPA (&AHK)

Discuss Robotics Process Automation. RPA is a rapidly growing field with 6 figure incomes and an extreme workforce shortage. This sub forum will be used to discuss aspects of RPA as it relates to both scripting languages and RPA software such as UIPath or Automation Anywhere
AviationGuy
Posts: 188
Joined: 17 Jan 2019, 10:13

Combining AI with RPA (&AHK)

21 May 2019, 10:22

My name is Max and I'm a 3rd year Aviation Studies student @Amsterdam.
ATM, I'm doing an internship and the goal of my research is to see if it is possible to combine AI with RPA (Intelligent RPA) and how AI can support complex decision-making tasks. Next to this I'm looking into appying AI to AHK.
Since you guys here have experience with RPA I was wondering about your opinion on this subject.
I came across ths forum about RPA but since I've only been working on this research for 5 months I don't see myself as a RPA nor AHK expert.

From what I've read until now in multiple articles is that AI could be very useful when applied within RPA. Unfortunately, I was not able to find any examples or models where was shown HOW AI is applied to RPA. ALso, I wasn't able to find anything about AI influencing decision-making. This is not really a problem for my research since I've explained how I think that the process should look like but if possible I want to go more indepth on this. It would also be nice to know how company's are doing this, of course this will be different for each company but isn't there something like a general approach?

Regarding my research I'm now focussing more on the appliance of AI with AHK.
I have some programming experience but I didn't came much further than the image I've created below.
AI&RPA.png
AI&RPA.png (62.04 KiB) Viewed 13046 times
I think this is roughly what the process should look like but I was wondering if you have any other ideas or suggestions?
For example, is it even possible to combine AI with AHK? My research focusses specifically on an AHK script that performs actions on the interface of a program and to be honest, I don't know if there is any way how AI can be involved in this.
The script basically fills in inputboxes with values (maybe here AI could play a role) and clicks certain buttons in an order.
Personally I think AI will come in handy when the software robot meets unstructured data and crashes because it doesn't know how to handle this.

Recap:
I would like to get more into detail with my research about combining AI and RPA, specifically about how this could be accomplished and how AI can influence decision-making. Next to this, I'm interested in when AI is useful for an AHK process. What do the conditions/tasks of a software robot need to be for AI to be useful?

Hopefully I explained everything well. I'm looking forward to your responses and if you have any questions, let me know :)

Regards,

Max
burque505
Posts: 1748
Joined: 22 Jan 2017, 19:37

Re: Combining AI with RPA (&AHK)

21 May 2019, 10:36

@AviationGuy, your workflow looks good to me. Although I've got close to zero experience in AI, it does occur to me that you might want to look at Bizagi. There is a strong overlap among BPM, RPA, and AI. From your workflow, I think Bizagi might be useful to you (community edition is free, by the way :)), to add to your RPA/AI/AHK toolbox.
Regards,
burque505
(Overigens, als je je ooit in die buurt bevindt, ik kan Cafe Oporto, Zoutsteeg 1 - tegenover De Bijenkorf - aanbevelen :))
User avatar
tank
Posts: 3165
Joined: 28 Sep 2013, 22:15
Location: CarrolltonTX
Contact:

Re: Combining AI with RPA (&AHK)

21 May 2019, 10:44

AviationGuy wrote:
21 May 2019, 10:22
is it even possible to combine AI with AHK?
First, let's talk about the role of RPA in this model. Most are now familiar with chat bots. Sure you are Cortona, Siri and Google assistant are all examples. The AI component is the part that identifies your questions and statements and then triggers an action. Sometimes these actions are set routines like opening an app or converting voice to text. Your bank probably has a Chat bot in the form of an IVR. When you ask for a bank balance for example RPA software is used to take previously submitted identifiers and perform a lookup. RPA software just automates the mundane tasks necessary in the chat bot. So we could reword your question to be something like "can AHK automate mundane tasks?" of course it can! :beer: Furthermore AHK can even be utilized in the machine learning portion. Most RPA software cannot. Blue Prism for example must interact with Watson for AI. AA is pretty limited in its built in machine learning.
AviationGuy wrote:
21 May 2019, 10:22
I don't see myself as a RPA nor AHK expert
While you may as you say, be a novice programmer, I would argue that you clearly have a firm grasp of AI and RPA by the nature of your question
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
Telegram is the best way to reach me
https://t.me/ttnnkkrr
If you have forum suggestions please submit a
Check Out WebWriter
burque505
Posts: 1748
Joined: 22 Jan 2017, 19:37

Re: Combining AI with RPA (&AHK)

21 May 2019, 11:01

Nice post, tank!
AviationGuy
Posts: 188
Joined: 17 Jan 2019, 10:13

Re: Combining AI with RPA (&AHK)

21 May 2019, 14:57

@burque505, interesting! Bizagi seems to be a company that provides software robots if I've seen correctly.
Seems to offer about the same products as the company I'm doing my internship atm. And they have AirFrance as a customer! I will definitely look more into that.
Also, ik kom uit de buurt van Amsterdam dus ken al aardig wat plekjes ;) Echter, deze nog niet! Zal snel is langs gaan voor een :beer:

@tank, very nice post, thanks for that.
Yes, you are completely right. I said that there weren't any examples given really, however, one example came by pretty often, chatbots! I think this is the most optimal example for an intelligent RPA setup.
AviationGuy wrote:
21 May 2019, 10:22
The goal of my research is to see if it is possible to combine AI with RPA (Intelligent RPA) and how AI can support complex decision-making tasks
As for my research question. As you already mentioned, yes it is possible to combine AI with RPA and chatbots are a great example of this. However, I think it depends on what you actually want to automate.
From here on I will be focussing on AHK in this post:
If we go back to my example of an AHK script, where the script just fills in input boxes and clicks at specific buttons. Could there be an AI database (if that's the right name for it) that could provide the right data, I called it ' predictions' in the image I posted, for the script to use? And what data could it provide? Only data for the input boxes or could the AI database also provide information on where to click on the screen based on what's on the screen?

Another thing is, do we actually want to apply AI to certain processes? Because AI operates based on predictions, these predictions could be wrong while when everything is hardcoded in the script, e.g.: If x -> activate y, if a -> activate b, nothing can go wrong really.
tank wrote: While you may as you say, be a novice programmer, I would argue that you clearly have a firm grasp of AI and RPA by the nature of your question
Thanks for this! Although I'm new to RPA and AI, I've got some knowledge about Big Data and other programming languages like 'R' and VBA ;)
User avatar
tank
Posts: 3165
Joined: 28 Sep 2013, 22:15
Location: CarrolltonTX
Contact:

Re: Combining AI with RPA (&AHK)

22 May 2019, 08:10

AviationGuy wrote:
21 May 2019, 14:57
Only data for the input boxes or could the AI database also provide information on where to click on the screen based on what's on the screen?
I could be wrong but i think you are confusing roles here. The AI could do these things but that is really the role of RPA. AI would use a large data set for "Training" and possibly assign values but the RPA aspect would likely handle the form itself.

Think of it this way a human can turn a screw by hand so far but needs a screwdriver to do it correctly. in this metaphor the human is the AI and the RPA is the Screwdriver.

Another example would be parsing invoices of multiple vendors. after a training exercise for the AI the AI can handle all invoice formats with ease. predicting where to read data from and knowing which layout the invoice is using. this intelligence then calls a rules driven algorithm(RPA) to complete the task. This is how Automation Anywhere IQ bots work. You use data and training for unstructured input to make decisions. You use RPA to carry out mundane tasks.
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
Telegram is the best way to reach me
https://t.me/ttnnkkrr
If you have forum suggestions please submit a
Check Out WebWriter
AviationGuy
Posts: 188
Joined: 17 Jan 2019, 10:13

Re: Combining AI with RPA (&AHK)

22 May 2019, 09:31

AviationGuy wrote:
21 May 2019, 14:57
Only data for the input boxes or could the AI database also provide information for the RPA bot on where the RPA bot needs to click on the screen based on what's on the screen?
Is it better when I phrase it like this?

Do you have any examples where AI could be applied to an AHK script, except for a chatbot script?
Since AI comes up with predictions, they are not always right. This isn't a problem really when this happens with a chatbot.
But what if... let's say, an RPA bot is responsible for ordering products based on the data it receives. This data will mostly be valid and useful, but in some occasions the amount of products that need to be ordered isn't present in the data and an AI needs to predict the amound based on previous data and passes this on to the RPA bot. Because, this amount is based on previous data, it will mostly be an acceptable amount but this won't always be the case. The wrong amount can be ordered because of the wrong prediction made by the AI. When you as a company work with high value products you probably don't want this and thus will rather not choose for an Intelligent RPA bot. Although, you can still use the Intelligent RPA bot but you'll need a human to check what the IRPA bot ordered. Is this then still preferable over an RPA bot which simply sends the data without an given order amount to a worker which then needs to order the product himself?

I hope you understand what I'm trying to say.
When is an IRPA bot really a benefit? And when do you still want to use a 'normal' RPA bot which let humans performs the tasks it isn't able to handle?
User avatar
tank
Posts: 3165
Joined: 28 Sep 2013, 22:15
Location: CarrolltonTX
Contact:

Re: Combining AI with RPA (&AHK)

22 May 2019, 10:37

AviationGuy wrote:
22 May 2019, 09:31
Is this then still preferable
What we are talking about here are exceptions. Exceptions require manual effort because they dont follow preset rules. In your example missing required data. Exceptions are part of AI processes and RPA. Sometimes the Algorithm and rules just aren't specific enough to handle a situation. an exception process must be handled by a human.
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
Telegram is the best way to reach me
https://t.me/ttnnkkrr
If you have forum suggestions please submit a
Check Out WebWriter
SOTE
Posts: 1430
Joined: 15 Jun 2015, 06:21

Re: Combining AI with RPA (&AHK)

09 Jun 2019, 05:05

AviationGuy wrote:
22 May 2019, 09:31
AviationGuy wrote:
21 May 2019, 14:57
Only data for the input boxes or could the AI database also provide information for the RPA bot on where the RPA bot needs to click on the screen based on what's on the screen?
Is it better when I phrase it like this?

...Since AI comes up with predictions, they are not always right. This isn't a problem really when this happens with a chatbot.
But what if... let's say, an RPA bot is responsible for ordering products based on the data it receives. This data will mostly be valid and useful, but in some occasions the amount of products that need to be ordered isn't present in the data and an AI needs to predict the amound based on previous data and passes this on to the RPA bot. Because, this amount is based on previous data, it will mostly be an acceptable amount but this won't always be the case. The wrong amount can be ordered because of the wrong prediction made by the AI. When you as a company work with high value products you probably don't want this and thus will rather not choose for an Intelligent RPA bot. Although, you can still use the Intelligent RPA bot but you'll need a human to check what the IRPA bot ordered. Is this then still preferable over an RPA bot which simply sends the data without an given order amount to a worker which then needs to order the product himself?

I hope you understand what I'm trying to say.
When is an IRPA bot really a benefit? And when do you still want to use a 'normal' RPA bot which let humans performs the tasks it isn't able to handle?
Maybe you might want to think of AI differently. When people are referring to AI, they are often referring to "narrow intelligence" or "weak AI". A lot of times, the fact that a program is automating something or can function by itself, many people that don't understand programming will often label it as "intelligent" or as AI. So we have to be careful with the term. The program can simply be following a set of rules or selecting options based on predetermined conditions, thus is automating a process.

A program can be excellent at "making decisions" based on predefined rules or statistical probabilities, within a specific scope, but not so in a general sense nor come up with creative solutions for a problem. If a task, situation, or parameter is outside the scope of what the program was made for or can/should handle, then a human has to intervene. When a program or robot can learn on it's own, in more generalized ways like humans do, then we might be talking about real AI. But where RPA stands now, we are usually referring to automation, more than AI.

Let's say you had an "intelligent program" that was designed to select pictures of certain fruits from any unknown source. It would determine if a picture contains a fruit by shape, colors, or patterns. Since we would be dealing with statistical probabilities and unknowns, this would fall under the umbrella of AI or Cognitive Automation. If on the other hand, the options and choices were set and known (selecting from a known set of pictures, colors, and shapes), this would usually be seen as Traditional RPA or just automation.

In the case of AI or Cognitive Automation, humans would still be involved in training and the setting of acceptable ranges of accuracy. If the program is certain at say 96% or greater, then it could mark the picture as the desired fruit. Below that percentage, it will mark as not desirable or not a fruit. This is the "intelligence" we are dealing with, where the program has to "guess" the right answer.

It can happen that the program gets some pictures wrong, because they are a new type. Something occurred that was outside the original design and capability of the program. A human would have to intervene to train/modify the program to recognize the new kinds of pictures.

If the program can sort through pictures of fruit 30X faster than a human, with say an accuracy rate of 99% or greater. Then that leaves just a small percentage of pictures that must be checked manually. And the human can retrain or modify the program to attempt to get the error rate even lower. How well they could raise the accuracy and reduce exceptions, would depend on many factors. Then for many people or companies, that might be a very acceptable situation. Less people or less hours are needed to sort through the pictures.

Also, a program has to go through testing to make sure it's working as intended. If the acceptable level of accuracy must be 99%, then they would not release or approve of the program until it showed that level of accuracy in testing. If the program is used by a company, the process would be designed so they wouldn't lose money or create a disaster. In those cases where the program was not sure, then it could be turned over to a human to check. Nevertheless, the amount of work or hours that humans had to do would be reduced, even if they had to handle the exceptions. And the program could be revised to reduce the number of exceptions, based on the knowledge and experience they have accumulated, but this can take time.
User avatar
tank
Posts: 3165
Joined: 28 Sep 2013, 22:15
Location: CarrolltonTX
Contact:

Re: Combining AI with RPA (&AHK)

11 Jun 2019, 13:25

What we are really talking about is decided by risk analysis. How much does a single mistake cost and how often do humans make them vs AI. When the risk is less with AI then it becomes the preferred choice. @AviationGuy Does that help?

BTW @SOTE Wonderful explanation. I am far to lazy to have done such a wonderful job
We are troubled on every side‚ yet not distressed; we are perplexed‚
but not in despair; Persecuted‚ but not forsaken; cast down‚ but not destroyed;
Telegram is the best way to reach me
https://t.me/ttnnkkrr
If you have forum suggestions please submit a
Check Out WebWriter
AviationGuy
Posts: 188
Joined: 17 Jan 2019, 10:13

Re: Combining AI with RPA (&AHK)

07 Feb 2020, 04:22

Thank you both for your explanations, sorry that I've been away for a while.

The problem I faced was that I needed it to be (almost) perfect, 99% wasn't enough but I understand what you are saying.
It can certainly be useful for some projects. Although it may not be perfect, it still will ensure that fewer people are needed to perform certain processes.

Return to “RPA”

Who is online

Users browsing this forum: No registered users and 0 guests