Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Learn how I made a working PokerBot for Pokerstars


  • Please log in to reply
38 replies to this topic
skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
Last Updated: 2009 JUL 20
I want to share my ideas and my approach of making a working poker bot.
The complete source code will not be provided, parts will.
For clarification: A pokerbot is an application playing unattended poker on online casinos, thus (hopefully) earning money. They do not need to generate huge earns per hour - a slight gain over break-even can summarise indefinitly when running on multiple tables 24h/365d.
Applications that support the player but do not play unattended are not considerd bots, yet they might work as a part of the bot´s logic.

What I have got so far
The bot reads cards and opponent actions, recognises current potsize and decides upon fold, call and bet depending on a wide set of expected-value-calculations based on monte-carlo-evaluation and opponent statistics extracted from pokerstars.

It plays all phases of a round, yet the AI of turn and river are not yet completed.

While nI only care about fixed-limit-cash-games at pokerstars, most principles can be transfered through.

Legal issues
None. Hey, online-poker is illegal in many countries anyway. From that point of view, who cares about bots? Furthermore, you need to understand that "forbidden" by the casino is not "illegal". It is simply not allowed. As this has raised to discussion, I add the fact that I do not care about any such issues, if there are some.

Concept of a pokerbot
Any Pokerbot will consist of thee parts: Input, processing, output.

You need to get data from the table. Your position, the number of your opponents, your cards, previous actions, potsize and so on. That is what we call input. As pokerstars does not leak any real-time-data into logfiles, nor does it use standard window-classes to get the text out of them, you will do a hell load of imagesearch. Whatever your Eyes can see can also be found via AHK.

Whenever you want to search on a specific spot for lots of different yet similar pictures (say cards), getting the color of a single pixel prior to run the bunch of searches is the way to go.

The next step is the processing, the "decision-making". All input is saved in variables, and after some woodoo there is a single variable passed. It contains fold, call, bet/raise. (As I play fidex limit, there is no numeric value)

The output in the end does the clicking. At this point I also use it to display the made decision before actually clicking the button. Anti-detection comes in at this place

Input at Pokerstars
As I told before, I am using imagesearch. Since Stars doen´t use standard API, there is no way to avoid imagescraping, unless you want to decompile the whole thing. You need screenshots ( I prefer .bmp) of every 52 cards, of your avatar, the dealerbutton, and many more. As you might realise, it is a good idea to choose a design and skin and to stick with it.

The workflow is
;;;;; INPUT PART;;;;;;;;;;
;startup
Loop until: Search my avatar successfull.
Save it´s exact position for further image-searches of my pocket cards.
Calculate my position on the table (e.g. 1 to 6) fot further reference.

;non-stop-loop
loop
{
scan for every 52 cards at the position of my avatar
if 2 different cards found withhin one loop break the loop
}

Convert the two cards into a simple format like "AKs" (suited) or "72o" (offsuite).

Imagesearch the dealer buttons position (E.g. 1 to 6).
Imagesearch for empty seats, open seats, and Sitting-outs.
Calculate my position (E.g. Button, Small Blind, Big blind...)
 - - - Wait until the Action-Buttons appear (it´s my turn now).
Scan for the Raise-Button (indicating there were previous bets)

Call the processing part. Submit pocket cards, position, and previous bets.

Processing
This is the most important part of any bot. It will make you win or loose money. At this point I will only show HOW to make the thing. You will need your own logic based upon your own style of game. That´s my approach for the preflop-game:

First of all scan for specific situations deserving special treatment. Sitting on the bigblind and facing a smallblind´s steal attempt calls a function doing exactly this. Sitting on the SB with no prev bets call on the other hand a function that cares about stealing. If no special functions are triggered, things continue rule-based:

As there are only 3 actions - bet/raise, call, fold - you will simply have a list of rules sorting your hands into one of these actions.

I created a list of the 50 strongest starting hands. "AAo" is rank 1, "JTs" shall be somewhere around rank 20 (both out of 169)... Their value is stored inside a pseudo-array. So if I got pocketcards:="JTs", I just read Rank%pocketcards% to get RankJTs=20. If there is no match, I will search RankTJs, as this is the same - that approach saves me sorting.
Cards not in the list are set a rank of 100. It doesn't matter, as they are always folded.

This rank will now be added a bonus for good positions like the Button, and will be substracted a penality for previous bets making this entry more expensive.

Let´s say I got JTs, I am sitting on the button, but there has been a previous bet. Initial rank is 20. Buton-Bonus is 12, so it currently ranks 8. Penality is (30-VPIPofOPPONENT) for previous bet, so it ranks about 14 upon a standard opponent. I do bet/raise with ranks below 12, and call with ranks below 25. So it is a call.

(pay attention that all of these values are just exemplaric. I use other numbers I am not going to post, for obvious reasons.)



Output
The simplest part.
Do some random sleep to avoid detection and look human, then click any of the buttons according to the action. Don´t click the same pixel over and over. Be sure to try clicking a check-button before folding, just in case you were bigblind.

Summary
Botting pokerstars is a lot of work, but it can be done.
Winning a 20 cent per hour at microlimit can really pay off with 10 tables and 12 hours a day.

Preview
Currently I am working on following Ideas:
-Reading the potsize via ocr (working, but not yet considered by the processing) now done
-Querying the postgres-database for statistics upon my opponents.now worked around


Question policy:
Basicly, feel free to aks anything, but not everything.
I will share single lines of my code if needed, and will go in-detail for further discussion. I am willing to support you by creating a bot, but not by copying mine. I´d love to hear some useful comments, for sure. Especialy when it comes nash-calculations.

TheIrishThug
  • Members
  • 419 posts
  • Last active: Jan 18 2012 02:51 PM
  • Joined: 19 Mar 2006

None. Hey, online-poker is illegal in many countries anyway. From that point of view, who cares about bots? Furthermore, you need to understand that "forbidden" by the casino is not "illegal". It is sinmply not allowed.
On a moralic view, I dont see the bot as an unfair advantage. It does exactly what I told him to do. It is following rules, there is no intelligence behind. Therefore it only automates my actions, and does not improve them.

So you are saying if I sold weed, there wouldn't be a difference if I killed people in the process? Millions of people play online poker (many in countries where it is legal) and many of them care. I agree that there is a difference between illegal and not allowed, but that doesn't mean you can do things that are not allowed.

I do see it as an unfair advantage. Yes it is true that the program is following the rules you set, but thinking about just that is not seeing the whole picture. You said it yourself, this program frees up 80% of your time. This is a huge amount of time that you are gaining that allows you to think harder on your other actions or play more tables that you wouldn't be able to play otherwise.

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
Summary of this post: There are no legal issues. Moralic issues might exist, while I only accept them when the bot is far superio to human brain, and so not just in matter of playing houres per day. Post shortened, as some statements of mine are no longer valid with the bot´s recent gain in quality. Also shortend to keep the topic clear for newcomers.
...
I agree that there is a difference between illegal and not allowed, but that doesn't mean you can do things that are not allowed.
As you see, I can. Difference between illegal and forbidden is the difference between cannot and shall not.

I do see it as an unfair advantage. Yes it is true that the program is following the rules you set, but thinking about just that is not seeing the whole picture.
And you do, while you are mixing things up.

The two possible unfair advantanges named are bot to be mixed up.

You said it yourself, this program frees up 80% of your time. This is a huge amount of time that you are gaining...
So far I agree, but it is not unfair to do some work or watch TV while playing.

...that allows you to think harder on your other actions...
Wrong. I cannot think about what to do on the turn before the turn is shown. I also cannot think about how to play my pocket cards when ther are folded automaticly. As poker takes place linearly, chronologicly, it cannot think about ONE part and give away the OTHER part. For every given moment, it´s all or nothing. EITHER the bot plays, OR I do.

...or play more tables that you wouldn't be able to play otherwise.
True. And this is not an unfair advantage, either. Think of it exactly: Being able to play more tables at a time has EXACTLY the same odds than playing simply more hours a day with less tables at a time. It does not improve my decisions, and it is certanly not unfair to be present at a table that otherwise I would not have the time. Entering the market is the hardest impact on it, yet it´s the most fair one.

...
[/i]

TheIrishThug
  • Members
  • 419 posts
  • Last active: Jan 18 2012 02:51 PM
  • Joined: 19 Mar 2006
I understand that this isn't going to move anywhere further, so I will not be posting in this thread after this.

So you are saying if I sold weed, there wouldn't be a difference if I killed people in the process?
Can´t remember to said that. Don´t get pathetic.

Obviously I created an extreme example, but it still follows your logic of "I'm already doing something people say is bad, so it doesn't matter if I do something else that is also bad in the process."

So far I agree, but it is not unfair to do some work or watch TV while playing.
If you don't want to concentrate fully, that is your decision because you decreasing your own ability and not gaining something on your opponents.

I cannot think about what to do on the turn before the turn is shown.
If you are not thinking about how the rest of the hand will play you are putting yourself at a disadvantage. If I have a flush and a straight draw, I might play the river differently based on which one comes. And if I don't want to make it obvious that this is what I am going for, in an attempt to be paid on the river, my turn decision might be different.

Being able to play more tables at a time has EXACTLY the same odds than playing simply more hours a day with less tables at a time.
This would be true if poker was 100% based on the odds. However, folding a hand that would have been best happens often. When you are playing two tables instead of one, you have twice as much to pay attention to. You have to distribute your thinking time between the two. By removing large portion of time that you have to think, you are decreasing the likelihood that you will have two complicated decisions at the same time. Playing 2 tables for an hour is not the same as playing one table for 2 hours.

Lenrius
  • Members
  • 23 posts
  • Last active: Sep 19 2009 02:04 PM
  • Joined: 25 Dec 2008
skydiver, you're my man. I admire your pursuit albeit anybody playing poker earns my disdain.

In finance the one making money is right. Period. St-rip those idealists of their cash. Appropriate whatever you can: It's economic Darwinism and everybody must adapt if he or she is not to drop out of the race.

Success is not the decisions we make, but the route we take. And because I think I can propel you forward

I´d love to hear some useful comments, for sure. Especialy when it comes to ocr or data-leaking.

I invite you to contact me.

lrh9
  • Members
  • 102 posts
  • Last active: Mar 14 2011 08:33 PM
  • Joined: 10 Jun 2009
I too was interested in a bot for poker stars. Admittedly I am conflicted in my desire to use it. On one hand, I'd definitely enjoy using a successful bot to make money. On the other hand, it really is unethical, if not illegal. Then again it would simply be nice to keep it out in the open. Only let it play for play money and let everyone know it is a bot to see how well it fares as an experiment in artificial intelligence. Either way (unethical way to make money or interesting experiment), it's definitely a good programming challenge that tests not only the ability to program, but also one's ability to use logic to play poker. Of course, most poker rooms allow assistant utilities as long as they comply with the following: 1) The utility may not perform any action in game such as auto-folding. 2) The utility may not collect or use information about hands in which you do not play and it may not share the information it collects with others. 3) The utility may not directly instruct you on which action to take. I doubt it would be possible to be caught using a program like autohotkey though. On the PokerStars rules website for instance, it admits that it checks your currently running processes for prohibited software. autohotkey is not prohibited software. Just certain actions with it are. So unless there is a way to check what your script is doing by analyzing the code, they really won't be able to be certain that you're cheating (all though if you go from a fish to a shark, they'll be suspicious and maybe have just cause to tell you to stop using the program, seeing as it's their private site and online casino). In fact, if you can make a rootkit, you can prevent your script from showing up as a running process at all as long as pokerstars doesn't use its own rootkit to obtain the list of running processes. Yes, they will give you one warning. From the viewpoint of Adam Smith (that individuals in economies do what is best for them), it might be appropriate to attempt to use such a utility, and if you are caught, the site won, but you only get a warning. Definitely heed such a warning, because they definitely are catching on if they issue one, and they will ban you from playing if you persist. That's at least 100 dollars down the drain. Speaking about the code specifically, using imagesearch to recognize actions and cards is incredibly inefficient. I'd try to read the information from memory. The only trouble is locating it. Of course, there are other programs out there that have autocard readers. If they utilize dlls to accomplish that, you could use dllcall so long as you know which function to call. I tried to learn to accomplish both, but had no luck.

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
First of all, I will contact who ever asks me to do so, but please be patient.
Lenrius, yo´ll get a message later today.

Irh9, thanks for your thought. I will try to comment some.
(I might miss some, as there are no new lines to indicate new ideas.)

To the ethnical concerns:
I am an advanced-beginner to medium player, winning at small stakes. I consider about 20 stats from pokertracker, and have a quite reliabel intuition. There is no way I could ever write a bot using intuition or an AI using 20 stats - that´s way above my knowledge. So whoever is loosing money to my bot would loose far more to me in person. And as told before: When I am free to play 16h/Day, I am also free to run a "low-quality-me" 24h/Day.

To anti-detection:
Google that. Anti-detection. You can run bot and pokerstars-client on different machines, one controlling the other. You can also run both systems on one, using VMWare. And there are another 100 ways.

As it comes to imput, randomisation is the key - dont click the same pixel over and over. Vary speed and location, and let the bot get a break every view hours.

With AHK, it might be easiest to recompile from time to time with different settings to get different executeables. :)

Implementation
Autohotkey is not ment for botting. If I could, I would do things in C++ or C#. Yet, I cannot. And I love AHK.

How ever. Normally you would go this way:
1)Sneak your self-written DLL into the Clients memory-space to do input right into the variables. 2)Detour everything the client wants to show on the screen to a self-made API to get them without screen-capturing and OCR. 3)Run low-level-high-optimised-code to do all the calcs withhin the 3 seconds you have.

With AHK, and unless I am able to use detour, I need to read screens. This limits my input to things that can be easily recognised, and forces me to slowly read screens. At botting, time is of the essence. THEREFORE, WE WILL NEED SOME C++DLLs TO BE CALLED VIA AHK. The Basic idea is to do the high-level basics in AHK and give some dirty gruntwork to the 50-times-faster Ccode.

So far, let me show what I´ve done the weekend.


Evaluation
Imagine, you have QcQp and the flop says JhQhKh. Got a trip.
But you are beaten by and KK, and AT, any two hearts. What are your chanes?

The modern way to answer that question is the monte-carlo-simulation.

Deal 2 random cards for your opponend based upon his characteristics, deal 2 random cards for turn and river, and see who has won.
Start over dealing other cards. Who won now?

After 100 loops, you have a percentage with high fluctuations.
After 1000 loops, you have the answer with +/- 5%
After 10000 Loops, you have the answer with +/- 1%.
We are good to go. Let´s say your chances of winning the pot are somewhere between 71 and 72%. Well raise :)
If your chancer are only 30 percent, but the pot-odds are 15% - raise.
(By the way, I get the pot-szie realtime with OCR)
But how long does it need to evaluate 2*10.000 7-card-hands in autohotkey? Guess?

Five seconds.

http://www.codingthe...ource_keithrule - I use the cactus-approach.


Conclusion
-Ethical issues are irrelevant.
-Speed of calculations is the key to a winning bot, as precise thoughts are complex.

[/b]

  • Guests
  • Last active:
  • Joined: --

(I might miss some, as there are no new lines to indicate new ideas.)


That's OK. It was more of a flow of my thoughts than anything.

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
I got my AHK-only evaluator to do 6000 evaluations (that is, 6000* getting the winning of 3 hands 7 cards each) in
2.5 seconds
on an 1.6ghz netbook.
The secret lies in a hash-table hardly 100kb in size.

The returned results derive no more than 0.5% from the exact mathematical value gotten with exhaustive enumeration. And who cares if your chancer are 72.3 or 72.8 % ?

That means, Hand evaluation is posible on AHK.


At the same time, I found 2 things I want to point your attention towards:

CMDRet (search the forum) allowed me to call a C#-made command-line tool and save the output into a variable. WOODOO ! Yet it needs several hundred milliseconds to get up and running, so dont use inside a loop - rather get that loop inside the command-line.

HHR_1.03 is a damn nice evaluator for ahk, yet it uses another tool and captures it´s window´s content. Maybe useable, anyway.

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
The more I get working, the more I realise where the limitations will be.
Maybe I will some day write a AHK-Script that converts an AHK script into something reasonably fast like a C#-file. Propably, I´ll learn walking on water before.

When it comes to data-gethering, the main question is:

Just in time or just in case?

Will we always gather any information availible just in case we might need them, or will we get the information needed in real-time?

If we don´t know if we will need the Potsize, should we get it?
Say you got AA dealed. You will propably act preflop no matter what previous actions have been taken. Yet getting the potsize does take about 300 ms (half an etarnity), and you might need it as soon as the flop is dealt.

The more I work with screan-capturing, the more I hate pokerstars for not writing logfiles realtime.

lrh9
  • Members
  • 102 posts
  • Last active: Mar 14 2011 08:33 PM
  • Joined: 10 Jun 2009
Maybe spy++ and managedspy can help you read the cards.

Or this: <!-- m -->http://www.codingthe... ... oker-bot-5<!-- m -->

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
Thanks. Yet cou can be sure that I have done some googling myself, so I sure know the pages of codingthewheel. Actualy, there is a whole page (pokerAI) on that topic.

Recently I found an article how pokerstars changed their API usage to prevent botting. In fact, right now there are two ways left: capture the screen or revere-engeneer and read the ram.

So we can:

Go to aonther casino
keep capturing with AHK
get something faster than imagesearch and implement it via DLLcall


Lenrius
  • Members
  • 23 posts
  • Last active: Sep 19 2009 02:04 PM
  • Joined: 25 Dec 2008
Thanks for the enlightenment in Performance junkie.

It hasn't caught on, but there are consequential, yet non-intrusive ways to get what might eventually be displayed into AHK even before the destination process gets to process the information. Moreover, CMDRet usage can be improved upon by making a straightforward DllCall() to one's function. The most can be gotten not from dynamic linking, but out of static linking.

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
Hello,

I am not sure I got your idea.
What I did understand is that you are suggesting to directly create DLLs instead of creating command-line-apps. That, definitly, makes sense.
I will try to get hands on a hallo-world-dll to understand the way of workflow and keep you posted.

I believe there are two things I would love to have in DLL: the evaluator and a screen-reader. Both together are taking about 80% of my critical CPU-time.

Please could you explane the first sentence "It hasn't caught on, but there are consequential, yet non-intrusive ways to get what might eventually be displayed into AHK even before the destination process gets to process the information. " To be honest, I have no idea what you ment.

Thanks

skyd1v3r
  • Members
  • 73 posts
  • Last active: Apr 18 2010 11:42 AM
  • Joined: 18 Jun 2009
So how about sharing?

I got some offers to contact people and exchange code. So far, none of them realy took a step forward. So I want to repeat my offer to exchange both code and knowledge.

To get concret: My AHK-native hand evaluator, one of the most important parts of any bot, can be obtained. Currently I need AI parts like a nash equilibrium calculator or a postgres-access. Don´t get me wrong, I have some general postgres-code. The deal is valid for everyone who provides a function where I can pass the player´s name as argument and get his VP$IP returned out of my pockertracker-DB.