01-17-07 02:20 PM
Reply
Would you like to prove to yourself that Sixth Sense Table Selection can really help your game? Take the Sixth Sense Challenge!
The link below will install and run a program on your machine that can analyze your past games based on Sixth Sense's Fish Score. If you have enough data, it will show you that your Win Rate will improve by selecting tables that have a high Fish Score.
Select the Download link, choose "Save As", save the file to your desktop, then run the program from your desktop.
Post your results here so we know how you did!
01-17-07 02:21 PM
Reply
The Sixth Sense Challenge
This program lets you see for yourself how powerful Sixth Sense really is. It does this by going back to games you have already played, scoring those games with our unique Fish Score, and generating a report that shows your winnings based on the good tables (high fish scores) and bad tables (low fish scores).
You will need to select the folder where you HH files are located. These must be the HH files that show YOUR hands. You can also add observed HH to the list. This will give you better stats on opponents, but take longer to load and process.
After selecting the folders to load, the report will be generated in the background. You can view the progress from the “Jobs” tab. You can also Refresh the report with the refresh button, this will use all the currently processed data.
Note that the report data is meaningless unless you have LOTS of hands, about 30,000 hands PER DIVISION is required to get somewhat reliable results. The more data the better.
Example Report:
Report Divisions: 2
1 of 2: Hands 302,821, Fish 8.9, BB/100 2.08 +/- 0.9
2 of 2: Hands 310,399, Fish -2.6, BB/100 1.17 +/- 0.8
Report Divisions: 4
1 of 4: Hands 147,622, Fish 13.2, BB/100 2.66 +/- 1.4
2 of 4: Hands 155,199, Fish 4.7, BB/100 1.54 +/- 1.2
3 of 4: Hands 155,200, Fish 0.3, BB/100 0.90 +/- 1.1
4 of 4: Hands 155,199, Fish -5.6, BB/100 1.44 +/- 1.2
Analyzing the Example Report
An example report is listed above. The report’s first section breaks down all the games played into 2 divisions, low fish scores and high fish scores. The player’s Win Rate (in BB/100) is calculated for both sets of games. Generally a player will win more money at tables with players with high Fish scores, and win less at low fish score tables.
In the report above, 612,000 hands were loaded, so this is enough data to make a good conclusion. For the good tables (with an average fish score of 8.9), the player won 2.08 BB/100. Note that there is a +/- value of 0.9. This means the calculation of the Win Rate is known to be 2.08 plus or minus 0.9 to give us a 95% confidence interval. That means we are 95% sure the Win Rate is between 1.18 and 2.98.
Note that the players Win Rate drops to 1.17 BB/100 when playing at tougher tables that have an average Fish Score of –2.6. The player’s Win Rate would have doubled if they only played at the top 50% of the tables based on Fish Score. So Table Selection does matter!
If the confidence intervals are large enough that the high win rate interval overlaps the low win rate interval, the results are inconclusive. This happens above. But since we have a large amount of data, we can be somewhat sure (confidence 75%) that Sixth Sense would’ve helped this player if they had played the top 50% of the tables by Fish Score.
The report will continue to break down the data into smaller divisions, and stops when a division would have less than 30,000 hands. In the report above with 4 divisions, you can see that the Win Rate goes down as the Tables get worse, which makes sense. But for the fourth division (the worst), the Win Rate bumps up a bit. This could probably be explained by variance.
01-17-07 05:38 PM
Reply
Bugs:-
Just run your tool up and a couple of things has sprung up
1) I run the tool, and it produces the results. However if i run the tool again without restarting and pointing at the same directory it counts the files twice. 3rd time of running, 3 times etc.
2) I know you say the results are meaningful for less than 30k in each category. However, my reported win rates are 70BB/100 (high) and 65BB/100 (low) on 6k hands for each division. Not sure how it comes to be as I was running more like 7ptBB/100. P.S. Are BB, Big Blinds or Big Bets (i.e ptBB/100)?
Suggestions:-
Cool tool, however couple of suggestions
1) Is there any way you could adapt this to run straight through a PT DB? I think that would be a serious improvement. Cos obviously I can get all the hands back out of PT into files using the available PT function, but I wondered if it would be a big job to change your tool to analyse the db?
2) Also another biggy for me would be able to do this against my custom rules.
All these extras are really making the monthly subscription worth it, keep it you man, your a star.
01-17-07 06:11 PM
Reply
I had considered doing it that way, but it was just easier to use the HH files. I think its faster, not sure. But it would be a LOT of work to convert it to using a PT database, so I won't be doing that.
I think most people save their HH files, don't they?
Let me know if you try it on your data and what you find.
-Ben
01-17-07 06:13 PM
Reply
One note, the job only gets started if you click the "Next" button on the "HH Files" page. If you just click straight to the "Results" tab the job won't start.
01-17-07 06:34 PM
Reply
Please see above for problems etc
01-17-07 10:39 PM
Reply
1) - ok, its not clearing out the old lists.
2) - 70 BB/100 sounds pretty high. It is supposed to measure Big Bets per 100 hands played. This tool is using the same parser I used in PokerManager. Maybe you could download the PokerManager trial, and load those hands into it, see if you see anything strange there. Or setup a new folder with just one small file in it, to help narrow down the problem. For 6k hands, you could see a Win Rate of 1-8 BB/100, with +/- 10 BB or so for the confidence interval.
Hmm, if you have 70 high, and 65 low, you're average is 67.5 with +/- 2.5? That doesn't sound right.
2.1) - Too much work to do that anytime soon.
2.2) - Custom rules would be a great addition, I might add that. I was anticipating that somewhat. If you look at the Player Stats tab, you'll see the last column is "Win Vs Player". This is just a stat how often you won or lost to that player. Its very high variance though (and it might be a bad way of doing it). I was hoping there would be some high correlation between that stat and fish score, but so far it hasn't correlated. If it correlates to your custom rules that would be great though.
01-18-07 08:21 AM
Reply
I get +- 12.5 in the results not +- 2.5.
I will try will a much bigger sample set in the next few days.
01-18-07 09:04 AM
Reply
OK, that makes more sense. So the 70 is for high fish scores, and 65 for low fish scores, each about +/- 12.5.
01-18-07 10:38 AM
Reply
Yeah that it.
Just run another test. Exported 30k hands out of my PT database to file. Ran the SS challenge on it and still crazy results.
Div 1 of 2: Hands 29,978, Fish 13.1, BB/100 55.39 +/- 4.4
Div 2 of 2: Hands 29,978, Fish 2.3, BB/100 54.05 +/- 4.8
Over that period I ran at 5 PTBB/100 according to PT.
Another thought I had, at the moment if you add a directory it just checks out files in that directory, and none in subdirectories. However, the standard way in which PP stores HH as I'm sure you are aware is Hand History->UserName->Date. Would be good to add just the Hand History Folder and program get all the files from below that in the file structure.
01-21-07 11:52 PM
Reply
Oracle,
Please download & install build 2 of SS Challenge. This has a "Game List" tab under the Results tab. After doing a refresh, you can press the button to export a list of all the games. This will list each game, along with the BB Hero Won. You could inspect this for strange values, and then see which HandNum's are giving problems. You could then email me that hand and I can check to see what is going wrong with the parser.
BTW, if you have new comments just post a new message rather than edit your old one. If you edit your old one I don't get an email notification, so I might not reply.
01-22-07 06:26 AM
Reply
Tried new version out on a test set and still issues. I look at the log file and I can't see anything obviously amiss (no you won 1000BB or anything stupid). However, when I look at results tab the amounts won seem to be out by a factor of approximately 10, hence the BB being out by a factor of 10. Although the actual number of hands doesn't match exactly what I have down in PT (from which the hands were exported).
Have sent an email to sixth sense support about.
01-22-07 09:11 AM
Reply
Could you send maybe 5-10 sample hands from this dataset.
In my code, there is a section that says set BB to 1 for the hand if it can't figure it out. That might be causing the problem if it can't figure out the BB for the hand.
01-22-07 09:18 AM
Reply
I also put up a Build 3 that now has a "Debug Parse" tab. You can paste in the text for one hand history, hit "Debug", and it will calculate all the stats for that hand. You can then check if the parser is wrong.
01-22-07 09:28 AM
Reply
FYI - my testing was with original PartyPoker HH files, if you have any of those maybe you could test with them.
01-22-07 01:50 PM
Reply
Thats seems to work ok. You are really starting to get a great packages of products together now.
01-22-07 03:14 PM
Reply
Cool. Try running it on a large data set, let me know how it goes.
-Ben
01-22-07 04:55 PM
Reply
Found another problem. Now, it seems like you count the hands twice. I imported 17k hands, and it showed up as 34k, and it also had my win amount as twice as large as it is. So obviously BB/100 is still correct, but not the no of hands and amount won.
01-22-07 05:26 PM
Reply
Well, if you shut it down and run it from the beginning, does it still show it twice?
01-22-07 05:31 PM
Reply
I just put up Build 5 which I think will fix the problem of not reseting the stats when you create the report a 2nd time.
01-22-07 06:03 PM
Reply
No still has twice as many hands and twice as much won. All the other stats look about right.
01-22-07 06:13 PM
Reply
Ooops, I missed another spot where it was keeping the old files loaded. Fixed now in Build 6.
01-22-07 06:31 PM
Reply
No this problem occurs on the first run. I have tried it again with build 6 just in case, and still problems.
Update:- I think I have an further info on the problem. I just tried importing files from one directory, and this counted the correct number of hands. However when I add the 2nd directory (incidently empty), it says it is parsing 2 x no of files in directory 1. I think you have a bug in your code there.
As an extra I think your parser should check that no hand is imported more than once, as if people export their hand histories out of multiple DB's there may well be overlaps between the data they store.
01-22-07 07:29 PM
Reply
Each folder added is that folder and ALL sub folders. So if you add a folder that is beneath one already added, you'll get them loaded twice.
I'm fixing it so that even if that happens they shouldn't get loaded twice, that will be Build 7.
Hope that made sense.
01-23-07 12:15 AM
Reply
Hi,
Does this tool work with HHs from FTP? I pointed the app to a directory where I had exported all my DEC & JAN hands out of PT. But when I hit 'Next' the job ran in about 1 second and the results showed zero hands.
Thanks!
01-23-07 03:36 AM
Reply
No, one is not a subdirectory of each other. They are actually both just folders on my desktop. One for observed hands and one for played hands.
01-23-07 08:19 AM
Reply
grunch - what file extension was used? It must be .hhf or .txt. Full Tilt hands should work, its better to use original hands than PT exported hands though.
Go to your "Log" tab, does it show any "Unknown Line formats"?
01-23-07 08:22 AM
Reply
Oracle - is there any chance that the observed hands and played hands overlap? Maybe possible if you had a 2nd machine doing the observing. And were you using Build 7?
I don't know what else could be causing that problem.
01-23-07 12:07 PM
Reply
Seems to be working fine now with build 7.
01-23-07 12:10 PM
Reply
I'm pretty sure the file extension was TXT. They were exported from PT. I don't think there were any error messages in the log tab. I'll double-check both these things tonight after work.