For those interested in playing online poker, issues of security have typically been over insecure network connections and identity theft. Recently, however, glitches and bugs in sites such as Ultimate Bet and Absolute Poker (both part of the Cereus network) and now Cake Poker have players questioning whether their rolls are safe on any online site. Others have begun to re-raise the question of whether a government or other outside entity should be charged with regulating online gambling.1
In early May of 2010, the Cereus sites were widely outed online by Poker Table Ratings (PTR), a website that found weak security code allowed for players’ hole cards (and possibly their personal information) to be spied on by hackers. The PTR revelation led to swift action by Cereus executives and the problem was fixed within a relatively short time. Among online players, it was widely assumed that all online poker sites would learn from the troubles faced by Cerebus and check over all their own code and encryption. Apparently this assumption was also made at Cake Poker.
On Monday, July 26, 2010, PTR revealed that nearly the same problem with weak, in house made security code existed at Cake Poker (and thusly, at all skins under the site umbrella of Cake, which include Doyle’s Room and Bruce Poker among others). The response was not so swift by Cake, the problem remaining active on their site until August 4 when the SLL was repaired for the original version of Cake’s software. The beta version was corrected on August 5 and additional skins were left possibly the same as before, with the burden on individual players left to check for a .dll file.
Response to this issue has been heated in online poker forums, especially at 2+2 where players repeatedly questioned whether this issue was actually a software problem or could have been insider hacking by Cake’s own programmers. Despite differences of opinion on the how and why, many on the forum stated intent never to play on Cake sites again, likening them to the wounded Cereus sites Ultimate Bet and Absolute Poker. Meanwhile, the majority of posters indicated that they feel confident only in top tier sites like Full Tilt Poker and Poker Stars. Cake’s spokesman, poker professional Lee Jones has taken a lot of heat on the boards and even snapped at PTR on 2+2 claiming that they should have brought the issue to Cake’s attention in private. This suggestion by Jones sparked further outrage that Cake players were never notified about the problem by anyone at Cake. Furthermore, many posters questioned why Cake did not freeze its site while the problem was investigated and fixed (as it had done in February following a pot-shifting problem).2
The pot-shifting bug should have set off alarms at Cake, since their issue was very similar to one at Ultimate Bet,3 which would seem to indicate that the two companies were employing similar buggy software.
Of specific issue were statements on Cake’s website that the fully protective SLL encryption had always been in use on Cake, which has been revealed was not the case, Cake was in fact using their own encoding, an XOL. In his statement after the incident and its fix, Cake spokesman Lee Jones stated:
“One of our programmers, under a schedule crunch, replaced the TwoFish implementation with the XOR encoding. Obviously, that was a bad idea. There was some technical discussion about this change, but unfortunately we didn’t go back and redeploy the TwoFish (or an SSL) encryption in the code. Equally badly, nobody thought to update the website to reflect the fact that the TwoFish implementation had been removed. That was a classic lack of communication between the technical people and the people who maintain the website. Again, no excuses; we dropped the ball.”
Posters on the 2+2 and other forums pointed to this as lying or at least reckless misrepresentation. PTR’s initial report on the issue indeed raises the question as to whether this misrepresentation was in fact malicious.
By checking Pokerscout.com, it is easy to see that Cake Poker is sliding. Poker Scout Stats Though it remains the fourth largest U.S. provider, its numbers have dipped significantly and have continued to suffer during the last six months. Although representatives of Cake Poker, including the aforementioned Lee Jones, have discussed the issue on various poker forums, the Cake Poker security page makes no mention of the problem, which means that new users to the site will have to have done their research in order to know anything about the issue.4
Regardless, online players want to be able to play poker safely, with confidence that the software which deals their cards is free of bugs, glitches, and cheating. These are not unreasonable demands. In fact, they are the very same expectations that players would bring to playing live poker. In the meantime, many posters to online forums have been encouraging online poker regulation and others have been saying that there will never be safe online play- that the only solution is to play your cards in person.
Thread on Two Plus Two
Lee Jones Answer to issues
History
* When the Cake software was first written five years ago, it included an implementation of the TwoFish encryption algorithm in the server-client communication.
* Approximately 18 months ago, the TwoFish code stopped working because of a change in an unrelated part of the client. One of our programmers, under a schedule crunch, replaced the TwoFish implementation with the XOR encoding. Obviously, that was a bad idea. There was some technical discussion about this change, but unfortunately we didn’t go back and redeploy the TwoFish (or an SSL) encryption in the code. Equally badly, nobody thought to update the website to reflect the fact that the TwoFish implementation had been removed. That was a classic lack of communication between the technical people and the people who maintain the website. Again, no excuses; we dropped the ball.
* When the Cereus issue came to light in May, I asked our VP of Software Development (who’d only joined the company five months prior) if our implementation was more secure than that. He asked the programming team and was told that we were more secure than Cereus; that is what he reported back to me. We are still trying to understand exactly what he was told and why. As you might imagine, for the last week, we’ve been much more interested in actually solving the problem than trying to figure out how we got here. We have now started the post-mortem process to completely understand what happened in the first place. We will, I promise, get to the bottom of what happened.
* Once the vulnerability was publicly published, we dropped everything and turned to fixing it. All discussion of the events leading up to the problem was put on hold. As somebody who’s been through that sort of thing before, I completely stand by that decision. Spending time pointing fingers when there’s work to be done achieves nothing.
Current situation
* As I said before, once we became aware of the specific problem, a team of our top people went to work on the problem. They worked exceedingly long hours under the highest pressure imaginable. Not only did they implement the SSL layer in all server-client communications, they also added peer verification to ensure that nobody could mount a man-in-the-middle (MITM) attack. That release was completed Thursday, August 5th.
* We also instituted a full security audit of all aspects of our software. It is still ongoing, but we want to be sure that we have no issues remaining. So far, we’ve not found any other problems.
* While we believe that nobody has lost any money to an exploitation of this vulnerability, we are taking no chances. We are doing a full audit of the top winners since July 26th (when the vulnerability was first reported) and also the largest pots that were played. Once we have completed that audit, we are going to expand the search and investigate back to the time that the TwoFish implementation was removed. Again, I don’t believe we’re going to find any player losses, but we have a responsibility to do the audit. Serge Ravitch (adanthar) is heading up that audit. You may recall that he’s one of the people who uncovered the PotRipper problem; he’s an expert at these things.
* As we complete each phase of the audit, we will turn the hand histories over to Jeff (Yellowsub) Williams, who many of you know. Jeff will have access to whatever data we used for the audit plus whatever else he requests. We will pay him some appropriate amount for his time, but we feel that his stature in the poker community is such that he won’t risk his integrity for the amount of money involved. Jeff will report his findings to the community whenever and however he sees fit.
The Future
Obviously, we don’t want to repeat anything like this. Our VP of Software Development has instituted new procedures and protocols that will ensure a single person can’t go in and change the design of any code without review, oversight, and approval. He is relatively new on the job but I am impressed with his understanding of software design methodology and his commitment to running a tight ship. I believe his new methods will make us a better software company and thus a better poker site.
Questions
Q: Why did you leave the site running once the vulnerability had been disclosed?
A: After discussion with our technical experts, we felt that the actual practical risk was low, taking into account the thorough processes our player security team has in place. Furthermore, we are on heightened security alert. We go over every cash-out carefully anyway (spending many man-hours each day reviewing them), but since we became aware of this situation, we’ve put an extra magnifying glass on the whole process. We intend to make sure that nobody cashes out illicit funds (if there are any). In short, we felt we could adequately manage what risk there was. In retrospect, I believe we were right; we have no reason to believe that anybody was cheated either before or after the vulnerability was made public.
Q: What has happened (or will happen) to the programmer(s) who originally did this and misrepresented the situation to you and others?
A: We believe that’s a personnel matter that should stay within the confines of Cake Poker.
Q: Why did it take you so long for you to provide this information to the public?
A: Two reasons:
1. As I said above, we had our hands full assessing and fixing the technical problem. It was made more complex because we had to release it across all of Cake’s partners as well.
2. There are many stakeholders in this company. The Cake Network has approximately 50 partner sites, many of whom had an opinion on what we should or shouldn’t say. Pursuant to my relationship with Cake, it is not my place to just say whatever I want whenever I want; that’s not how it works. So those decisions – what to say and when – had to be hammered out across many emails, IMs, and phone calls.
Lessons
All of us at Cake have been humbled by this whole experience. It was disturbing not only on a technical and organizational level, but because of the strain it put on our relationship with our customers and the poker community at large. Personally speaking, I have learned some important lessons too and they’ll go forward with me in my career in this business.
We are sorry for the trouble and concern this has caused and hope we can go back to simply running a site which gives people a great place to play poker.
Best regards,
Lee Jones
Cake Poker Cardroom Manager
Play Online Poker
Subscribe to PokerBloggs.com
Recent Comments