What makes people shy away from risk, risk aversion or regret aversion? The current empirical evidence is mixed. Based on the domain-specific perspective, this paper pointed out that not all risks are created equal, whether decision makers are risk aversion or regret aversion is domain-specific. To test this hypothesis, we conducted this study in Macau, the largest casino city in the world. A total of 370 Macau adult residents with gambling experience completed a questionnaire survey dealing with thirteen types of gambling. The results showed that the negative effects of risk perception and anticipated regret on gambling behavior were domain-specific. Participants have not shown themselves to be consistently risk averse or regret averse across different gambling types. In the first category domain which included Baccarat and Greyhound Racing, both risk aversion and regret aversion worked together to lower participants’ gambling frequency. In the second domain of Roulette and Stud Poker, it was risk aversion that was responsible for participants’ unwillingness to bet. In the third domain which included Mahjong and Paikao, it was regret aversion that was responsible for the lower gambling frequency. In the fourth domain of Fantan, Cussec, Horse Racing, Chinese Lottery, Football Lottery, Blackjack, Slot Machines, neither risk aversion nor regret aversion was responsible for participants’ unwillingness to bet. Our findings indicated that neither risk aversion nor regret aversion can uniquely explain an individual’s risk-taking behavior consistently. Instead, which factor plays a greater role in lowering gambling frequency—regret aversion, risk aversion, or both—is itself dependent on the type of gambling involved. These findings could be useful for the further understanding of individual’s risk-taking behavior. Practical applications for problem gambling prevention and advertising appeal were given by providing a basis for understanding the role that cognitive and emotional factors play in different types of gambling.