Moral responsibility judgement and legal responsibility judgement are the two major contents of the social informative learning. Recently, social psychologist and neuroscientists have become greatly interested in exploring the cognitive process and neural basis of moral and legal judgment. Plenty of studies have showed that the belief attribution and outcome of moral events played key role in moral judgment. The traditional moral judgment theorist believed that the moral attribution develops with the growing of age: people made moral judgment based on outcome of event at the beginning of their life and then transferred to focus on the belief of agent. However, recent researchers challenged this statement and suggested a dual process model to interpret the moral judgment of actions. Some studies showed that the judgments of the wrongness or permissibility of action were relied mainly on the mental states of an agent, while judgments of blame and punishment were relied jointly on mental states and the causal connection of an agent to a harmful consequence. Although many theories existed to account for moral judgment, few studies have focused on the different cognitive mechanism of moral responsibility and legal responsibility. In addition, previous researchers have neglected the difference whether the agent was an actor of harmful action or an observer in moral (legal) stories. Therefore, in this study, we intended to investigate how belief and outcome valence influence moral and legal responsibility judgment in these two conditions. In the current study, two scenarios were differentiated by whether the agent was an actor of harmful action or an observer, which labeled as “conductor” scenario and “bystander” scenario. The belief of agent and causal responsibility were manipulated within each scenario, yielding four different conditions separately: neutral belief and neutral results, negative belief and neutral results, neutral belief and negative results, negative belief and negative results. The participants were asked to make moral and legal judgment for the action of agent in these different conditions. Results demonstrated that the differences of moral and legal responsibility judgments in the two scenarios were threefold. Firstly, the belief of agent is more important than outcome when participants made moral responsibility judgment, while they are more sensitive to the outcome of events when they judged legal responsibility. These data demonstrated that the role of belief and outcome were different when different kinds of responsibility judgment were asked to make. Secondly, the moral and legal responsibility scores are significant higher in the “conductor” scenario than that in the “bystander” scenario, especially the legal responsibility scores. The different scenarios yield different responsibility judgment might due to the different sense of agency and self-involvement influenced social normative judgments. Thirdly, a significant interaction between belief and result was only observed in the Exp. One. And further test showed that the difference between neutral belief and negative belief was larger when the result was neutral and the difference between neutral result and negative result was larger when the belief was neutral. Together with prior studies, these results might suggest the psychological states and causal attribution work in a dual model of principle judgment. The findings in our study help to understand social normative judgments in complex situations.