Memory retrieval involves two processes: familiarity (indexed by low self-confidence recognition) and recollection (indexed by high self-confidence recognition). It is unclear which memory process is impacted by emotion that leads to widespread emotional memory enhancement phenomenon. Previous studies have found that negative or positive stimuli can call cognitive resources involved in encoding to improve memory, and arousal stimuli can activate the connection between amygdala and sensory brain area, which leads to recollection of more details. Therefore, we speculated that valence and arousal of emotion may have different impact on different memory process. Valence may impact familiarity and recollection by calling sufficient cognitive resources in encoding, but this effect would be adjusted by the amount of cognitive resources; whereas, arousal may affect recollection automatically, not restricted by the amount of cognitive resources. Therefore, in experiment 1, we manipulated valence and arousal separately though orthogonal design to examine their effects on different memory processes under sufficient cognitive resources condition, and, in experiment2, we added a secondary task in encoding stage to manipulate the amount of cognitive resources for encoding to examine the cognitive mechanisms underlie the emotional effect on memory process. A total of 86 subjects participated in the experiments, with 56 in experiment one and 30 in experiment two. 432 emotional pictures were selected from IAPS and were divided into five categories: negative low arousal, negative high arousal, positive low arousal, positive high arousal and neutral. There were 144 neutral images and 72 pictures in each other subsets. Half pictures of each emotional subset were old item and half were new, balanced across subjects. In order to meet the requirements of orthogonal design, only the data of emotional pictures were analyzed to ensure that no difference in arousal values between positive and negative pictures and no difference in valence values between low arousal and high arousal pictures. Both the experiments were divided into an encoding stage and a retrieval stage. In Experiment 1, participants assessed the valence and arousal values of images in a 9-points scale in the encoding stage, and made high and low confidence recognition judgment to old and new pictures at the retrieval stage. In Experiment 2, in order to restrict the encoding cognitive resources, a distraction task was used in the encoding stage in which subjects were asked to count numbers while assessing value of valence and arousal of images. The rest of the programs were same in both experiments. The independent variables of present study were valence (negative, positive) and arousal (low arousal, high arousal), the dependent variables were memory performances on high and low confidence recognition judgments, including hit rate, correct rejection rate and discrimination accuracy (Pr). In Experiment 1,the results showed that compared to negative images, the discrimination accuracies (Pr) of high and low confidence recognition were significantly higher for the positive images. The hit rate of high confidence recognition was significantly higher for low arousal pictures than high arousal pictures. In Experiment 2, the results showed that the concurrent task reduced the enhancement effect of positive pictures, Discrimination accuracy (Pr) of high confidence recognition of high arousal pictures was significantly higher than those of low arousal pictures, which implicated that the effect of arousal was not affect by the secondary task. These results suggest that positive emotions can call sufficient cognitive resources for encoding, thereby improve the memory accuracies based on familiarity and recollection process. High arousal emotions can enhance the memory performance based on recollection process even when the encoding resources is diverted to the secondary task, so the enhancement memory performance of high arousal stimuli is caused by automatically encoding details of the stimuli.