How we perceptually organize a visual stimulus depends not only on the stimulus itself, but also on the temporal context in which the stimulus is presented as well as on the individual who is processing the stimulus and context. Earlier research found both attractive and repulsive context effects in perception: tendencies to organize visual input in a similar way as preceding context stimuli (i.e., hysteresis) co-exist with tendencies that repel the current percept from the organization that is most dominant in these contextual stimuli (i.e., adaptation). These processes have been studied mostly on a group level. In this study using multistable dot lattices as stimuli (N = 75), we implemented a Bayesian hierarchical model comparison approach to investigate whether true individual variation exists in the size of these effects, and whether everyone shows both effects in the expected direction. Furthermore, we investigated temporal stability of these individual differences across time (one to two weeks apart). The results demonstrate that large individual differences in the size of these attractive and repulsive context effects exist. Furthermore, these individual differences are highly consistent across timepoints. Although almost everyone showed both effects in the expected direction, not every single individual did. In sum, the study reveals how individuals differ in how they combine previous input and experience with current input in their perception, and more generally, this teaches us that different individuals can perceive identical stimuli differently, even within a similar context.