To aesthetically appreciate a stimulus, we first need to perceive it. The principles underlying perceptual organization could therefore play a fundamental role in delineating our aesthetic experiences. From a Gestalt psychological perspective, we aim to perceptually and psychologically organize the incoming stimulation in the best way possible given the prevailing (internal and external) conditions. Two interacting tendencies support the emergence of such a better organization: as we compare the incoming stimulation to a reference (either internally represented or locally present), both downplaying unessential deviations of the input from the reference (i.e., simplification) and intensifying characteristic differences between input and reference (i.e., complication) can increase the clarity and goodness of the experienced organization. But how exactly does goodness of organization relate to aesthetic appreciation? Whereas some researchers argue for a direct one-on-one relation between the goodness of an experienced organization and its positive aesthetic appreciation, other researchers emphasize how an increase in organizational goodness (i.e., experiencing stronger organizational tendencies) may drive aesthetic appreciation. Although this debate is unresolved and requires further empirical investigation, we argue for a nuanced view encompassing both perspectives. Just like artists first need to acquire artistic conventions before they can meaningfully deviate from them, our perceptual systems first need a minimal reference system before they can welcome new levels of intricacy and complexity. On the other hand, increasing complexity can stimulate the development of new reference organizations and ultimately allow for a more organized perceptual world in the long term. As artists apply the mentioned perceptual organizational tendencies of simplification and complication in their artistic practice, one could stipulate that “perception is artistic”. Individual and contextual differences will have a considerable influence on the structure of the reference system for a particular type of stimulation, and therefore also an influence on which organizations are positively aesthetically appreciated.