Effects of Facial Symmetry and Gaze Direction on Perception of Social Attributes: A Study in Experimental Art History

Introduction

In this article, we examine the possibility to test hypotheses about art production in the past by collecting data in the present. Similar work has gained recent attention in experimental archeology (e.g., Coles, 2010; Ferguson, 2010), where hypotheses about the past are investigated through the production and use of tools and objects in the present. We are not as concerned with the technical aspects of art production per se, but the plausible detection of artistic intentions and reception of artworks in the past. Specifically, we examine basic perceptual processes that presumably have not changed significantly over the centuries. Artistic norms might counteract these basic perceptual processes, but it is more likely that they are in line with basic perceptual and emotional processes and biases. Our contribution provides an example of how interdisciplinary research that includes art historians and psychologists might address a question from Western Medieval and Renaissance art by means of psychological experiments. This endeavor is called “experimental art history”.

We start with the observation that in this period the majority of portraits of Christ were frontal with a gaze directed toward the beholder. In this context, we define portrait as an image of one person alone showing only his/her face or the upper part of the body, painted on canvas or wooden support. The representations of identifiable persons in larger compositions, group portraits or narrative settings are not considered in our study. The frontal portraits (with very few exceptions) represent Christ as God, i.e., what is labeled the Holy Face (as opposed to the profiles, where He is the sufferer, the Man of Sorrows). This study looks at Christ as God and will exclude the Man of Sorrows. Previous studies have demonstrated that in the 15th and 16th centuries, almost all profane portraits (in contrast to the depictions of Christ) were depicted in different degrees of profile, very seldom in frontal view (Hodne, 2013). Famous exceptions were Albrecht Dürer and Hans Holbein the Younger. We notice the same tendency in later periods as well. Why did these artists prefer to paint Christ with his face directed towards the beholder, while profane faces were noticeably more often painted in different degrees of profile? Art historians usually take recourse to historical sources in order to answer such questions. There is a strong tradition in the West of copying the veil of Veronica as a template for the face of Christ. The blood and sweat on the relic was thought to be imprinted on the veil directly from the face of Christ by blood wiped from his face during his way to Golgotha. According to tradition, the intensity of Christ’s gaze in the veil made it necessary to cover the relic with a piece of cloth. The highly venerated relic was kept in the Old St Peter’s basilica (Rome). This symmetrical face with the strong direct gaze became the standard way to represent the Holy Face in Western art of the Renaissance both north and south of the Alps (Morgan, 2012: p. 55–62). But could there not be other reasons for the strong preference of full frontal portraits with a directed gaze? Such reasons could point to factors deep in human emotional responses to face perception.

We wanted to find out whether convention is the only answer to the Renaissance preference for representing faces of the holy in frontal view. Can we exclude that preference for frontal faces in the depiction of deity may have deeper evolutionary and biological reasons? Can there be other and biologically driven mechanisms leading to a preference for the strongly symmetrical face (i.e., the portrait in frontal view, as opposed to half-profile) for holy persons, and can this answer the question why Christ is depicted in frontal view and profane faces in half-profile?

We develop the idea that experimental art history can examine the plausibility of different hypotheses by assessing the effect of different face and gaze directions on modern viewers of portraits. Hence, are there deeper reasons for painting Christ in frontal view and profane faces in half-profile? Surely these masters were able to paint a face in whatever orientation they liked, so the possibility that the painter’s competence accounts for the result is not a valid explanation. We instead assume that the difference stems from the differential effects artists wanted to achieve through using our perceptual and emotional biases in face perception. However, as the short review of the reception and meaning of the veil of Veronica shows, the prototype became the paradigm about how the Holy Face should be represented. But could it have stayed the prototype for so long, if it had no other support?

Moving now to modern psychology, both the literature about gaze and the knowledge about the activation in the brain, in response to both face direction and gaze direction, has expanded tremendously during the last few years. A fast capture of head and gaze direction is a significant factor for detecting the other’s intention (Emery, 2000), and it must have represented a strong selective pressure during evolution. The accurate detection of gaze direction depends on the great contrast between the dark iris and the bright sclera. Kobayashi and Koshima (1997) found that among 88 primate species, only humans had eyes with a white sclera and a dark iris. In language acquisition, the detection of shared or joint attention, at the lowest level through detection of gaze direction, “may be important for language learning in human infants” (Emery, 2000, pp. 588; see also Tomasello and Farrar, 1986; Dunham et al., 1993; Mundi and Gomes, 1998). This illustrates the importance of gaze direction in human evolution, with precursors in other primates (see Emery, 2000, particularly pp. 584–587).

Researchers have focused on the perception of gaze direction and facial expression under different head orientations, (e.g., Langton, 2000; Todorović, 2006; Todorović, 2009; Ewbank et al., 2009; see also Wollaston, 1824). Advanced single cell recordings from the temporal cortex of macaque monkeys yield important data on the neuronal correlates to different directions of eye gaze, and head orientation, laterally as well as vertically (Perrett et al., 1985, 1992).

People who look directly at their counterpart could signal aggression or superiority, they could want to monitor the other’s actions, but they could also express the wish to communicate with or to care for their counterpart. This activates the brain’s so called Theory of Mind (ToM)-network, which is the social network through which individuals analyze another person’s intentions (e.g., Perrett and Emery, 1994; Baron-Cohen, 1997; Conty et al., 2007; Yang et al., 2015). The five main anatomical hotspots in the ToM network are the superior temporal sulcus (Perrett et al., 1992 ; Allison et al., 2002; Calder et al., 2002, 2007; Pageler et al., 2003; Conty et al., 2007; Hoffman et al., 2007; Carlin et al., 2011 [single cell recordings from macaque]), the fusiform gyrus (Wicker et al., 1998; George et al., 2000; Pageler et al., 2003), the medial prefrontal cortex (e.g., Calder et al., 2002; Conty et al., 2007), the orbitofrontal cortex (see Conty et al., 2007; Senju and Johnson, 2009) and the amygdala (Senju and Johnson, 2009)¹. The definition of ToM is currently under development; while the macro results are not questioned, the micro explanations are still unclear (Schaafsma et al., 2015). For a review of research on the neural mechanisms in face and gaze perception, see Emery (2000), Haxby et al. (2000), Itier and Batty (2009, pp. 849–857), Nummenmaa and Calder (2009), as well as Senju and Johnson (2009).

Could the perceived face and gaze direction be related to fundamental emotional reactions such as fear or fight or, on the positive side, empathy and sympathy? When it comes to the question of how the different neural structures listed above interact during eye contact, Senju and Johnson (2009) explain the eye contact effect through a so called fast-track modulator model (Johnson et al., 2015): according to this model, there is a fast and unconscious subcortical route of face and gaze information (the fast track modulation route) including the superior colliculus, pulvinar, and amygdala (Laeng et al., 2010). Activity in this pathway mediates the affective valence of the stimulus by amygdala-activation and activation of the orbitofrontal cortex. The existence of such a pathway is supported by evidence from research in human newborns (Johnson et al., 2015), as well as from research on patients with cortical blindness (resulting from destruction of the primary visual cortex); despite this loss of primary visual cortex, some patients show activation of the amygdala as a response to facial and bodily expressions of emotion (Burra et al., 2013). In addition to the fast-track-modulation, there is a slow information processing path running through the lateral geniculate nucleus, visual cortex/lateral occipital cortex and inferior ventral temporal cortex, where the gaze direction is decoded by the anterior superior temporal sulcus, and face identity in the fusiform face area. The intentionality of gaze is monitored by the posterior superior temporal sulcus (Nummenmaa and Calder, 2009; Senju and Johnson, 2009). There is also a top-down modulation of the aforementioned anatomical structures from the dorsolateral prefrontal cortex (Senju and Johnson, 2009, p. 130).

Recent research indicates that the amygdala stands central in the analysis of gaze direction (Kawashima et al., 1999; Hoffman et al., 2007; Straube et al., 2010; Boll et al., 2011; Sauer et al., 2014). The function of the amygdala in gaze processing is far more complex than previously suggested: much research has explored which gaze direction leads to the greatest activation of the amygdala. While Kawashima et al. (1999) and George and Conty (2008) found an increased amygdala activation as response to direct gaze, recent studies report an increased amygdala activation to averted vs. direct gaze (Straube et al., 2010) in spite of variation in emotional valence of the faces; moreover, it has been demonstrated that different sub-regions of the amygdala (Boll et al., 2011) are activated differently as response to gaze direction, head orientation, and the valence of facial expression. Sauer et al. (2014) reports that averted vs. direct gaze activates the right dorsal amygdala independent of facial expression and head orientation, while valence effects activated the ventral amygdala, strongly dependent of the direction of head. The left ventral amygdala was strongly activated as response to angry and neutral vs. happy faces when the faces were directed toward the beholder. In the averted head condition, there was an increased activity to happy vs. angry and neutral faces in the left ventral amygdala.

We present three experiments on the effects of face orientation and gaze direction on the judgments of social attributes. As the methods are very similar across experiments, we present them for the sake of simplicity for all experiments together. The same is done for the results for the analysis of association and the inference statistics for all experiments.

Materials and Methods

In order to test how subjects react to different face and gaze directions, we conducted three experiments in which direction of face and gaze were manipulated: (1) Dutch photographs deriving from the Dutch Radboud Faces Database (RaFD: Langner et al., 2010); (2) Brazilian photographs from the Brazilian FEI face database; and (3) two separate independently repeated experiments using the Holy Face and secular portraits.

A few words about the two different face databases are required. The RaFD² contains colorful pictures of eight emotional expressions: anger, disgust, fear, happiness, sadness, surprise, contempt and neutral. It consists of “face stimulus sets in which special facial characteristics are systematically varied while other important picture characteristics are kept constant; all frontal images were rated according to the shown facial expression, intensity of expression, clarity of expression, genuineness of expression, attractiveness and valence of expression” (Langner et al., 2010). Hence, this database not only presents neutral expressions, as those selected for in our study, but also a huge spectrum of emotional valence.

The FEI Face Database³ has 14 images for each of 200 individuals, a total of 2800 images. These are all in color and taken against a white homogenous background in an upright frontal position with profile rotation of up to about 180° (see Thomaz and Giraldi, 2010, for further information).

In our Experiments 1 and 2, we made a selection of faces with no concern for their emotional ratings (see RaFD database as described by Langner et al., 2010), neither have we transformed the FEI faces in any way (see Thomaz and Giraldi, 2010). In Experiment 1 and 2 the gaze direction in profile direct was manipulated in Photoshop⁴. Frontal indirect/averted faces in Experiment 1 were derived from non-manipulated photos from the RaFD database. Since, in the FEI database, the frontals with averted gaze were not satisfactorily averted for our purpose, they were manipulated in Photoshop.

In the study of the Holy Face, all faces were derived from non-manipulated original paintings. One obvious and essential difference between modern photographs and the painted portraits is that the portraits were painted with intent to present not only a likeness to the physical model, but also to convey a positive response towards the subject of the image. We have chosen to keep the artwork intact because we do not want to interfere with the artist’s design (see Bullot and Reber, 2013a). Each study is evaluated formally by statistical hypothesis testing of our planned contrasts. We explore the structure in the data using a graphical analysis of association. But first we present the material used in the studies.

The advantage of using modern portraits lies in experimental control. In photographs of both frontal and half-profile views, gaze directions can be manipulated. We realized a full 2 × 2-factorial design with two independent factors (face and gaze), resulting in four conditions: frontal view, direct gaze, frontal view, averted gaze, half-profile, direct gaze, half-profile, averted gaze (see Table 1).

TABLE 1

Table 1. These sketches illustrate the experimental manipulations in each study (1, 2 and 3).

In order to gather information about possible responses to holy vs. profane faces in the 15th century, we decided to examine responses to reproductions of real portraits (see Bullot and Reber, 2013a; Silvia, 2013; but see Graham, 2013; and Bullot and Reber, 2013b; for further discussion). In addition to the first two studies, using material from face databases, a third experiment therefore examined the same responses with reproductions of paintings from the 15th century (Figure 1). Half of the portraits depicted the face of Christ while half of the paintings represented profane faces. As paintings of Christ in frontal view with averted gaze were not available to us and we decided to use originals only, the design remained incomplete, and face orientation is confounded with holiness.

FIGURE 1

Figure 1. From left: Holy Face frontal view direct gaze, secular half-profile averted gaze, secular half-profile direct gaze.

Ten adjectives were selected as relevant for inferred personality traits in the collection of Renaissance portraits that is the subject of the investigation on the Holy Face. These adjectives all denote mental traits that are hard to observe directly and objectively. At the same time, these adjectives are common, familiar and often applied as descriptions of the perceived persona that we hypothesize can be inferred from how a person looks at an observer. The adjectives can be divided into two different groups. The positive group consists of “harmonious”, “trustworthy”, “caring, ”inclusive" and “respectable” and the negative group consists of “authoritarian”, “monitoring”, “evasive”, “intimidating” and “dominant”. Some of the adjectives have been used in other related research, see for instance (Todorov et al., 2008b; Figure 2), where the trustworthiness dimension is plotted along the x-axis and dominance along the y-axis. All of our adjectives are treated statistically as random factors, i.e., there could be other sets of adjectives that can be used to describe such inferred personality traits. We do not assume that these are the only relevant adjectives for the purposes of our study, but we assert that they are a relevant selection. In an item analysis (see later hypothesis testing) it is possible to estimate whether the items are applied consistently. If the item analysis turns out non-significant, it is indicated that there is a high level of variance, or insecurity, in which adjectives are applied to the experimental material. A significant item analysis confirms that the items are used consistently in line with the results. Four experiments were conducted using ratings on a scale from 0 to 10 on how well an adjective fits an image.

FIGURE 2

Figure 2. Dutch Radboud Faces Database (RaFD). Abbreviations: FD, Frontal Direct; FA, F. Averted; PD, Profile Direct; PA, P. Averted; harm, harmonious; trus, trustworthy; cari, caring; incl, inclusive; resp, respectable; auth, authoritarian; moni, monitoring; evas, evasive; inti, intimidating; domi, dominant. Note: Each Pearson residual is represented by its area; therefore, no y-axis is depicted.

Methods Specific to Experiments

Experiment 1

In Experiment 1 we used photographs from the Dutch RaFD. Fifty-two respondents (36 females and 16 males) volunteered and completed the study. For each participant who completed the study, 20 Norwegian Crowns (about $3) were given to a charity organization for children in need. Each participant received information on the principle of voluntary participation, including the right to withdraw from the study at any time without having to justify their reasons for withdrawal, and all were informed that all data is kept anonymous.

In order to secure the anonymity of older participants who were few in number, we asked participants for age ranges (18–22, 23–27, etc., up to 57 and older). Most participants were 27 years old or younger (n = 36), the other participants were evenly distributed across the age ranges older than 27 years. Experiment 1 collected 2080 observations from 52 subjects. Face orientation in Experiment 1 was either frontal or half-profile (45°).

Experiment 2

In Experiment 2 we used a photograph collection of Brazilian faces deriving from the FEI face database. This experiment was essentially identical to Experiment 1, although we used pictures from a different database. Images in Experiment 1 and 2 thus come from two different populations, with slight unplanned differences in face shape and skin tone. Images from the Dutch database are arguably closer to a prototypical Norwegian face. We cannot exclude that this gives rise to familiarity effects. Face orientation in Experiment 2 was either frontal or half-profile, slightly more towards frontal than in Experiment 1.

Forty-nine respondents (39 females and 10 males) completed the questionnaire. Most participants were 27 years old or younger (n = 39), the other participants were evenly distributed across the age ranges older than 27 years, similar to Experiment 1. The material was thus similar to study one, except that the models were collected from the Brazilian database. The procedure was identical to Experiment 1. Experiment 2 collected 1960 observations from 49 subjects.

Experiments 3 and 4

In Experiment 3 we used a sample of Renaissance paintings, where the faces represent a profile direct gaze, a profile averted gaze and a holy face with a full face frontal direct gaze (Figure 1). Experiment 4 is an independent replication of Experiment 3. Experiment 3 consists of 1980 observations of how 66 subjects applied the ten adjectives. Experiment 4 consists of 2430 observations of how 81 subjects applied the ten adjectives. The procedure was identical to the first two experiments.

In all experiments the participants gave their active, informed consent by providing their e-mail address, whereupon they received a hyperlink to an online questionnaire hosted by SurveyXact.

Overview on Statistical Analysis Strategy

Analysis of Association Between Image Types and Perceived Personality Traits

Our experiments resulted in measurements for 10 adjectives that denote personality traits. The measurements are divided up by the type of Image that the traits were assigned to by the subjects in the experiments. For the experiments on the photographs from the Brazilian (FEI Database) and Dutch (RaFD) databases, the images were classified into four different categories based on Frontal or Profile view with a Direct or Averted gaze. The categories are abbreviated FA, FD, PA and PD. In the investigations on the Holy Face, there are only three categories: Holy, Direct and Averted (see Table 1 and Figure 1 above). Thus we have many combinations of Image types and Traits to consider, and we will use the mean raw score of assignments in each combination to illustrate how images are associated with traits. One intuitive technique is the Cohen-Friendly Association Plots (Cohen, 1980; Friendly, 1992) as implemented in the program package R. Each plot indicates the deviations from statistical independence of rows and columns in a matrix. Each image category has an indicated line that marks statistical independence, and deviance is marked by boxes that could either be higher (shaded in blue above the line) or lower (shaded in red below the line) than expected from statistical independence. The association graph makes it easy to spot which adjectives are positively or negatively associated with each image type. We decided to use extended association plots with color coded Pearson Residuals (Meyer et al., 2003).

It must be stressed that the association plots are not used as a formal hypothesis test, but rather to illustrate the structure in the data set, and to help us understand the results of the inference statistic. We also want to confirm the quality of the experiment by investigating how the different adjectives contribute to relevant observed differences in the experiments. Additionally, we want to confirm that positive and negative adjectives are assigned differently to the image categories, and thus confirm the validity of the experimental model.

Inference Statistics

Assuming that we have found adjectives that correctly associate with positive and negative value assignment, we can make this assignment explicit by multiplying the ratings for the negative adjectives with a constant −1. The assumption is confirmed by analysis of association. If positive and negative adjectives are assigned at random (i.e., unsystematically), we expect the values to sum near zero, i.e., a neutral evaluation on average. However, we can also fail to detect differences between the experimental factors when there is another constant bias (i.e., if all or most images receive a score that deviates from zero by a constant amount, either positively or negatively) resulting in no differences in our experimental conditions (face and gaze direction).

Deviance from any constant assignment can be detected by statistical methods. We have chosen a mixed effects model with random effects for subjects and adjectives. The large number of observations motivates the use of this fairly robust model, as the responses for each adjective are close to normal distribution.

We analyzed all experiments using a mixed effects model implemented in the LmerTest package (see Schaalje et al., 2002; Kuznetsova et al., 2015) in the R statistics software (R Core Team, 2015). The LmerTest implements the Satterthwaite approximation of degrees of freedom, and uses this to evaluate and present the statistical model. This makes it possible and feasible to test a planned model that also includes interaction effects for each experiment, given that there are enough data points to successfully estimate the needed parameters. Previously it was common to perform model comparisons to determine the effect of each experimental factor. The model comparison procedure was awkward and it typically did not provide the degrees of freedom of the final test (see Baayen, 2008, section 7.2). This in turn made it cumbersome to look further into the subject (F1) and item (F2) analyses in random effects models (see Raaijmakers et al., 1999, for a discussion on F1, F2 and alternatives). In this article, the F1 and F2 provide useful insights into the robustness of our findings.

We used random factors for subject and adjectives, thus assuming that there could be other subjects and other adjectives than those we have included. The sufficiency of the chosen adjectives is indicated by how consistently subjects associate the adjectives to the image categories, both within experiments and between repetitions of the experiment. This requires a subject analysis and an item analysis, where we may have a more specific slope (i.e., rate of change between conditions) for either subjects or adjectives. Adding specific slopes adds details to the model and reduces the degrees of freedom that are available for random variance. If the model is too complex for the data, we test the model without a specific slope. When a model uses the slope parameter (see formulas below) it indicates that there are not only different starting points (intercepts) for each subject and item, but also that the model considers reactions to test images that are specific to the subject or the adjective.

For Experiment 3 and 4 we built models where we explain scores by the type of image (holy, direct, or averted) as a fixed factor. The model labeled F1 corresponds to a more detailed subject analysis, and the model labeled F2 corresponds to a more detailed item analysis. The statistics program uses an estimation procedure that may fail if the model is too complex for the available data. The procedure is therefore to start with the full model, and if the estimation procedure fails then a simpler model will be tried that does not estimate specific random effects called slopes, such as type per subject in the formulas below. All models converged.

F1 = score ~ type + (type | subject) + (1 | adjective)

F2 = score ~ type + (1 | subject) + (type | adjective)

For Experiment 1 and 2 we built similar models, but we include gaze and face direction as two fixed factors. We include slopes for all combinations of gaze and face direction for either the subject or the adjective. All these models also converged.

F1 = score ~ direction * gaze + ((direction * gaze) | subject) + (1 | adjective)

F2 = score ~ direction * gaze + (1 | subject) + ((direction * gaze) | adjective)

Results

Analysis of Association Between Image Types and Perceived Personality Traits

First, we investigate the structure of the data sets, using extended association plots with color coded Pearson Residuals (Meyer et al., 2003). Each box in the graphs encodes the size of the Pearson Residual for that combination of adjective and type of stimulus. The height of the box indicates deviance of the sum of points, for that combination, from the expected sum if adjectives and types are statistically independent (i.e., the familiar Observed—Expected from a standard chi-square test). The base of each box indicates the square root of the expected sum for that cell. The similar base in all the graphs, thus shows that the expected sums are overall similar, i.e., we have approximately equal evidence from all the adjectives and types of stimuli. The Pearson Residual of a cell is simply the square root of the contribution to significance for that cell in a standard cross tabulation chi-square test. The contributions of the individual Pearson Residuals in the presented graphs are not significant by themselves, which supports that the effects we will see later are not driven entirely by any one adjective.

In the following, the graphs will have four-letter abbreviations for each adjective. The first five adjectives are positive adjectives (harmonious, trustworthy, caring, inclusive, respectable). The last five adjectives are negative adjectives (authoritarian, monitoring, evasive, intimidating, dominant).

Experiment 1

Dutch Photographs (see Figure 2). We can see that Front Direct and Profile Direct differ in scores for positive and negative adjectives. Likewise, Profile Direct and Profile Averted are close mirror images of each other. This shows that both face direction and gaze direction plays a role in the assignment of personality traits to the photographs. However, we can also note that the largest effect stems from the adjective evasive, which is the adjective that may be the easiest to associate with an averted gaze. It can be noted that there are somewhat larger effects for negative adjectives, although mostly driven by evasive⁵.

Experiment 2

Brazilian photographs from the FEI face database (see Figure 3).

FIGURE 3

Figure 3. Brazilian photographs from the FEI face database. Abbreviations: FD, Frontal Direct; FA, F. Averted; PD, Profile Direct; PA, P. Averted; harm, harmonious; trus, trustworthy; cari, caring; incl, inclusive; resp, respectable; auth, authoritarian; moni, monitoring; evas, evasive; inti, intimidating; domi, dominant. Note: Each Pearson residual is represented by its area; therefore, no y-axis is depicted.

The results are very similar to Experiment 1, in that the effect is driven mainly by the adjective evasive. There are similarly larger effects for the negative adjectives. We see a tendency for FD/FA and PD/PA to be mirror images of each other, again showing that there is a consistent assignment of personality traits to these photographs as well.

Experiment 3

Holy Faces and secular portraits (1). Figure 4.

FIGURE 4

Figure 4. Holy Faces and secular portraits (1). Abbreviations: FD, Frontal Direct; FA, F. Averted; PD, Profile Direct; PA, P. Averted; harm, harmonious; trus, trustworthy; cari, caring; incl, inclusive; resp, respectable; auth, authoritarian; moni, monitoring; evas, evasive; inti, intimidating; domi, dominant. Note: Each Pearson residual is represented by its area; therefore, no y-axis is depicted.

In the Experiment on the Holy Face we see a clear tendency for the Holy Face to be associated positively with positive adjectives and a negative association with negative adjectives. The Profile with averted/indirect gaze shows a mirror image of the assignment of traits, compared to the Holy Face. The Profile with direct gaze shows an intermediate profile, with lower effects in general. Importantly, the effects are not driven by any single adjective.

Experiment 4

Holy Faces and secular portraits (2) (see Figure 5).

FIGURE 5

Figure 5. Holy Faces and secular portraits (2). Abbreviations: FD, Frontal Direct; FA, F. Averted, PD; Profile Direct, PA, P. Averted; harm, harmonious; trus, trustworthy; cari, caring; incl, inclusive; resp, respectable; auth, authoritarian; moni, monitoring; evas, evasive; inti, intimidating; domi, dominant. Note: Each Pearson residual is represented by its area; therefore, no y-axis is depicted.

This Experiment replicates all the findings in Experiment 3 for associations between adjectives and personality traits in the collection of portraits. This importantly shows that the results are robust and likely to replicate well.

Analysis of Association Between Image and Personality Traits

Experiment 1 and 2 using photographs, show that subjects are indeed very consistent in how they assign personality traits to images. However, it seems that differences between the experimental conditions mainly stem from the application of the adjective evasive, which is also the adjective that may more easily be associated with the lack of direct eye contact through plain observation.

Experiment 3 and 4 show something that is a little more interesting. Again the subjects are very consistent in how they assign the adjectives, but this time there is a noticeable difference between the experimental conditions when it comes to assigning positive or negative traits to the portraits. This effect is not driven by any particular adjective, but applies across all adjectives.

One main difference between photographs and the portraits is that the portraits are constructed by an artist, with the intention to present a persona. Reflecting on the differences between the photographs and the portraits it should be noted that they differ in many dimensions. The portraits include mainly men at the peak of their careers, whereas the photographs are generally of younger individuals, and include both men and women, as well as ethnic differences noticeable from for example skin tone and face shape.

There is obviously a need to follow up these differences in controlled experiments, rather than speculating post hoc on how other factors contribute to our perception and assignment of personality traits. It is clear that our subjects are very consistent in their judgments, but it is also clear that there are other factors to consider, such as age and social class of the person depicted in the pictures.

We are now ready to move on to hypothesis testing using inference statistics. We have shown that assignment of personality traits to images are consistent and related to our experimental conditions, but are these findings significant, or can they be explained by the variance in our samples?

Inferential Statistics

We start with the analysis of the photographs for Brazilian and Dutch faces. The regression model shows the intercept, and the effects of profile, indirect gaze and the interaction effect of both profile and indirect gaze. The intercept is positive for both experiments, but this is not significant when tested. The main effect of face direction is negative (Brazilian: −0.15, Dutch: −0.74) for profile, which is significant for the Dutch set only (F1: **p < 0.002, F2: *p < 0.03). The main effect of gaze direction is negative for indirect gaze (Brazilian: −0.05, Dutch −0.35), which is significant for the Brazilian and Dutch subject analysis (*p < 0.05 and **p < 0.01, respectively), but not in the item analyses. The interaction effect shows a more positive effect for profile with an indirect gaze for both Brazilian and Dutch sets (accounts for 0.36 and 1.05 points respectively), which is significant for subject analysis only (*p < 0.02 and ***p < 0.001, respectively). The failures to reach significance for the item analyses show that there is considerable variance for how the adjectives affect ratings for the photographs, and this was previously (see Figures 2, 3) identified as an effect driven by a few adjectives, evasive in particular.

The regression formulas for the Brazilian (a) and Dutch (b) study are presented below. For estimates of the values for each combination, see the interaction graph (Figure 6).

(a) points = 0.31 −0.15(if profile) −0.05(if indirect gaze) + 0.36(if profile and indirect gaze)

(b) points = 0.64 −0.74(if profile) −0.35(if indirect gaze) + 1.05(if profile and indirect gaze)

FIGURE 6

Figure 6. Interaction between face and gaze direction for the Brazilian and Dutch faces (Experiment 1 and 2).

Figure 6 shows the net result of all combinations of face and gaze direction. Experiment 1 and 2 look similar. In both datasets, the profile that looks directly at the viewer has the lowest score. In both datasets the scores are slightly biased towards positive scores. Frontal direct gaze has a more positive score, which is more noticeable in the experiment on Dutch faces. Both experiments show interaction between face and gaze direction, but there is considerable insecurity in the assignment of scores, and with the exception of Dutch face direction, the item analysis is not significant. This indicates that although subjects assign positive and negative adjectives consistently there is little agreement for when a specific adjective is used, and some adjectives such as “evasive” contributes more to the effects.

The Studies on the Holy Face (Experiments 3 and 4)

The following table gives the results of the statistical analysis of Experiment 3 and 4, relating to painted portraits. Holy1 and Holy2 are independent replications of the same experiment. As can be seen from Table 3, and the point calculations below, there are significant differences in the application of positive and negative values between the three variants of portraits. The portraits with a direct gaze are neutral (at a non-significant intercept of −0.22 and +0.007, respectively). The Holy Face is more positive (0.45 (i.e., +0.67 −0.22), and 0.44 (i.e., +0.37 +0.07) points respectively). A face with an averted gaze is more negative (−0.88 (i.e., −0.66 −0.22), and −0.77 (i.e., −0.78 +0.007) respectively). The expected distance between the Holy Face and the averted profile is about 1.2 points (1.33 and 1.15, respectively). The ANOVA model tests that the effects for type of portrait are persistent across both subject and item analyses. Significance tests reveal 2–3 stars for all relevant comparisons. Holy2 almost perfectly replicates Holy1. This means that the observed differences in adjective assignment are very likely to be true differences. The Direct images are equal to the intercept, as neither Holy nor Averted is added in the formula.

The regression formulas for the Holy1 (a) and the Holy2 (b) study are presented below.

(a) points = −0.22 +0.67(if Holy) −0.66(if Averted)

(b) points = 0.007 +0.37(if Holy) −0.78(if Averted)

Looking more closely at the relationship between adjectives and type of portrait indicates that Experiment 3 and Experiment 4 are indeed good replications of the same experiment. Figures 7A,B show this for the range of the original data for all adjectives in both experiments. From the analysis of association, we confirmed that the effects are not driven by individual test items, and this shows up here in highly significant item analyses for both experiments.

FIGURE 7

Figure 7. (A) How well the adjectives fit with the type of picture. Experiment 3. (B) How well the adjectives fit with the type of picture. Experiment 4.

General Discussion

The basic motivation of this study was to examine the potential reasons for why Medieval and Renaissance painters depicted Christ with a face shown from a frontal view and looking directly at their viewers but showed portraits of profane sitters from the half-profile. The results with modern pictures of faces showed that people gave frontal faces with direct gaze high scores on attributes deemed relevant to divinity (Dutch faces more so than the Brazilian ones), but profile views with averted/indirect gaze were also more positive, as far as the modern faces are being concerned (see Table 2). The findings were even more pronounced for historical portraits of Christ, where the Analysis of Association shows a consistent overrepresentation of positive adjectives for the Holy Faces.

TABLE 2

Table 2. Results of statistical analysis of experiment 1 and 2.

The Holy Faces and the Dutch modern frontal faces gazing at the viewer were regarded to be more caring, trustworthy, harmonic, inclusive and respectable. Experiment 1 (Dutch) however, showed similar effects for profiles with averted/indirect gaze (see Figure 2). In Experiment 2 (Brazilian) the most positive combinations were profiles with averted gaze (see Figures 3, 6), and in both Experiment 1 and 2 profiles with direct gaze got a lower score (see Figure 6). Hence, the profiles of modern faces gazing at us were scored more negatively, in contrast to the results on the artworks, where direct gaze was scored as more positive than averted, for the half-profiles. The “eye contact effect” (Senju and Johnson, 2009), in the photographs and in the artworks, is discussed later. Although the effects are more uncertain for Experiment 2 (Brazilian), our findings raise the question whether the frontal Holy Face is more than a convention, that is, whether there are deeper biological or evolutionary reasons for such a convention.

Despite the support of our hypothesis that Medieval and Renaissance artists tried to convey saintly attributes by depicting faces from the frontal view and with direct gaze, our study points to the limits of experimental art history. First, it is impossible in principle to know whether modern viewers provide the same scores as a medieval audience would have done. While the neuroscientific theories of art (e.g., Ramachandran and Hirstein, 1999; Zeki, 1999) claim that the brain has been the same the last millennia and judgments are therefore comparable, proponents of a historical theory of art appreciation emphasize the cultural and experiential differences between medieval and modern people, which renders comparison difficult (see Bullot and Reber, 2013a). The more closely the scientific hypothesis is related to the biological perceptual system, the more we can generalize results across time. For example, we might assume that the biological basis of color vision or perception of movement did not change across time. However, most hypotheses, and presumably the most interesting ones, must take art-historical knowledge (see Bullot and Reber, 2013a) into account, rendering such generalization more difficult.

Second, both modern faces and historical materials have their advantages and disadvantages. As there are face databases with portraits from different angles, and as gaze direction is relatively easy to change by technical means, experimental control of the materials is possible. Moreover, with such materials, we can remove much of the influence of cultural knowledge on the judgments. This strength turns into a weakness, however, if cultural knowledge moderates the effects of facial features. For example, knowing how Christ is depicted might provide different or—as in our study—stronger interpretations of facial orientation and gaze direction. Modern materials may be inadequate substitutes for the historical paintings. However, the latter are problematic, too. As there are few depictions of the Holy Face from half-profile and almost no depictions of profane frontal faces, experimental manipulation of materials is difficult, especially as the manipulation of face orientation would demand building a 3D model from one image in profile. Some of the effects observed in the last study could stem from material effects; for example, that profane people are perceived to be more dominant and intimidating may derive from the perceived age of the depicted person, an assumption that awaits further research.

Moreover, we must take into consideration that there are some physiognomic traits that are perceived to be more trustworthy or dominant (Oosterhof and Todorov, 2008; Todorov, 2008; Todorov et al., 2008a,b; Geniole et al., 2014) and intelligent (Kleisner et al., 2014) than others (for a recent review and discussion of this field of research: Todorov et al., 2015). In our study, we had no possibility to control for such parameters. Neither did we manipulate the Renaissance portraits into neutral expressions; hence, where the photos from the databases were displaying the same neutral mouths and eyes, the art portraits vary to a certain degree from neutral expressions.

We consider evaluating the materials on a perceived gender scale. As has been demonstrated by Todorov et al. (2015, p. 522) and Figure 2, perceptions of dominance are strongly related to masculine appearance. The Holy Face is sometimes feminine in its beauty. This is particularly interesting when it comes to the judgment of aggression, which strongly correlates with the facial width-to-height ratio in male faces; increasing the ratio leads to stronger judgments of aggression (Geniole et al., 2014).

When it comes to the question of symmetry vs. non-symmetry in faces, only the Holy Faces are strictly symmetrical. It has been demonstrated (Ewing et al., 2010) that frontal faces with direct gaze are rated to be more attractive than those with averted gaze. Turning the face upside down did not, however, confirm the results, which indicate that the symmetry of faces does not explain the full effect. On the other hand, the so called Thatcher Illusion demonstrates that perception of facial features is generally suppressed if a face is inverted (Thompson, 1980).

Problematic for the interpretation of our results that a representation of Christ as God, for bio/evolutional reasons, had to be represented symmetric is the fact that the Byzantine Christ the Pantocrator (All Ruler), in many cases, is not symmetric, combining profile and full-face, particularly conspicuous in the Deesis mosaic (ca. 1300) on the Gallery of Hagia Sofia, Istanbul⁶. Nevertheless, this “manipulation” with the physiognomy is so subtle that one may have to scrutinize the face to find out how asymmetric it really is. On the other hand, the Byzantine so called Mandylion (Nicolitti, 2014 inter al.), said to be an imprint of Christ’s face, as in the veil of Veronica, is as precisely symmetric as the Holy Faces of Western Renaissance. Hence, in all these cases, Christ is represented full-face, in the Western Renaissance with a direct gaze, in the Mandylion with a direct or weakly averted gaze, in the Byzantine Pantocrator more or less with a direct gaze, but also sometimes averted (a famous example of averted gaze being the Pantocrator in the cupola of the monastic church of Dafni, Greece (early 12th century).

Experimental art history, like experimental archeology, can only test the plausibility of assumptions. Even if we could assume that judgments from modern respondents correspond to those by historical respondents, in that the attributes of modern materials map the attributes of historical materials and that historical materials yield reliable and valid responses, we still cannot prove that painters indeed intended to achieve the effects that were obtained with modern viewers. In general, experiments to test hypotheses about the intended or real effects of historical art on the audience of that time are of some merit and might provide us with insights about the plausibility of art-historical assumptions. However, the interpretation of such studies always has to take change in cultural knowledge of the audience into account.

In this study, we have seen that frontal faces with direct gaze (Experiment 1, 3 and 4) were more highly associated with positive adjectives. The exceptions from Experiment 2, and the increased insecurity in that experiment, might be explained by the fact that the photographs included more factors such as gender and ethnicity that was not an issue in the painted portraits, as argued earlier in the results section on the Analysis of Association. Does this also help to associate positive values to the Holy Face in a Western context? This raises the question about the depiction of Christ as a white Western man. It looks like the explanation for our readiness to apply positive values to the Holy Face might be a combination of many factors. Familiarity may be such a factor as familiarity will signify a more typical face, and it is found to be rated as more trustworthy than an extremely beautiful face (Sofer et al., 2015).

Face and gaze direction might reinforce the general impression. There could be other factors that influence our perception of beauty that work in the same direction, such as symmetry and possibly androgyny. That means that several factors may conspire together towards a positive perception of the image. In order to explore these factors in more detail and how they influence the cognitive processing of faces, neuroimaging studies may be applied that address the differential processing of face and gaze direction on a neuronal level. This would also open the possibility for exploring the underlying mechanism from a more implicit perspective, i.e., without directing the subject’s attention onto possible attributes of the perceived face. This would correspond to a more natural and ecologically more valid observation situation.

The Holy Faces were perceived as particularly caring, inclusive and harmonious, but also trustworthy and respectable. This strong bias towards the positive adjectives will partly seem to find its explanation in cultural factors: that the face of Christ will be associated with such positive adjectives. Still, there is another feature with the Holy Faces that may override the cultural one: many of the Holy Faces appear feminine when the beard is occluded, a feature that has to be followed up with new studies.

In our two studies on the Holy Face, the secular portraits with direct gaze were neutral (see Table 3) and those portraits with averted gaze were negative. This result may be explained through a combination of the eye contact effect (see Senju and Johnson, 2009), and the effect of the general impression of the faces: if the general impression of the face is that it is authoritarian, monitoring and intimidating, this will bias the perception of direct gaze in the negative direction. By contrast, if the face gives a positive impression, this will also strongly interact with the eye contact effect resulting in a bias toward the positive adjectives. The portraits that we use in our survey show different valences in face expression. Most of the faces with direct gaze are close to neutral, but there are also some that are scary and some with inclusive features.

TABLE 3

Table 3. Results of statistical analysis of Experiment 3 and 4.

To sum up, our study suggests that experimental art history can provide evidence for the plausibility of psychological assumptions that follow universal evolutionary or biological laws. However, cultural and historical factors prevent experimental art history from providing more direct evidence on the processes that guided the generation of art in the remote past.

Ethics Statement

All research reported in this manuscript was in accordance with the ethical guidelines of the American Psychological Association and the ethical guidelines for the social sciences, law, and the humanities by the Norwegian National Research Ethics Committees.The study was exempt from the requirement to submit it to one of the Norwegian bodies for ethical approval because it was conducted with healthy adults who could provide informed consent for themselves, all data were collected anonymously, and thestudy did not involve invasive methods or other potential harm, deception, potential concerns about privacy, etc.

Author Contributions

POF: outlining theory, designing experiments, conducting study, writing manuscript. CJ: outlining theory, data analysis, data visualization, writing manuscript. LH and RR: outlining theory, designing experiments, writing manuscript. ØOS: data analysis, writing manuscript. AEA and LCS: preparing materials. KS: writing manuscript.

Funding

The Meltzer Foundation, LLE at the University of Bergen and NTNU supported the project “Symmetry in Art and Science An interdisciplinary research project”.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Maria Stiegler Folgerø and Mogens Hestholm for preparing and running the experiments, Per Sigurd Styve for collecting source material, Pauline Hoath, Samantha Smith and Jill Walker Rettberg, at University of Bergen, for reading through parts of the text and suggesting improvements. We also thank two reviewers for useful and constructive suggestions that have improved our manuscript.

Footnotes

1. ^ The regions here listed serve different functions: the superior temporal sulcus is considered to be a hub region in social perception and cognition, including language, head and gaze stimuli. It consists of a number of sub regions responding selectively to particular stimuli, organized along a posterior-to-anterior axis (see particularly Deen et al., 2015, for new evidences). The fusiform gyrus responds to face and body stimuli; it has recently been suggested that face and body patterns each add unique information to the response patterns to whole persons (see Kaiser et al., 2014). The medial prefrontal cortex is strongly involved in decision making and in memory processes (Euston et al., 2012). It is also involved in judgments of self and others (Denny et al., 2012), moral judgments, and even aesthetic judgment tasks (Jacobsen et al., 2006; Forbes and Grafman, 2010; Zaidel and Nadal, 2011; Avram et al., 2013). The orbitofrontal cortex receives exteroceptive and interoceptive stimuli; it is a significant part of our reward network, is strongly connected to the emotional network, i.e., the limbic system, and can be considered as a higher order cortex for smell and gustation. It is activated as a response to what a subject finds beautiful either in art or in music (Kawabata and Zeki, 2004; Ishizu and Zeki, 2011). All these areas are involved in many tasks, and may perform sub-functions in an intricate network.
2. ^ http://preview.tinyurl.com/gp5bpbo
3. ^ http://preview.tinyurl.com/h6nbsma
4. ^ The eyes in the databases were not corresponding to the even more averted eyes in the historical material.
5. ^ It is therefore strange that profile direct (PD) was more evasive than PA. This may be explained by a more general uncertainty for how subjects apply adjectives in Experiment 1 and 2, which is exemplified by the fact that typically the item analyses (see Table 2) do not reach significance.
6. ^ The asymmetry of Christ’s physiognomic traits in the mosaic in Hagia Sophia corresponds to the prescriptions in the Greek Painters Manual, the so called Long Paragraph (LP), a canon presented under the name of the Macedonian painter Manuel Panselinos (Torp, 1984, p. 39–40).

References

Allison, T., Puce, A., and McCarthy, G. (2002). Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278. doi: 10.1016/s1364-6613(00)01501-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Avram, M., Gutyrchik, E., Bao, Y., Pöppel, E., Reiser, M., and Blautzik, J. (2013). Neurofunctional correlates of esthetic and moral judgments. Neurosci. Lett. 534, 128–132. doi: 10.1016/j.neulet.2012.11.053

PubMed Abstract | CrossRef Full Text | Google Scholar

Baayen, R. H. (2008). Analyzing Linguistic Data: A Practical Introduction to Statistics Using R. Cambridge, UK: Cambridge University Press.

Google Scholar

Baron-Cohen, S. (1997). Mindblindness: An Essay on Autism and Theory of Mind. Cambridge, MA: MIT press.

Google Scholar

Boll, S., Gamer, M., Kalish, R., and Büchel, C. (2011). Processing of facial expressions and their significance for the observer in subregions of the human amygdala. Neuroimage 56, 299–306. doi: 10.1016/j.neuroimage.2011.02.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Bullot, N. J., and Reber, R. (2013a). The artful mind meets art history: toward a psycho-historical framework for the science of art appreciation. Behav. Brain Sci. 36, 123–137. doi: 10.1017/s0140525x12000489

PubMed Abstract | CrossRef Full Text | Google Scholar

Bullot, N. J., and Reber, R. (2013b). A psycho-historical research program for the integrative science of art. Behav. Brain Sci. 36, 163–180. doi: 10.1017/s0140525x12002464

PubMed Abstract | CrossRef Full Text | Google Scholar

Burra, N., Hervais-Adelman, A., Kerzel, D., Tamietto, M., de Gelder, B., and Pegna, A. J. (2013). Amygdala activation for eye contact despite complete cortical blindness. J. Neurosci. 33, 10483–10489. doi: 10.1523/JNEUROSCI.3994-12.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Calder, A. J., Beaver, J. D., Winston, J. S., Dolan, R. J., Jenkins, R., Eger, E., et al. (2007). Separate coding for different gaze directions in the superior temporal sulcus and inferior parietal lobule. Curr. Biol. 17, 20–25. doi: 10.1016/j.cub.2006.10.052

PubMed Abstract | CrossRef Full Text | Google Scholar

Calder, A. J., Lawrence, A. D., Keane, J., Scott, S. K., Owen, A. M., Christoffels, I., et al. (2002). Reading the mind from eye gaze. Neuropsychologia 40, 1129–1138. doi: 10.1016/s0028-3932(02)00008-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Carlin, J. D., Calder, A. J., Kriegeskorte, N., Nili, H., and Rowe, J. B. (2011). A head view-invariant representation of gaze direction in anterior superior temporal sulcus. Curr. Biol. 21, 1817–1821. doi: 10.1016/j.cub.2011.09.025

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, A. (1980). On the graphical display of the significant components in a two-way contingency table. Commun. Stat. Theory Methods A9, 1025–1041. doi: 10.1080/03610928008827940

CrossRef Full Text | Google Scholar

Coles, J. M. (2010). Experimental Archaeology. London: Blackburn Press.

Google Scholar

Conty, L., N’Diaye, K., Tijus, C., and George, N. (2007). When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing. Neuropsychologia 45, 3024–3037. doi: 10.1016/j.neuropsychologia.2007.05.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Deen, B., Koldewyn, K., Kanwisher, N., and Saxe, R. (2015). Functional organization of social perception and cognition in the superior temporal sulcus. Cereb. Cortex 25, 4596–4609. doi: 10.1093/cercor/bhv111

PubMed Abstract | CrossRef Full Text | Google Scholar

Denny, B. T., Kober, H., Wager, T. D., and Ochsner, K. N. (2012). A meta-analysis of functional neuroimaging studies of self- and other judgments reveals a spatial gradient for mentalizing in medial prefrontal cortex. J. Cogn. Neurosci. 24, 1742–1752. doi: 10.1162/jocn_a_00233

PubMed Abstract | CrossRef Full Text | Google Scholar

Dunham, P. J., Dunham, F., and Curwin, A. (1993). Joint-attentional states and lexical acquisition at 18 months. Dev. Psychol. 29, 927–931. doi: 10.1037/0012-1649.29.5.827

CrossRef Full Text | Google Scholar

Emery, N. J. (2000). The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci. Biobehav. Rev. 24, 581–604. doi: 10.1016/s0149-7634(00)00025-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Euston, D. R., Gruber, A. J., and McNaughton, B. L. (2012). The role of median prefrontal cortex in memory and decision making. Neuron 76, 1057–1070. doi: 10.1016/j.neuron.2012.12.002

CrossRef Full Text | Google Scholar

Ewbank, M. P., Jennings, C., and Calder, A. J. (2009). Why are you angry with me? Facial expressions of threat influence perception of gaze direction. J. Vis. 9, 16.1–16.7. doi: 10.1167/9.12.16

PubMed Abstract | CrossRef Full Text | Google Scholar

Ewing, L., Rhodes, G., and Pellicano, E. (2010). Have you got the look? Gaze direction affects judgments of facial attractiveness. Vis. Cogn. 18, 321–330. doi: 10.1080/13506280902965599

CrossRef Full Text | Google Scholar

Ferguson, J. R. (2010). Designing Experimental Research in Archaeology: Examining Technology Through Production and Use. Boulder, CO: University Press of Colorado.

Google Scholar

Forbes, C. E., and Grafman, J. (2010). The role of the prefrontal cortex in social cognition and moral judgment. Annu. Rev. Neurosci. 33, 299–324. doi: 10.1146/annurev-neuro-060909-153230

PubMed Abstract | CrossRef Full Text | Google Scholar

Friendly, M. (1992). “Graphical methods for categorical data,” in SAS User Group International Conference Proceedings (Vol. 17), 190–200. Available Online at: http://preview.tinyurl.com/h29ffha

Google Scholar

Geniole, S. N., Molnar, D. S., Carré, J. M., and McCormick, C. M. (2014). The facial width-to-height ratio shares stronger links with judgments of aggression than with judgments of trustworthiness. J. Exp. Psychol. Hum. Percept. Perform. 40, 1526–1541. doi: 10.1037/a0036732

PubMed Abstract | CrossRef Full Text | Google Scholar

George, N., and Conty, L. (2008). Facing the Gaze of Others/Le regard de l’autre. Clin. Neurophysiol. 38, 197–207. doi: 10.1016/j.neucli.2008.03.001

CrossRef Full Text | Google Scholar

George, N., Driver, J., and Dolan, R. J. (2000). Seen gaze-direction modulates fusiform activity and its coupling with other brain areas during face processing. Neuroimage 13, 1102–1112. doi: 10.1006/nimg.2001.0769

PubMed Abstract | CrossRef Full Text | Google Scholar

Graham, D. J. (2013). Integrating holism and reductionism in the science of art perception. Behav. Brain Sci. 36, 145–146. doi: 10.1017/S0140525x12001653

PubMed Abstract | CrossRef Full Text | Google Scholar

Haxby, J. V., Hoffman, E. A., and Gobbini, I. (2000). The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223–233. doi: 10.1016/s1364-6613(00)01482-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Hodne, L. (2013). Symmetry versus frontality in the renaissance holy face. Symmetry Cult. Sci. 24, 437–450.

Hoffman, K. L., Gothard, K. M., Schmid, M. C., and Logothetis, N. K. (2007). Facial-expression and gaze-selective responses in the monkey amygdala. Curr. Biol. 17, 766–772. doi: 10.1016/j.cub.2007.03.040

PubMed Abstract | CrossRef Full Text | Google Scholar

Ishizu, T., and Zeki, S. (2011). Toward a brain-based theory of beauty. PLoS One 6:e21852. doi: 10.1371/journal.pone.0021852

PubMed Abstract | CrossRef Full Text | Google Scholar

Itier, R. J., and Batty, M. (2009). Neural bases of eye and gaze processing: the core of social cognition. Neurosci. Biobehav. Rev. 33, 843–863. doi: 10.1016/j.neubiorev.2009.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacobsen, T., Schubotz, R. I., Höfel, L. V., and Cramon, D. Y. (2006). Brain correlates of aesthetic judgment of beauty. Neuroimage 29, 276–285. doi: 10.1016/j.neuroimage.2005.07.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, M. H., Senju, A., and Tomalski, P. (2015). The two-process theory of face processing: modifications based on two decades of data from infants and adults. Neurosci. Biobehav. Rev. 50, 169–179. doi: 10.1016/j.neubiorev.2014.10.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaiser, D., Strnad, L., Seidl, N., Kastner, S., and Peelen, M. V. (2014). Whole person-evoked fMRI activity patterns in human fusiform gyrus are accurately modeled by a linear combination of face- and body-evoked activity patterns. J. Neurophysiol. 111, 82–90. doi: 10.1152/jn.00371.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Kawabata, H., and Zeki, S. (2004). Neural correlates of beauty. J. Neurophysiol. 91, 1699–1705. doi: 10.1152/jn.00696.2003

PubMed Abstract | CrossRef Full Text | Google Scholar

Kawashima, R., Sugiura, M., Kato, T., Nakamura, A., Hatano, K., Ito, K., et al. (1999). The human amygdala plays an important role in gaze monitoring. A PET study. Brain 122, 779–783. doi: 10.1093/brain/122.4.779

PubMed Abstract | CrossRef Full Text | Google Scholar

Kleisner, K., Chvátalová, V., and Flegr, J. (2014). Perceived intelligence is associated with measured intelligence in men but not in women. PLoS One 9:e81237. doi: 10.1371/journal.pone.0081237

PubMed Abstract | CrossRef Full Text | Google Scholar

Kobayashi, H., and Koshima, S. (1997). Morphological uniqueness of human eyes and its adaptive meaning. Nature 387, 767–768. doi: 10.1038/42842

CrossRef Full Text

Kuznetsova, A., Bruun Brockhoff, P., and Haubo Bojesen Christensen, R. (2015). Package imertest. Available Online at: https://cran.r-project.org/web/packages/lmerTest/lmerTest.pdf

Laeng, B., Profeti, I., Sæther, L., Adolfsdottir, S., Lundervold, A. J., Vangberg, T., et al. (2010). Invisible expressions evoke core impressions. Emotion 10, 573–586. doi: 10.1037/a0018689

PubMed Abstract | CrossRef Full Text | Google Scholar

Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D. H. J., Hawk, S. T., and van Knippenberg, A. (2010). Presentation and validation of the radboun faces database. Cogn. Emot. 24, 1377–1388. doi: 10.1080/02699930903485076

CrossRef Full Text | Google Scholar

Langton, S. R. H. (2000). The mutual influence of gaze and head orientation in the analysis of social attention. Q. J. Exp. Psychol. A 53, 825–845. doi: 10.1080/027249800410562

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, D., Zeileis, A., and Hornik, K. (2003). “Visualizing independence using extended association plots,” in Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC2003) eds K. Hornik, F. Leisch and A. Zeileis (Vienna, Austria), ISSN 1609-395X. Available Online at: https://www.r-project.org/conferences/DSC-2003/Proceedings/

Google Scholar

Morgan, D. (2012). The Embodied Eye. Religious Visual Culture and the Social Life of Feeling. Berkeley, CA: University of California Press.

Mundi, P., and Gomes, A. (1998). Individual differences in joint attention skill development in the second year. Infant Behav. Dev. 21, 469–482. doi: 10.1016/s0163-6383(98)90020-0

CrossRef Full Text | Google Scholar

Nicolitti, A. (2014). From the Mandylion of Edessa to the Shroud of Turin. The Metamorphosis and Manipulation of a Legend. Leiden: Brill.

Google Scholar

Nummenmaa, L., and Calder, A. (2009). Neural mechanisms of social attention. Trends Cogn. Sci. 13, 135–143. doi: 10.1016/j.tics.2008.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Oosterhof, N. N., and Todorov, A. (2008). The functional basis of face evaluation. Proc. Natl. Acad. Sci. U S A 105, 11087–11092. doi: 10.1073/pnas.0805664105

PubMed Abstract | CrossRef Full Text | Google Scholar

Pageler, N. M., Menon, V., Merin, N. M., Eliez, S., Brown, W. E., and Reiss, A. L. (2003). Effect of head orientation on gaze processing in fusiform gyrus and superior temporal sulcus. Neuroimage 20, 318–329. doi: 10.1016/s1053-8119(03)00229-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrett, D. I., and Emery, N. J. (1994). Understanding the intentions of others from visual signals: neurophysiological evidence. Curr. Psychol. Cogn. 13, 683–694.

Google Scholar

Perrett, D. I., Hietanen, J. K., Oram, M. W., and Benson, P. J. (1992). Organization and functions of cells responsive to faces in the temporal cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 335, 23–30. doi: 10.1098/rstb.1992.0003

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrett, D. I., Smith, P. A. J., Mistlin, A. J., Chitty, A. J., Head, A. S., Potter, D. D., et al. (1985). Visual analysis of body movements by neurons in the temporal cortex of the macaque monkey: a preliminary report. Behav. Brain Res. 16, 153–170. doi: 10.1016/0166-4328(85)90089-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Raaijmakers, J. G. W., Schrijnemakers, J. M. C., and Gremmen, F. (1999). How to deal with “The Language-as-Fixed-Effect Fallacy”: common misconceptions and alternative solutions. J. Mem. Lang. 41, 416–426. doi: 10.1006/jmla.1999.2650

CrossRef Full Text | Google Scholar

Ramachandran, V. S., and Hirstein, W. (1999). The science of art. A neurological theory of aesthetic experience. J. Conscious. Stud. 6, 15–51.

Google Scholar

R Core Team. (2015). “R: a language and environment for statistical computing,” in R Foundation for Statistical Computing (Vienna, Austria), Available Online at: https://www.R-project.org/

Google Scholar

Sauer, A., Mothes-Lasch, M., Miltner, W. H. R., and Straube, T. (2014). Effects of gaze direction, head orientation and valence of facial expression on amygdala activity. Soc. Cogn. Affect. Neurosci. 9, 1246–1252. doi: 10.1093/scan/nst100

PubMed Abstract | CrossRef Full Text | Google Scholar

Schaafsma, S. M., Pfaff, D. W., Spunt, R. P., and Adolphs, R. (2015). Deconstructing and reconstructing theory of mind. Trends Cogn. Sci. 19, 65–72. doi: 10.1016/j.tics.2014.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Schaalje, G. B., McBride, J. B., and Fellingham, G. W. (2002). Adequacy of approximations to distributions of test statistics in complex mixed linear models. J. Agric. Biol. Environ. Stat. 7, 512–524. doi: 10.1198/108571102726

CrossRef Full Text | Google Scholar

Senju, A., and Johnson, H. M. (2009). The eye contact effect: mechanisms and development. Trends Cogn. Sci. 13, 127–134. doi: 10.1016/j.tics.2008.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Silvia, P. J. (2013). Aesthetic meanings and aesthetic emotions: how historical and intentional knowledge expand aesthetic experience. Behav. Brain Sci. 36, 157–158. doi: 10.1017/s0140525x12001781

PubMed Abstract | CrossRef Full Text | Google Scholar

Sofer, C., Dotsch, R., Wigboldus, D. H. J., and Todorov, A. (2015). What is typical is good: the influence of face typicality on perceived trustworthiness. Psychol. Sci. 26, 39–47. doi: 10.1177/0956797614554955

PubMed Abstract | CrossRef Full Text | Google Scholar

Straube, T., Langohr, B., Schmidt, S., Mentzel, H.-J., and Miltner, W. H. R. (2010). Increased amygdala activation to averted versus direct gaze in humans is independent on valence of facial expression. Neuroimage 49, 2680–2686. doi: 10.1016/j.neuroimage.2009.10.074

PubMed Abstract | CrossRef Full Text | Google Scholar

Thomaz, C. E., and Giraldi, G. A. (2010). A new ranking method for principal components analysis and its application to face image analysis. Image Vis. Comput. 28, 902–913. doi: 10.1016/j.imavis.2009.11.005

CrossRef Full Text | Google Scholar

Thompson, P. (1980). Margaret Thatcher: a new illusion. Perception 9, 483–484. doi: 10.1068/p090483

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, A. (2008). Evaluating faces on trustworthiness. An extension of systems for recognition of emotions signaling approach/avoidance behaviors. Ann. N Y Acad. Sci. 1124, 208–224. doi: 10.1196/annals.1440.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, A., Baron, S. G., and Oosterhof, N. N. (2008a). Evaluating face trustworthiness: a model based approach. Soc. Cogn. Affect. Neurosci. 3, 119–127. doi: 10.1093/scan/nsn009

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, A., Said, C. P., Engell, A. D., and Oosterhof, N. (2008b). Understanding evaluation of faces on social dimensions. Trends Cogn. Sci. 12, 455–460. doi: 10.1016/j.tics.2008.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorov, A., Olivola, C. Y., Dotsch, R., and Mende-Siedlecki, P. (2015). Social attributions from faces: determinants, consequences, accuracy and functional significance. Annu. Rev. Psychol. 66, 519–545. doi: 10.1146/annurev-psych-113011-143831

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorović, D. (2006). Geometrical basis of perception of gaze direction. Vision Res. 46, 3549–3562. doi: 10.1016/j.visres.2006.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Todorović, D. (2009). The effect of face eccentricity on the perception of gaze direction. Perception 38, 109–132. doi: 10.1068/p5930

PubMed Abstract | CrossRef Full Text | Google Scholar

Tomasello, M., and Farrar, M. Y. (1986). Joint attention and early language. Child Dev. 57, 1454–1463. doi: 10.2307/1130423

PubMed Abstract | CrossRef Full Text | Google Scholar

Torp, H. (1984). The Integrating System of Proportion in Byzantine art. An essay on the Method of the Painters of Holy Images. Rome: Giorgio Bretschneider.

Google Scholar

Wicker, B., Michel, F., Henaff, M.-A., and Decety, J. (1998). Brain regions involved in the perception of gaze: a PET study. Neuroimage 8, 221–227. doi: 10.1006/nimg.1998.0357

PubMed Abstract | CrossRef Full Text | Google Scholar

Wollaston, W. H. (1824). On the apparent direction of eyes in a portrait. Philos. Trans. R. Soc. Lond. 114, 247–256. doi: 10.1098/rstl.1824.0016

CrossRef Full Text | Google Scholar

Yang, D. Y.-J., Rosenblau, G., Keifer, C., and Pelphrey, K. A. (2015). An integrative neural model of social perception, action observation and theory of mind. Neurosci. Biobehav. Rev. 51, 263–275. doi: 10.1016/j.neubiorev.2015.01.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Zaidel, D. W., and Nadal, M. (2011). Brain intersections of aesthetics and morals: perspectives from biology, neuroscience and evolution. Perspect. Biol. Med. 54, 367–380. doi: 10.1353/pbm.2011.0032

PubMed Abstract | CrossRef Full Text | Google Scholar

Zeki, S. (1999). Art and the brain. J. Conscious. Stud. 6, 76–96.

Google Scholar