“3D·CoD”: A New Methodology for the Design of Virtual Reality-Mediated Experiences in Digital Archeology

Introduction

Digital Archeology is nowadays a mature research area, at the intersection of Archeology and Digital Technologies. A milestone in the consolidation of the field was the establishment in 2011 of the Seville Principles (López-Menchero, 2013), an extension of the London Charter (Denard, 2012). The Principles open the way for different kinds of application, contexts of use, purposes, and methodologies. They also refer to interdisciplinarity and to the need to “adapt” to the aims, goals, and methods of each community of practice. In contrast, many projects consist of 3D navigable photorealistic reconstructions of architectural remains, built with a general visualization purpose, in the context of sequential workflows with clearly delimited roles. If, as declared in most publications, the goal of archeological virtual reconstructions is to help users understand the archeological site, then both theory and experience (Forte and Siliotti, 1997; Barceló, 2001; Dourish, 2001; Bonini, 2008; Tringham, 2012a,b; Hamilakis, 2013; Paliou and Knight, 2013; Papadopoulos et al., 2015), regardless of the archeological epistemological stance, recommend and support the use of virtual reality (VR) to create immersive, populated, fully interactive environments that reproduce the multisensory dimension of the world. In this sense, evaluations seem to indicate that photorealism can be counterproductive for understanding (Gooch and Gooch, 2001) and that empty architectural 3D models generate only superficial knowledge about specific recognizable elements (Bonini, 2008; Pujol and Economou, 2009; Ibrahim et al., 2015). On the other hand, the research conducted in other fields suggests that, for virtual experiences to be effective, specific goals and contexts of use need to be clearly defined (Turner and Turner, 2002;) and that end users and stakeholders need to be also involved in the design process since the beginning (Norman, 1990; Sanders and Stappers, 2008).

This implies a change in the work pipelines typically adopted in Digital Archeology; a change that seems to have started in the most recent projects (Pietroni et al., 2013; Ename, 2015; ACdPC, 2017) thanks to the integration of new professionals and new concepts coming from Media Arts and Interaction Design. Nonetheless, it is our belief that for this transition from 3D architectural models to VR-mediated experiences to be fulfilled, these new design (and evaluation) workflows in Digital Archeology need to be fully developed. This paper presents the first results related to the iterative process of building and testing “3D⋅CoD,” a specific methodology for the design of VR-mediated experiences. This process was achieved by means of a combination of user-centered and participatory design workshops (Scaife et al., 1997; Muller and Druin, 2002), as well as multimodal analysis (Jewitt, 2013), all common methods in Interaction Design.

User-centered design (UCD) and participatory design (PD) are nowadays considered different philosophies in Human–Computer Interaction. The term UCD was coined by the American Donald Norman in the late 80s (Norman, 1990) and refers to the need to take the end user into account since the first stages and throughout the whole process of design. The typical UCD design process is iterative and comprises the following steps: (1) ethnographic research (to understand the needs and expectations of target end users); (2) creation of scenarios of use; (3) specification of user requirements; (4) prototyping; (5) evaluation with end users; and (6) refinement. In comparison with previous design practices, the goal in UCD was to adapt the product to users (rather than the other way round); yet, these had an external, reactive role, and their input was “translated” by the designers (Scaife et al., 1997), which implied potentially longer processes of refinement.

On the other hand, PD was consolidated also around the 80s, mostly in Scandinavia, and aimed to integrate users as peer cocreators since the earliest stages of design (Scaife et al., 1997; Muller and Druin, 2002). This allows users to act as codesigners, informants, users, and testers (Malinverni et al., 2016), and the result are products or experiences that are truly tailored to the needs and expectations of users (Sanders and Stappers, 2008). Yet, this is not without challenges; namely, how to facilitate understanding and negotiation among very diverse codesigners (Muller and Druin, 2002). This is why research has focused on the politics of the creative process, as well as on the methodologies for its elicitation (e.g., bodystorming) and analysis (e.g., multimodal analysis). The choice of PD as the philosophy underlying “3D⋅CoD” and its design had two aims: first, to test if the advantages verified in Interaction Design (Sanders and Stappers, 2008) may also apply to Digital Archeology; and second to ground empirically our theoretical and methodological proposition linked to Cultural Presence.

This paper will first present the concept of Cultural Presence, as it constitutes the theoretical and methodological basis for the creation of VR-mediate experiences. Second, it will provide an overview of selected projects that illustrate the evolution of design workflows in Digital Archeology and their relation with the final outcomes. Then, it will describe the process of building “3D⋅CoD.” This process started with an initial workshop at the UNESCO World Heritage site of Çatalhöyük (Turkey), in which site experts were engaged to define a specific instantiation of Cultural Presence and how to depict it by means of VR (Section “Defining VR-Mediated Experiences: The UCD Workshop”). Subsequently, the results of the fieldwork were translated into a design methodology (Section “Building a Design Method for VR-Mediated Experiences”) and tested with a different group of archeologists (Section “Evaluation: The Participatory Design Workshop”). The results of this evaluation provided interesting conclusions about the suitability of the “3D⋅CoD” methodology, as well as ideas for future improvement.

Cultural Presence

The term Cultural Presence was coined in the early 2000s in the field of Presence, a highly interdisciplinary area at the intersection of human–computer interaction, psychology, and computer graphics. The notion of Cultural Presence was based on James J. Gibson’s ecological theory of perception (Gibson, 1950), and it defined a culturally meaningful interactive environment in which users could communicate and cooperate (Riva et al., 2002; Spagnolli et al., 2003). The concept was imported into the Cultural Heritage field by Erik Champion (2005), and was subsequently developed in several publications (Champion, 2007, 2015; Pujol and Champion, 2012). In the new domain, Cultural Presence was not the goal, but a means to serve the different, evolving purposes of Cultural Heritage. The term corresponded to the feeling that people belonging to a specific culture occupied or had occupied a virtual environment. This implied not only coherence in behaviors and material culture but also agency, consciousness, and ambiguity (Champion, 2007).

The integration of (Cultural) Presence as a framework for Digital Archeology allows potentially to benefit from the advantages of its well-grounded theories of perception, interaction, learning, and psychology, as well as of its diversity of well-tested methodologies for assessment. The only “missing” aspect is the design process, which follows different strategies depending on the area of application (e.g., engineering, videogames, arts, etc.) and seems to be taken for granted in publications. On the other hand, it has also been emphasized the importance for Presence research to take into account the context of use (Turner and Turner, 2002), and the fruitfulness to expand the analytical scope of Presence with theoretical insights coming from other fields (Klimmt and Vorderer, 2003).

In this context, {LEAP] (Learning of Archeology through Presence¹), a 2-year MSCA funded by the European Union (FP7-PEOPLE-2013-IEF, n. 625537), aimed to develop the fields of Digital Archeology and Presence by: (1) importing and expanding the concept of Cultural Presence; (2) devising new methodologies of design (“3D⋅CoD”) to (3) build a VR-mediated experience (“ÇH3D”) based on the UNESCO World Heritage site of Çatalhöyük; and (4) creating new evaluation tools (the “Cultural Presence Questionnaire”) in order to (a) verify the reality of the concept, (b) compare its manifestation in different kinds of models, and (c) test its suitability for learning. The different outcomes of this project are the object of several forthcoming publications.

Design Pipelines in Digital Archeology

One of the first and most famous projects is the virtual reconstruction of the Roman baths near Caerleon, in the UK (Reilly, 1992). Archeologists provided the floor plans and historical information, used by specialists in computer graphics to reconstruct the building by means of a “solid modeling” system purposefully programmed to that end. A “walk-through” with multiple sequential views was manually created to be displayed in different contexts (e.g., on site, in TV documentaries, etc.). Several other projects, with similar workflows and underlying concepts (visualization of the site) were developed at that time, e.g., Dudley Castle (Boland and Johnson, 1997).

Many of the experiences conducted during the 90s crystallized in 2000–2002 in the EU-funded CHARISMATIC project.² This project aimed to investigate and provide tools for the fast modeling of high-fidelity environments populated with intelligent characters (Niccolucci, 2012). During the 2000s, projects were mostly oriented toward the production of accurate 3D models for visualization. The natural evolution of CHARISMATIC was 3D CO-FORM,³ which during 2008–2012, undertook to build standards and tools for the use and creation of 3D models in Cultural Heritage, from data capture to dissemination. As (mass) digitization was the top trend at the moment, 3D modeling pipelines focused more than ever in the production of accurate/photorealistic architectural models (both terms were synonyms), and started and ended with computer graphics: 3D scanning (instead of the traditional floor plans provided by archeologists), processing of point clouds, texturization, rendering, and navigation programming.

One of the big highlights of the 2000s is “Rome Reborn” (Moltenbrey, 2008). This impressive project aimed at the reconstruction of Rome in A.D. 320. It started at the University of California—Los Angeles and continued at the University of Virginia, involving an international team of experts, mostly from Italy and the USA. It was developed as an online tool for research and dissemination, and its latest upgrade was presented in 2009. The scientific team comprised archeologists, architects, historians, linguists, computer scientists, and 3D modelers. The design pipeline consisted broadly of three successive phases of development and refinement: (1) historical research and content providing (gathering the available information about Rome’s urban configuration); (2) modeling (creation of the architectural model of Rome, which later was populated with dynamic characters to “increase realism”); and (3) technical implementation: implementation (optimization of the model for online visualization).

In spite of the differences in technological capacities (e.g., processing power) and in 3D modeling (manual programming vs. ready software), the design process from the 1990s until the first decade of the 2000s was the same: archeologists provided the basic information about the site and then 3D modelers created an architectural reconstruction, which was refined following the archeologists’ revision. However, things started to change during the second decade of the new century. One of the reasons is probably the introduction of two interrelated elements acting at different levels: on the one hand, the adoption of new theoretical concepts, such as storytelling, gamification, and Virtual Museum; on the other hand, the incorporation in the interdisciplinary teams of specialists in communication, interactive visualization, and/or computer games. As a result, virtual reconstructions ceased to be pure 3D models to come closer to “user experiences.”

V-MUST,⁴ an EU-funded Network of Excellence running during 2011–2015, is the most representative example of this new trend. V-MUST generated a series of similar projects, such as “Livia’s Villa Reloaded”⁵ or the “Virtual Museum of the Tiber Valley.”⁶ The novelty in these projects is the use of gesture-based navigation, narration, and scenes played by real characters inserted in the virtual environment. While the accurate reconstruction of the site is the main tenet, more attention is given to the user experience. In the same context, we find “Etruscanning 3D,”⁷ another EU-funded project running during 2011–2012. This project aimed at the visualization of an Etruscan tomb by means of gesture-based interaction, manipulation of objects, and evocative storytelling (Pietroni et al., 2013). The interdisciplinary team comprised archeologists, museum professionals, experience designers, 3D modelers, and VR programmers. As a result, the user experience took even more importance; however, it was defined at the end of the traditional design process and, therefore, the experience remained close to the usual exploration of a model to obtain specific information. A similar project is “Ullastret 3D” (ACdPC, 2017).

While previous projects were built from scratch, “Ename 1290”⁸ represents the re-use and transformation of previous 3D models from virtual reconstructions into more elaborated user experiences. “Ename 974” (Callebaut, 2002; Pujol and Economou, 2007), a reconstruction of a lost Belgian abbey, was initially part of an interactive installation displayed at the Ename Archeological Provincial Museum. Its goal was to show the evolution of the abbey in time and contextualize the archeological finds displayed in the same room. This model was originally built collaboratively by archeologists and 3D modelers. In 2015, with the introduction of a studio specializing in interactive visualization, the user experience increased dramatically, and the old “Ename 974” model, aimed purely at visualization, became “Ename 1290,” a game using gesture-based interaction to explore the abbey and manipulate objects to unlock descriptions about them (Ename, 2015).

A completely different approach is that of “Okapi Island,” a reconstruction of the Neolithic site of Çatalhöyük, created several years ago in Second Life (Morgan, 2009). While several other models of Çatalhöyük have been created in the last years, e.g., at the universities of Southampton,⁹ Duke,¹⁰ and Boston and Virginia,¹¹ Okapi stands out because it¹² was built by different specialists in a more horizontal and participative way. Unlike the projects previously mentioned here, its goal was not to recreate the architecture and/or the environment as they were in the past, but to focus on humans in the present, their social interactions, and their scientific interpretations. As a result, Okapi Island reproduced both the settlement and the archeological campsite, and users were also agents who could modify the model. This experience was based in a completely different approach to design and communication, since it blurred the hierarchical boundaries between researchers and non-experts.

In this section, we have seen a range of VR applications in Archeology spanning across time (1992–2015) and space (UK, USA, Belgium, Spain, EU). Its goal was to outline the general evolution of design pipelines in Digital Archeology. This overview shows a transformation from 3D reconstructions for visualization to more elaborate user-centered experiences relying on enhanced exploration and storytelling. This process may ultimately end up in full-body, multisensory, VR-mediated experiences about past cultures. However, there is a clear distinction of roles: archeologists provide the basic information, which 3D modelers use (sometimes they are the same person) to generate a model aimed at representing objectively what the site was like; to that model, different technologies aimed at enhancing the user experience are applied (by professionals from the creative industries), usually at the end of the process. The technology is decided since the beginning, but this is not equivalent to user experience. To be able to complete this transition toward VR-mediated experiences, we put forward the hypothesis that new design (and evaluation) workflows are needed.

Defining VR-Mediated Experiences: The UCD Workshop

The LEAP project based its work on the Neolithic site of Çatalhöyük in Turkey, declared World Heritage by UNESCO in 2012. Thanks to its state of conservation and the methodology set up to investigate it (Hodder, 2000), Çatalhöyük provided the optimal resolution to implement and test the concept of Cultural Presence. A general definition of the concept had been initially established, but since we started from the premise that the notion of culture is not universal (Pujol and Champion, 2012), we needed to characterize a specific instantiation of the concept and how to depict it by means of VR. This was the goal of the field work conducted at Çatalhöyük from July 25, 2015 to August 6, 2015, which was organized as a UCD exercise (aimed at defining the system requirements). To that end, the collaboration of nine expert archeologists working at Çatalhöyük for several years, and representing the major specializations involved in the study of this site, was sought: human remains, scientific illustration, chipped stone, field archeology, post-chalcolithic occupations, conservation, faunal remains, and heritage interpretation. It was decided not to involve any member of the digital excavation team to avoid any influence by current practices. It is important to note that while these experts may belong to different epistemological schools, they all had in common the use of the “reflexive method” (Hodder, 2000), which is a post-processual theory that aims to acknowledge in a systematic way the role of subjectivity in archeological interpretation.

Methodology

To obtain a specific characterization and implementation of Cultural Presence, a combination of objective and subjective approaches was used. In the first case, the experts were video-recorded while describing Çatalhöyük 9,000 years ago under three different conditions (Figure 1): at the archeological site (North Shelter); at the Experimental House (the reconstruction of a typical Çatalhöyük dwelling built close to the Visitors Center); and by means of two different illustrations of Building 45 (labeled for the purposes of this project as Images I and II). While Image I (not included in Figure 1) depicted only the architecture, Image II (included in Figure 1) represented the same building with people inside, in a scene of daily life. VR applications share features with the original site (raw data, environment), the Experimental House (immersivity), and the images (presence or absence of humans); on the other hand, VR has some capacities (dynamism and simulation) that cannot be found in the previous conditions because they come from VR’s computational virtuality. We analyzed the video recordings to see which features were spontaneously used or considered important by experts when describing verbally and/or visually a past culture. On the other hand, in the case of the subjective approach, the experts replied to a questionnaire comprising 16 questions. These questions gathered their opinion about the way the different conditions supported their descriptions; the five most defining features of Çatalhöyük in the past; how to depict them; and the general features of an ideal virtual reconstruction of Çatalhöyük. The questionnaire also helped triangulate the observations.

FIGURE 1

Figure 1. The three conditions used during the UCD workshop to define Cultural Presence and how to depict it with VR. Left: the North Shelter of Çatalhöyük. Centre: an illustration of Building 45 by illustrator Mesa Schumacher, labeled as Image II (Image I, not included here, corresponded to the same building, without people). Right: the Experimental House at Çatalhöyük (Credits: Çatalhöyük Research Project).

The procedure lasted around 20 min for each participant, who were naïve to the final goal of the experience. The researcher would meet with each expert in the main working room, where she would make a first introduction to the experience and would handle an informed consent form including: detailed information about the LEAP project; the workshop procedure and data gathering techniques; protocols for data management; and explicit requirements for permission to be recorded and to allow the use of some images in academic papers and the project’s website. All participants signed the informed consent form. These procedures, as well as the different informed consent forms used during the project, had been submitted at project proposal stage and approved both by the Bioethical Committee of the University of Barcelona and the European Commission.

Subsequently, the expert was first taken to the site, then to the Experimental House, and finally back to the main room where he/she would be handled Images I and II. In all three cases he/she had to do the same exercise: describe what life was at Çatalhöyük 9,000 years ago. This instruction was purposefully wide or ambiguous, so that experts would have the freedom to provide explanations ranging from existential disquisitions to pure feature description. The researcher would not intervene at all, and the expert would stop whenever he/she considered he/she was done. Then he/she would receive the self-administered questionnaire, and would have 3 days to answer it and return it to the researcher.

The field work collected 9 questionnaires and 478 observation entries. The data analysis followed different methods: qualitative for questionnaires and quantitative for observations. In the first case, for each question, the universe of possible answers was reduced to a set of categories. In the second case, the observations were transformed into a database. The experts’ speeches were broken down into themes (topics), articulated through basic, subordinated units of meaning (subtopics), for which different characteristics were recorded: gestures (e.g., look at, point at, perform); affordances used (e.g., oven, ladder, house); emphasis (e.g., general aspects, tasks and space; people); approach (e.g., illustrative, critique, descriptive); verbal person; and tense. These characteristics constitute a combination of analytical categories used in the different levels of multimodal (Jewitt, 2013) and discourse analysis (Ruiz, 2009). To ensure attribution consistency in the case of ambiguous sentences, a protocol was established based on semantic fields, keywords, and observations. While the categories in Emphasis, Approach, Gesture, and Tense were defined a priori, the potential values for the rest of variables were established afterward, by recording all the different values and/or reducing them to a set of common, clearly distinct categories.¹³

All statistical analyses were conducted with IBM SPSS 19. A general hypothesis was formulated that the three different conditions or “media” (site, Experimental House, and Images I and II) would trigger significantly different gestures, approaches, and especially, topics and emphases. On the other hand, the human factor may also be playing a role. Therefore, to test the aforementioned hypotheses, a series of Chi square tests were undertaken. They looked for significant associations between “medium” (or expert) and discourse features. Unfortunately, the cell count in the contingency tables was low in too many cases, which could invalidate the Chi square tests. The measures undertaken were the following: eliminate Image I from the tests (because it was seldom used by participants); eliminate silences and moments of difficulty; and perform again Chi square tests, this time running the Monte Carlo test, a simulation procedure used for contingency tables with low cell count. Afterward, a series of Multiple Correspondence Analyses were performed, in order to better describe the nature of the aforementioned significant relationships.

Results

Questionnaire

With regard to the usefulness of the different media to support a discourse about the past, the site was considered suitable to talk about general, relational aspects of the settlement, such as urbanism, social networks, subsistence, and the diachronic dimension (changes in time, tradition and evolution, and seasonal rhythms of life). It was also ranked the best to talk about methodology, because it is the source of raw data. The Experimental House was considered suitable to talk about distribution of spaces and activities; about the “opposition” between life inside and outside the house (in “middens” and roofs); about cleanliness of spaces; and about the close relation between life and death. The Experimental House was ranked as the best to talk about both the past and architecture because “it was immersive and contained objects, without imposing too much interpretation.” Image I (the isometric depiction of Building 45) was clearly associated with architecture, but in general participants were not comfortable with it. It was ranked the worst to talk about the past because it was empty (of people and objects) and focused purely on architectural aspects. Two positive aspects though were underlined: (1) it provided information not available on the other “media” (e.g., heights); and (2) it did not impose a too “complete” image of the past, allowing imagination and personal interpretation. For this reason, it was ranked the second best to talk about architecture (after the site). Image II (showing a scene of daily life inside Building 45) was clearly associated with people, the activities they did, and the spaces they occupied. However, several problems were reported: too much interpretation; lack of dynamism; lack of diachronic change; and being colorless (“it fails to show life”). This is the reason why it was ranked second in terms of suitability to talk about the past.

According to the participants, an ideal virtual reconstruction should: contain basic data (like the site); be immersive and multisensory (like the Experimental House); and show people and objects (like Image II). But at the same time, it should show evolution/change in time; enhance learning with different kinds of representations (e.g., image, text); incorporate dynamism; provide different perspectives; and allow natural, full-body, multisensory interaction to make users feel what it was like to be there. That is, it should go beyond the simple reconstruction of reality and take advantage of VR’s capacities (virtuality, multimediality, real time interaction) to enhance representation and communication of our knowledge of the past.

Finally, in order to depict the defining elements of Çatalhöyük as a culture the VR experience should be realistic from an archeological and human perspective, which means: navigation should happen at eye’s view (instead of the bird’s view that is usual in many virtual models); the approach to the site should go from the house to the outside world (instead of going from the landscape to the site, as it usually happens in virtual reconstructions); the model should definitely include objects, people, and animals to provide an impression of a busy, crowded place (instead of the usual ghostly aspect of many current virtual models); it should include detail and variability (as opposed to standardized textures); it should provide a multisensory, (socially) interactive experience from the inside of the culture (instead of just allowing passive, external observation); it should convey synchronous dynamism as well as temporal depth; it should make a clear distinction between actual remains and reconstruction; and finally, it should allow the possibility to complement the reconstruction with written information at certain points, as a way to enhance learning about the site and about archeological methodology.

Observations

When describing the life at Çatalhöyük 9,000 years ago, experts mentioned by order of importance the following aspects:

1. Distribution of spaces and activities in the house.

2. Social relations/collaboration for subsistence/belonging, household.

3. Temporal depth, stratigraphy, preservation/House cycles/Traditions and changes over time.

4. Meaning/role of features (pits, caches, niches, paintings, etc.).

5. Rhythms in life/tasks defined by seasons/light.

6. Different life/tasks/space for different people.

7. Houses very close/living together/population density.

In order to seek for associations between topics and experimental conditions (Site, Experimental House, and now only Image II), we conducted Chi Square and Correspondence Analyses. With regard to the former, there was a significant and quite intense association between Condition and General Topic [Chi square = 78.809 dl × 38 dl; Sig. Monte Carlo (10,000 samples and 99% CI) = 0.0001; Cramer’s V = 0.518]. In the case of Correspondence Analysis, the three conditions show (Figure 2) a triangular opposition and trigger some different topics, while share many others. Image II is mostly associated with different life/tasks/spaces for different people, and with the notion of a crowded environment. Life outside vs. inside is also a usual topic, but shared with the other two media. (Experimental) House is specially associated with sensory stimuli and body movements. Organic objects and cleanliness were exclusively mentioned there. The distribution of spaces and activities, and the importance of rhythms were some of the most mentioned topics, but shared with the other conditions. Finally, the site was heavily associated with social networks (shared with Image II). This would be confirmed by a major use of the third plural person [Chi square = 22.358 dl × 8 dl; Sig. Monte Carlo (10,000 samples and 95% CI) = 0.0033]. The role of specific archeological features and subsistence were almost exclusive of the site.

FIGURE 2

Figure 2. Correspondence analysis of general topic by condition.

In the case of Emphasis, there is also a significant but moderate association with Condition [Chi square = 86.994 dl × 4 dl; Sig. Monte Carlo (10,000 samples and 99% CI) = 0.0001; Cramer’s V = 0.319]. In the Correspondence Analysis (Figure 3), the three conditions show again a clear triangular opposition, and confirm the hypothesis that the site triggers a general perspective related to life and settlement; the Experimental House is linked to space and activities; and Image II is about people. In this case, the first axis opposes (Experimental) House to Site and Image II, indicating a clear distinction between them. This distinction is based on the House’s reconstructive dimension: the topics associated with this condition had to do with sensory stimuli and bodily movements. Furthermore, the Experimental House elicited a strong sense of physical presence, which is confirmed by the fact that it was the only condition in which experts mostly used the present tense [Chi square = 17.648 dl × 4 dl; Sig. Monte Carlo (10,000 samples and 95% CI) = 0.0002].

FIGURE 3

Figure 3. Correspondence analysis of emphasis by condition.

Regarding the affordances used in each condition, significant and quite intense associations were found [Chi square = 324.250 dl × 46 dl; Sig. Monte Carlo (10,000 samples and 99% CI) = 0.0001; Cramer’s V = 0.609]. In the Correspondence Analysis (Figure 4), the three conditions show a clear triangular opposition, and affordances tend to form clusters around them. The opposition in this case is between Site (on one side of the first axis) and Image and House (on the other side). The site is mostly associated with specific archeological features and buildings. Image II is clearly linked with the human figures depicted on it. The light is also an important element, but shared with House. The Experimental House is the richest in affordances and also relies on specific features and objects. The distinction between areas of activity is mostly done here.

FIGURE 4

Figure 4. Correspondence analysis of affordances by condition.

In the case of gestures, there is a significant but low association with the different conditions [Chi square = 34.889 dl × 8 dl; Sig. Monte Carlo (10,000 samples and 99% CI) = 0.0001; Cramer’s V = 0.200]. The reason for the lowest score is that there was also an influence of the expert’s personality. Again, the first axis of the Correspondence Analysis (Figure 5) opposes Image and House to Site. Although performing gestures is common to all conditions, it would seem they are more closely associated to the Site. Indeed, on site experts used a lot of gestures to represent their explanations. The House, richer in affordances, prompted experts to only point at things. In the case of Image II, probably because of its small size, experts mostly looked at it. This would seem to confirm the hypothesis that the richer the medium, the smaller the need to complement explanations with gestures.

FIGURE 5

Figure 5. Correspondence analysis of gestures by condition.

Toward VR-Mediated Experiences

The combined analysis of observations and questionnaires shows there was no contradiction between the experts’ opinions and behaviors regarding the defining elements of Çatalhöyük as a culture (topics and subtopics) and the best way to depict them (affordances, media, emphasis) by means of VR. The featuring elements declared in the questionnaires and inferred from the most mentioned topics during observations can be generalized in order to define the general categories that should be included in any virtual world: architecture and settlement; subsistence; social organization; symbolic aspects; and time. It is important to note that most categories are divided into a general perspective (corresponding to the way archeologists describe the culture) and a human perspective (corresponding to the way that individuals would perceive it). In our opinion, Cultural Presence should attempt to disseminate the former through the latter. That is, users should have a first person experience in order to understand the general features of that culture.

To that end, virtual reconstructions should combine different characteristics, represented in our study by the different experimental conditions. In order to talk about general aspects related to social organization, subsistence, and urbanism, they should adopt the general perspective of the site. They should include detail and variability as in the real world. The basic archeological data also come from there and, therefore, should be included in a secondary layer that could be switched on and off. To convey the human dimension, virtual reconstructions should take elements from both the Experimental House and Image II. This image is valuable because it depicts objects and especially humans, showing their different roles and spaces of activity. As declared by experts, virtual worlds should contain animated objects, animals, and people to give the impression that it was a busy, dynamic world. On the other hand, the Experimental House adds immersivity, multisensory approach, and full-body interaction, which is necessary to understand the world from a more phenomenological point of view (Dourish, 2001; Tringham, 2012a,b). It provides the sense of inhabitation of spaces both in the past and the present. This requires that navigation is done at eye’s view, and that it starts from the house/settlement toward the environment, because this corresponds to the way humans apprehend and relate to the world. Finally, the temporal perspective can only be provided by endowing the virtual world with a dynamic, narrative component.

The fieldwork conducted in summer 2015 provided conclusions about the main features virtual environments should have in order to achieve a feeling of Cultural Presence. These guidelines were used, on the one hand, to build “ÇH3D,” a virtual reconstruction of Çatalhöyük with six different versions (Figure 6) aimed at testing the validity of the concept and its suitability for learning (Pujol, 2016, 2017); and on the other hand, to devise a design methodology (“3D⋅CoD”) for the interdisciplinary co-creation of VR-mediated experiences.

FIGURE 6

Figure 6. The VR-mediated experience “ÇH3D.” From left to right, building 49 as depicted in experimental conditions 1 (architecture only), 3 (hotspots with information), and 5 (dynamic scenes).

Building a Design Method for VR-Mediated Experiences

“3D⋅CoD” attempted to integrate design strategies and tools coming from PD (co-creation, low-tech prototypes) with basic archeological methodology (case description, specification of sources, etc.). It is loosely based on the “Storyboard method” (Charreau and Johnson, 2015). The new method was divided in four phases, labeled by the researcher as follows: (1) Case Description (15 min); (2) Experience design (20 min); (3) Model Building (20 min); and (4) Evaluation (35 min). The activity was designed to last 90 min so that it was not too tiring and could be repeated as many times as needed. It required the use of paper, cardboard, a small card and its envelope, colored post-its, markers, pens and pencils, rubbers, glue, and/or adhesive tape.

Case Description

The process starts by choosing a site or monument and writing its basic information on a cardboard divided in four sections: period, cultural aspects, sources, and site features. This is done so that the basic information is agreed upon and available to everybody (especially the sources, for alternative reconstructions and metadata/paradata).

Experience Design

Experience Design comprises two steps: Message Definition and Content Definition. Regarding the former, designers write on a small card the main “secret message” to be conveyed with the experience, which is then put in the “mystery envelope.” This step has two aims: first, to reinforce the idea that the goal of the VR-mediated experience is to convey cultural knowledge (not just to describe an archeological site) and, second, to introduce a playful element to increase motivation during the design process. The second step corresponds to the definition of the content. Again, designers use a cardboard divided in four sections: environment, structures, people, and objects. Each section is color-coded and designers have to fill in each section with elements or ideas written down on the corresponding post-its. The goal of this step is to help designers take into account all archeological perspectives (economic, social, symbolic, etc.) and not just architecture or the environment. Moreover, for each section “actions” need to be defined, so that dynamic events and/or user actions are specified.

Model Building

Subsequently, this 2D board is translated into a 2D or 3D paper mockup. Structures, environmental elements, people, and material culture are cut and built on paper with the respective colors previously established. Drawings can be added, as well as post-its indicating actions and metadata/paradata. The goal of the model is to visualize and make tangible the experience as a whole; the use of the color code helps establish clear visual correspondences between the conceptual phase and the model.

Evaluation

Evaluation comprises two phases: Model Exploration and Experience Verification. Once designers have finished building their prototype, they need to call a small group of people (e.g., museum staff) who did not participate in the design process. This group of “users” will have some time to explore the model and fill in a “questionnaire” asking three questions: (1) what they have learnt; (2) thanks to what elements; and (3) what is the main message of the experience. The evaluation is complete when the “mystery envelope” is opened and designers compare what “users” have learnt/understood and what they intended to convey. This final playful element intends to stress in a “dramatic way” the importance of evaluation and iterative design: the contrast between communicational intention and actual reception may show that simulation and visualization are not necessarily universal, and help refine the experience.

Evaluation: The Participatory Design Workshop

“3D⋅CoD,” the design methodology for VR-mediated experiences, had been created by merging the results of the fieldwork conducted at Çatalhöyük about the implementation of Cultural Presence with participatory design strategies from Interaction Design. Now, we needed to test it in a different context. The evaluation took place on November 16, 2015 at the central premises of the Archeological Service of the Catalan Government. In contrast with the UCD workshop conducted at Çatalhöyük, the evaluation was organized as a PD exercise (aimed at jointly creating VR-mediated experiences with the help of the methodology). 21 staff members belonging to several departments of the Archeological Service took part in it. Archeologists were chosen again as designers/testers for the following reasons: they are experts in the basic involved knowledge domain; they are a fundamental part in the design of VR applications; they are used to playing a very specific role in the design workflow (as initial content providers); and in this case, they shared a processual concept of Archeology. Hence, the workshop would show the initial reception of a novel methodology by a different group of end users and, at the same time, help refine it.

Methodology

The evaluation of the design methodology comprised two parts: observation and debriefing. At the beginning of the workshop, the researcher made an introduction to the goals of the experience, the data gathering techniques, and the future uses of this information. Formal consent to take pictures and eventually use them in scientific publications was requested by means of a general form, signed by all attendees. Subsequently, observation was organized as a PD workshop, in which the researcher acted as facilitator. Participants were divided in three groups of five people and one group of six people. The researcher first proposed the goal of the exercise: to choose a site currently under investigation and to create a VR-mediated experience based on it with the help of the new design methodology. Then, she described the methodology and presented the available materials. As this was the first, exploratory evaluation of the design methodology, she gave loose indications about the different design possibilities to see what ideas the participants would come up with spontaneously. Once the design exercise began, the researcher was free to move around, taking photographs and notes about the development of the session. She had made explicit her availability to solve doubts, but the groups carried out their tasks without requiring any assistance and, therefore, she only intervened in the process by controlling time and indicating the need to move from one phase to the next.

The participatory design workshop lasted overall 90 min, but the relative timings established by the methodology were not kept: 15 more minutes were allocated to the conceptual phases, to the detriment of the final evaluation phase. Afterward, a break was taken and the participants reconvened for a final 30 min debriefing about the workshop. Given that, as previously said, this was the first evaluation of the design methodology, a qualitative, exploratory, and flexible approach was chosen. Thus, the debriefing was organized as a focus group in which the participants would reply to and debate around five questions: first impressions from the experience; things liked and disliked; opinion about the usefulness of the design method; things they would improve; and if it had changed their perception of 3D modeling.

Results

Observations

Participants committed to the activity (Figure 7) with enthusiasm but also a certain initial skepticism. They seemed to feel more comfortable and took longer time for “Case description” and “Experience Design.” As a result, in order to have enough time for “Model Building,” time had to be taken from the final phase, corresponding to “Evaluation.” The intra-group dynamics depended on people’s familiarity and personality: in some cases, a person arose spontaneously as leader; in others, the task was developed thanks to more or less coordinated individual initiatives. In any case, all the team members contributed according to their skills and personality. During the conceptual phases, two groups spontaneously used complementary tools (tablet, mobile phone) to retrieve or verify information about their case.

FIGURE 7

Figure 7. Archeologists from the Catalan Archaeological Service during the different phases of the design/evaluation workshop. Written informed consent was obtained from all research participants for the publication of identifiable images. Still, we have blurred faces as expressions are not relevant for this paper.

With regard to the VR-mediated experiences, participants chose Catalan archeological sites from the Iberian (2 groups) and Neolithic periods (2 groups). Three groups focused on the economic perspective, and aimed to explain the human relation with the environment by describing a specific moment in time. As a result they produced 3D paper mockups of 3D models. These groups understood “actions” both as simulational dynamism and as user actions. Regarding the former, they specified the representation of people doing different tasks. No sounds were foreseen. Regarding the latter, they designed the model as a navigable virtual world, with bird’s view perspective to explore the overall environment, and human perspective to explore the settlement. No manipulation of objects or social interaction was foreseen. Two groups mentioned during their presentations specific interfaces: head-Mounted Displays for a fully immersive experience; and mobile devices for an on-site augmented experience.

The fourth group, who had chosen a painted rock shelter, focused on the symbolic perspective and aimed to describe the role of the site along history. As a result, they proposed a documentary-like experience, which they presented by means of a 2D storyboard. Since the experience was conceptualized as a sequence of scenes providing an abstract diachronic approach, there was no interaction foreseen. Seemingly, they did not specify any sounds or human presence, but it is possible that at this design stage they took them for granted. Neither they nor the three previous groups indicated the use of metadata/paradata or any distinctions between actual remains and reconstruction. Only in one case alternative sources were listed in the “Case description” poster board, but they were not included in the model. This seems to indicate all participants had in mind a strictly simulational, processualist concept of 3D models.

As there were four groups, and in order to enhance enjoyment and understanding of the design process, no external “users” were called for the Evaluation phase. Instead, all groups moved to the right and explored the experience created by their neighbors. As previously mentioned, due to the longer time allocated to previous phases, Evaluation was done more in haste. Groups explored and commented the models with evident enjoyment and then filled in the “questionnaires.” The final Experience Verification revealed a perfect correspondence between the information the “modelers” intended to convey and the message “users” understood after exploring the models. Only in one case, “users” perceived a social dimension that was not included in the mystery envelope but that was reflected in the model by means of the inclusion of people doing tasks.

Debriefing

The final debriefing aimed to gather the participants’ impressions about “3D⋅CoD” and triangulate the observations made during the design workshop. With regard to the first impressions, participants expressed their positive reception of the experience, which was completely novel for most of them. They valued especially two aspects: the participatory dimension, and the emphasis on the “message” to be conveyed over the description of the site. On the negative side, they could not see initially the benefit of physically building the model, especially because a big leap from the paper mockup to the VR model was required.

While they could see the potential of the overall methodology, they had reservations about its universal application because of the lack of archeological accuracy (i.e., metrics, materials, full scale, etc.). This is why they considered such methodology may be useful for the first stages of design, as a common communication tool between archeologists and 3D modelers. For the same reason, they also mentioned a higher usefulness for 3D models aimed at dissemination: as one participant said, “it makes you think about the purpose and the [user] experience as a whole, without the effort of going into modeling and without the specific constraints of the 3D modeling tool.”

Regarding how much the experience had changed their perception about 3D models/3D modeling, there were mixed answers. Some participants said they continued thinking 3D modeling was aimed at generating virtual reconstructions for universal visualization. Hence they did not see the value in building the paper mockup, but appreciated the participatory dimension. Some others said that they were not sure how much they would apply the method (as is) in the future, but the workshop had certainly changed their perspective about the capacity of current navigable 3D models to show what the past was like: it had opened a window into the concept of “user experience” and the need to put more emphasis on dynamism and on (at least) interaction with objects.

Discussion

As in other participatory design/evaluation workshops conducted by the researcher, archeologists showed a lot of enthusiasm, but also a certain initial skepticism regarding the activity. In general, design and evaluation methodologies that are commonplace in Interaction Design are still alien to the Cultural Heritage field. This is probably the reason why, as previously mentioned, participants felt more comfortable and took longer time to describe the case, to establish the message to be conveyed, and especially to define the content.

With regard to the VR-mediated experiences, they consisted of the usual 3D models aimed at the simulation and visualization of a settlement in ancient times. Contrary to the perspective and guidelines defined during the fieldwork at Çatalhöyük, only general aspects related to economy or chronological evolution were expressed. There was no human, personal perspective, related either to the inhabitants’ daily life or to the user’s digital experience. Dynamism was used purposefully to convey a specific message (instead of a static description of a site), but there were no metadata/paradata; or distinction between actual remains and reconstruction; or user interaction beyond navigation. Some participants said during the debriefing that, in order to enhance the user experience, interaction with objects may be included; but social interaction involved too much “subjective interpretation for a scientific model.” This may indicate that, in contrast to Çatalhöyük site specialists, who shared a postmodern concept of the archeological method and had stressed the importance of daily life and human experience, the participants of the evaluation started from a processual concept of Archeology, which may have influenced their perception about VR-mediated experiences and the design methodology.

The Evaluation phase provided useful insights about current 3D models and their design. For example, the communication of the main message was successful in general; only in one case, “users” mentioned (because of the inclusion of people doing tasks) a social dimension that was not initially foreseen by the experience “designers.” However, these findings cannot be extrapolated beyond this workshop because the participants were experts who knew very well the chosen cases, and therefore needed few clues to understand the message to be conveyed. In order to increase understanding in the case of general audiences, previous studies (Pujol and Economou, 2007, 2009) suggest that VR-mediated experiences need to display dynamic human characters or, even better, aim at an “enhanced virtuality” (e.g., hotspots, metadata/paradata, storytelling, etc.) rather than at pure reconstruction. These findings were verified during the evaluation of “ÇH3D,” the VR-mediated experience that arose as second outcome of the fieldwork at Çatalhöyük, and will be disseminated in forthcoming publications.

With regard to the opinions expressed during the debriefing, they confirmed some of the conclusions drawn from the observations: first, the appreciation of the participatory dimension; and, second, the acknowledgment of the importance to convey a message in contrast to pure architectural description. For some participants, the impact of the workshop was more substantial, and they expressed that the experience had opened a window into the concept of “user experience” and the need to put more emphasis on dynamism and interaction. That is, on Cultural Presence: our proposal as theoretical foundation for Digital Archeology. Yet, participants expressed their belief this method was more suitable for 3D models aimed at dissemination. This indicates they took for granted the current uses of virtual reconstructions (visualization, description) and did not consider other potential research applications of VR, for example as a tool capable of transforming archeological questions, theories, and methods (Reilly, 1991; Tringham, 2012b; Hugget, 2015).

What does this tell us about “3D⋅CoD”? First, that the method did not manage to convey the importance to emphasize the human perspective, as intended with the concept of Cultural Presence. “Experience Design” focused too much on specific descriptive elements (architecture, material culture, environment, and people) and not enough on dynamism and interaction. Building a static 3D paper mockup probably reinforced this artistic, descriptive perspective. Seemingly, the method did not manage to stress (in the “Mystery Envelope”) the importance of user experience over communication of information. In other words, we proposed a methodology still too much based in current modeling practices and, therefore, we did not encourage the exploration of alternative perspectives or uses through VR-mediated experiences.

It can be argued that our second group of participants were not experts in Interaction Design and VR. This may be true for the former, as indeed participants expressed with amazement their discovery of participatory methods. However, during the event they mentioned several times the possibilities of videogames, which were dismissed as a scientific communication means and restricted to a certain type of “very informal” dissemination, necessary for the younger audiences. This would indicate again the influence of the participants’ archeological stance: they were in favor of developing the “user experience” to enhance knowledge, as long as this did not require to go beyond the limits of “objective” representation. As a matter of fact, this conclusion was also confirmed during the evaluation of “ÇH3D,” the VR-mediated experience based on Çatalhöyük: the statistical analyses (which will be disseminated in forthcoming publications) indicated that the perception of 3D models is not universal, but rather is influenced by specific demographic variables, one of them being the level of expertise in the related knowledge domain.

Conclusion and Future Work

This paper presented the first steps of development of “3D⋅CoD,” a new methodology for the design of VR-mediated experiences in Digital Archeology. This methodology was built and evaluated in the context of the EU-funded Marie Curie project {LEAP]. Such research endeavor arose from the fact that, despite VR’s capacity to generate immersive, multisensory, interactive experiences, and despite the indication from empirical studies that photorealism is insufficient for learning, many VR applications aim to show what the past was like by means of empty, photorealistic architectural models. {LEAP] chose the UNESCO World Heritage Neolithic site of Çatalhöyük (Turkey) to develop the concept of Cultural Presence as theoretical and methodological foundation for a new kind of archeological virtual reconstruction. In contrast to current 3D models, VR-mediated experiences are human/user-centered, and based on the notion of “enhanced virtuality” (as opposed to pure reconstruction) to develop a more phenomenological approach to the past. A survey of design pipelines in Digital Archeology from its origins to the current moment, indicated that, to create such experiences, a new design and evaluation strategy needed to be built.

Hence, {LEAP] undertook the following process. A first workshop, based on a UCD approach and multimodal analysis, was held at Çatalhöyük with the aim to define the best way to implement the concept of Cultural Presence in VR-mediated experiences. The results of this first step were then translated into a design method (based on paper mockups and participatory design strategies), which tried to emphasize the human experience (both in the past and in the presence). Finally, the suitability of this method was tested by means of a PD workshop with another group of archeologists. The conclusion is that the design methodology should: support interdisciplinary design in a tangible and systematic, but also enjoyable way; clearly set the different stages of iterative design and guide experts through them, from the choice of the specific case (site, culture, event) to the evaluation and refinement of the experience; emphasize multisensory human perspectives over the general visualization of architectural and/or natural spaces; start from the user’s experience instead of from the 3D model; and, finally be generic and flexible enough so that it can be used in different design contexts (e.g., research and dissemination).

The results of the evaluation provided the basis to develop a new version of “3D⋅CoD.” There are two possibilities. On the one hand, we could keep the basic methodology and include more user actions for designers to take into account and more corresponding “building” materials for the mockup 3D model. However, it is possible that such strategy, based on a descriptive, reconstructive concept of 3D modeling, generates again the same kind of experiences found currently in Digital Archeology. Therefore, there is the alternative possibility to change completely the approach and focus on intangible heritage, human experiences, and actions. In order to create multisensory, phenomenological VR-mediated experiences of the past, the methodology should not comprise a description of the site, the information to be conveyed, and a poster board defining elements, but rather start with bodystorming exercises (Schleicher et al., 2010), the definition of the global user takeaway, and a poster board defining events and interactions. In this case, the paper mockup should consist of a storyboard made of images showing different, successive states of the VR-mediated experience.

The suitability of “3D⋅CoD_2.0” should then be tested by comparing both methodological approaches with archeologists belonging to the same and to different archeological schools. Indeed, the contrast between the suggestions made by archeologists at Çatalhöyük (belonging to a postmodern trend) and the results of the evaluation at the Catalan Archeological Service (belonging to a processual trend) suggest that the theoretical stance may also play a fundamental role in the perception and design of VR-mediated experiences in Digital Archeology. On the other hand, the refinement of “3D⋅CoD” would also benefit from including a more varied set of profiles in the codesign workshops: for instance museum professionals, educators, modelers, experience designers, and last but not least, representative members of different audiences.

Ethics Statement

This study was carried out in accordance with the recommendations of: article 8 in the Charter of Fundamental rights in the European Union (specifically the article concerning the protection of personal data); Directive 95/46/EC of the European Parliament and the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data; and Spanish Organic Law 15/1999—13th of December for the Protection of Personal Data (LOPD). All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Bioethical Committee of the University of Barcelona.

Author Contributions

The project development and writing were performed by LP-T.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The author would like to thank: the several specialists from Çatalhöyük Research Project, especially Sara Perry and James Taylor, who provided the basic information and participated in the first design workshop; the archeologists from the Archeological Service of the Catalan Government who participated in the evaluation of the methodology; the trainees from CEV-Barcelona, the 3D artist and VR programmer from Athens (Hara Sfyri and Dimitris Christopoulos), the students, professors, and engineers from UPF, and the multimedia designers from Athens and Lleida who built the virtual environment; the several end users, who participated in its evaluation; and finally, Prof. Joan A. Barceló (UAB) for his assistance with the statistical analyses, and Prof. Narcís Parés (UPF), for his invaluable suggestions during the project development and the writing of this paper.

Funding

The research leading to these results has received funding from the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme FP7/2007–2013/under REA grant agreement no. 625537.

Footnotes

1. ^www.upf.edu/leap.
2. ^http://cordis.europa.eu/project/rcn/55067_en.html.
3. ^http://cordis.europa.eu/project/rcn/89256_en.html.
4. ^http://www.v-must.net/.
5. ^https://vimeo.com/81825976.
6. ^https://vimeo.com/129867454.
7. ^https://www.youtube.com/watch?feature=player_embedded&v=iiW4dbfo5yU.
8. ^https://www.youtube.com/watch?v=ttGVlQW297s.
9. ^https://www.youtube.com/watch?v=pAV8z6NesOA.
10. ^https://www.youtube.com/watch?v=uzjga2GThpo.
11. ^http://www.catalhoyuk.com/content/3d-catalhoyuk-project-animation.
12. ^https://vimeo.com/119412330.
13. ^For example, topics would be separated by their social, economic, funerary component, etc. Emphasis would look for keywords, such as “life,” “space,” “activities,” or “people,” and check their syntactic position and semantic role in the sentence. Regarding Approach, “descriptive” corresponded to the expert strictly describing an archaeological feature; “illustrative” meant he/she was using the affordance to infer or synthesize knowledge about the society; “critique” implied a positive or negative judgment about the medium.

References

ACdPC. (2017). Ullastret 3D, a Virtual Reconstruction of an Iron Age Town. ViMM Platform. Available at: http://www.vi-mm.eu/2017/03/13/ullastret-250-b-c-a-virtual-reconstruction-of-an-iron-age-town/

Google Scholar

Barceló, J.A. (2001). Virtual reality for archaeological explanation. Beyond picturesque reconstruction. Archeologia e Calcolatori 12:221–44.

Google Scholar

Boland, P., and Johnson, C. (1997). Archaeology as computer visualisation: “virtual tours” of Dudley Castle c. 1550. In Imaging the Past. Electronic Imaging and Computer Graphics in Museums and Archaeology, 227–234. London: The British Museum.

Google Scholar

Bonini, E. (2008). Building virtual heritage environments: the embodied mind at the core of the learning processes. International Journal of Digital Culture and Electronic Tourism 1:113–25. doi: 10.1504/IJDCET.2008.021402

CrossRef Full Text | Google Scholar

Callebaut, D. (2002). The experiences of the Ename 974 project with new media: where and how to do virtual reality and interactivity fit in? In Virtual Archaeology. Proceedings of the VAST Euroconference (Arezzo 24–25 November 2000), 179–185. Oxford: Archaeopress.

Google Scholar

Champion, Erik. (2005). “Cultural presence,” in Encyclopedia of Virtual Communities and Technologies, ed Subhasish Dasgupta (Hershey, PA: George Washington University, Idea Group), 95–101.

Google Scholar

Champion, E. (2007). Social presence and cultural presence in oblivion. In Proceedings of the 7th International Digital Arts and Culture Conference: The Future of Digital Media Culture, Perth, Australia.

Google Scholar

Champion, E. (2015). Role-playing and rituals for cultural heritage-oriented games. In Proceedings of DIGRA 2015 “Diversity of Play: Games – Cultures – Identities”, Luneberg, Germany: DIGRA.

Google Scholar

Charreau, M., and Johnson, J. (2015). The Storyboard Method. Available at: http://english.thestoryboardmethod.com/

Google Scholar

Denard, H. (2012). A new introduction to the London charter. In Paradata and Transparency in Virtual Heritage, eds A. Bentkowska-Kafel and H. Denard (London: Routledge), 57–72.

Google Scholar

Dourish, P. (2001). Where the Action Is. The Foundations of Embodied Interaction. Cambridge: MIT Press.

Google Scholar

Ename. (2015). Visualisation of the Benedictine Abbey of Ename. How a Bunch of Stones Becomes an Intriguing Story. Ename Blog. Available at: https://enameabbey.wordpress.com/1290-game/

Google Scholar

Forte, M., and Siliotti, A. (1997). Virtual Archaeology. London: Thames & Hudson.

Google Scholar

Gibson, J.J. (1950). The Perception of the Visual World. Boston: Houghton Mifflin.

Google Scholar

Gooch, A., and Gooch, B. (2001). Using non-photorealistic rendering to communicate shape. Non-photorealistic rendering. In SIGGRAPH 99 Course Notes, Los Angeles, 8.

Google Scholar

Hamilakis, Y. (2013). Archaeology and the Senses. Human Experience, Memory and Affect. Cambridge: Cambridge University Press.

Google Scholar

Hodder, I. (2000). Towards Reflexive Method in Archaeology: The Example of Catalhöyuk. Cambridge: McDonald Institute for Archaeological Research.

Google Scholar

Hugget, J. (2015). Challenging digital archaeology. Open Archaeology 1:79–85. doi:10.1515/opar-2015-0003

CrossRef Full Text | Google Scholar

Ibrahim, N., Mohamad Ali, N., and Mohd Yatim, N.F. (2015). Factors facilitating cultural learning in virtual architectural heritage environments: end user perspective. ACM Journal on Computing and Cultural Heritage 8:8. doi:10.1145/2660776

CrossRef Full Text | Google Scholar

Jewitt, C. (2013). Multimodal methods for researching digital technologies. In Handbook of Digital Technology Research, eds S. Price, C. Jewit, and B. Brown (Los Angeles: SAGE), 250–265.

Google Scholar

Klimmt, C., and Vorderer, P. (2003). Media psychology “is not yet there”: introducing theories on media entertainment to the presence debate. Presence 12:346–59. doi:10.1162/105474603322391596

CrossRef Full Text | Google Scholar

López-Menchero, V.M. (2013). International guidelines for virtual archaeology: the Seville principles. In Good Practice in Archaeological Diagnostics, eds B. Slapšak and C. Corsi (Berlin: Springer), 269–283.

Google Scholar

Malinverni, L., Schaper, M.M., and Parés, N. (2016). An evaluation-driven design approach to develop learning environments based on full-body interaction. Educational Technology Research and Development 64:1337–60. doi:10.1007/s11423-016-9468-z

CrossRef Full Text | Google Scholar

Moltenbrey, K. (2008). History in the making. Computer Graphics World 31. Available at: http://www.cgw.com/Publications/CGW/2009/Volume-31-Issue-12-Dec-2008-/History-in-the-Making.aspx

Google Scholar

Morgan, C. (2009). (Re)Building Çatalhöyük: changing virtual reality in archaeology. Archaeologies 5:468. doi:10.1007/s11759-009-9113-0

CrossRef Full Text | Google Scholar

Muller, M.J., and Druin, A. (2002). Participatory design: the third space in HCI. In Human-Computer Interaction Handbook, eds A. Sears and J. A. Jacko (New Jersey: Lawrence Erlbaum Associates), 1051–1068.

Google Scholar

Niccolucci, F. (2012). Setting standards for 3D visualization of CH in Europe and beyond. In Paradata and Transparency in Virtual Heritage, eds A. Bentkowska-Kafel and H. Denard (London: Routledge), 23–36.

Google Scholar

Norman, D.A. (1990). The Design of Everyday Things. New York: Doubleday.

Google Scholar

Paliou, E., and Knight, D.J. (2013). Mapping the senses: perceptual and social aspects of late antique liturgy in San Vitale, Ravenna. In Proceedings of Computer Applications and Quantitative Methods in Archaeology International Conference, Granada: CAA.

Google Scholar

Papadopoulos, C., Hamilakis, Y., and Kyparissi-Apostolika, N. (2015). Light in a Neolithic dwelling: building 1 at Koutroulou Magoula (Greece). Antiquity 89:1034–50. doi:10.15184/aqy.2015.53

CrossRef Full Text | Google Scholar

Pietroni, E., Pagano, A., and Rufa, C. (2013). The Etruscanning project. Gesture-based interaction and user experience in the virtual reconstruction of the Regolini-Galassi tomb. In Digital Heritage International Congress, Marseille: IEEE.

Google Scholar

Pujol, L. (2016). Being there and then. Cultural presence for archaeological virtual environments. In Proceedings of EuroVR International Conference, Athens: EuroVR.

Google Scholar

Pujol, L. (2017). Depicting “çatalhöyükness”. Çatalhöyük Research Project. Available at: http://www.catalhoyuk.com/content/depicting-catalhoyukness

Google Scholar

Pujol, L., and Champion, E. (2012). Evaluating presence in cultural heritage projects. International Journal of Heritage Studies 18:83–102. doi:10.1080/13527258.2011.577796

CrossRef Full Text | Google Scholar

Pujol, L., and Economou, M. (2007). Exploring the suitability of virtual reality interactivity for exhibitions through an integrated evaluation: the case of the Ename museum. Online International Museology Journal 4:84–97.

Google Scholar

Pujol, L., and Economou, M. (2009). Worth a thousand words? The usefulness of immersive virtual reality for learning in cultural heritage settings. International Journal of Architectural Computing 7:157–76. https://doi.org/10.1260/147807709788549367

Google Scholar

Reilly, P. (1991). Towards a virtual archaeology. In Proceedings of Computer Applications and Quantitative Methods in Archaeology International Conference 1990, 133–139. Oxford: Archaeopress.

Google Scholar

Reilly, P. (1992). Three-dimensional modelling and primary archaeological data. In Archaeology and the Information Age. A Global Perspective, eds P. Reilly and S. Rahtz (London: Routledge), 147–176.

Google Scholar

Riva, G., Castelnuovo, G., Gaggioli, A., and Mantovani, F. (2002). Towards a cultural approach to presence. In Proceedings of the 5th International Workshop on Presence, 305–309. Porto: ISPR.

Google Scholar

Ruiz, J. (2009). Sociological discourse analysis. FQS 10. Available at: www.qualitative-research.net/index.php/fqs/article/view/129812882/

Google Scholar

Sanders, E.B., and Stappers, P.J. (2008). Co-creation and the new landscapes of design. CoDesign 4:5–8. doi:10.1080/15710880701875068

CrossRef Full Text | Google Scholar

Scaife, M., Rogers, Y., Aldrich, F., and Davies, M. (1997). Designing for or designing with? Informant design for interactive learning environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’97), 343–350. New York: ACM.

Google Scholar

Schleicher, D., Jones, P., and Kachur, O. (2010). Bodystorming as embodied designing. Interactions 17:47–51. doi:10.1145/1865245.1865256

CrossRef Full Text | Google Scholar

Spagnolli, A., Varotto, D., and Mantovani, G. (2003). An ethnographic, action-based approach to human experience in virtual environments. International Journal of Human-Computer Studies 59:797–822. doi:10.1016/S1071-5819(03)00120-4

CrossRef Full Text | Google Scholar

Tringham, R. (2012a). Households through a digital lens. In New Perspectives on Household Archaeology, eds B. J. Parker and C. P. Foster (Winona Lake, IL: Eisenbrauns Publishing), 81–120.

Google Scholar

Tringham, R. (2012b). “Sensing the place of çatalhöyük: the rhythms of daily life,” in. Last House on the Hill. BACH Area Reports from Çatalhöyük, eds R. Tringham and M. Stevanović (Los Angeles, CA: Cotsen Institute of Archaeology Press).

Google Scholar

Turner, P., and Turner, S. (2002). Embedding context of use in CVE design. Presence 11:665–76. doi:10.1162/105474602321050767

CrossRef Full Text | Google Scholar