Narrative of the annotated Space–Time Cube – revisiting a historical event

ABSTRACT The Space–Time Cube (STC) is a suitable representation to display multiple characteristics of movement data and will especially reveal temporal patterns in the data. By adding annotations to the cube’s paths and stations, the narrative of the display is enhanced. To illustrate the STC’s storytelling capabilities, a historical event, Napoleon’s crossing of the Berezina River during his Russian campaign in 1812 is presented and linked to an event in 2012 when the authors made a similar trip. Also, a set of different visual queries and their results are presented, emphasizing the STC as an alternative addition to a more extended visualization environment.


Introduction
Maps are known to be the ultimate graphic representations to reveal spatial patterns, and to offer insight into spatial relations. When the time component is explicitly involved as well, maps can tell stories or explain events.
The strength of maps is that they do this so well because they are abstractions and simplifications of reality. However, this also brings limitations, because to remain readable, the information content might be limited. This will especially affect paper maps.
For on-screen maps, this is not necessarily a problem because it is possible to change scales, and to link all kinds of additional information using media such as sound, video, and pictures to individual map elements, which becomes available by clicking or mouse-over. This was explored first in electronic atlases (Siekierska, 1984), and developed into interactive multimedia atlas information systems. The national atlas of Switzerland was a prominent example (Hurni, Bär, & Sieber, 1999).
Individual maps with these capabilities are sometimes called hypermaps after Apple's hypercard. The term was introduced by Laurini and Milleret-Raffort (1990). According to Kraak and Van Driel (1997), these can be described as 'georeferenced multimedia systems that can structure individual multimedia components with respect to each other and to the map'. They allow the user to navigate the data according to themes or location.
Today, this principle has gained popularity in socalled story maps, which are maps embedded with other media, like the above hypermaps, and allow the user to follow a selected story line based on the geography depicted in the map. GIS vendors have embraced story maps. For instance, ESRI advertises them as: ' … let you combine authoritative maps with narrative text, images, and multimedia content'. 'They make it easy to harness the power of maps and geography to tell your story' (ESRI, 2016). Not much different from earlier approaches, but today, the market is ready for them, especially because of progress in web mapping over the last years.
These approaches allow the user to follow a predefined story path or explore the story according to self-selected paths. Such approach is often implemented in Coordinated Multiple View environment as commonly found in exploratory tools (Heer & Shneiderman, 2012;Roberts, 2005), despite some criticism (Andrienko & Andrienko, 2007). Figure 1 offers an example of the story told in this paper, Napoleon's crossing of the Berezina River.
New technological opportunities have also raised the interest of the scientist in the phenomena of mapping stories or narrative mapping (Caquard, 2013;Caquard & Cartwright, 2014;Caquard & Fiset, 2014). However, in most examples, time and space are not that well linked. A visual representation that inherently links space and time is the Space-Time Cube (STC). Although mostly used to show trajectories only, it also allows one to express attribute information.
In our contribution, we take the STC as the core representation of the narrative. To illustrate our arguments, we use a historical event, Napoleon's crossing of the Berezina River in 1812, linked to a contemporary event, the authors' visit to the same area in 2012. The main objective is to demonstrate the STC in the role of storyteller, allowing to compare and link two events. It should be seen as an alternative view next to other (map-like) representations. The annotations will convert the STC into an infographic bringing it alive, and engaging the reader into the story.

Space-Time Cube
The STC was originally introduced as part of time geography, developed by the Swedish geographer Hägerstrand (1970) who studied space-time activities of humans. Time geography sees space and time as inseparable parts, which can visually be well represented by the cube. The cube's horizontal plane represents space, and the vertical axis represents time. The basic concepts in time geography are paths, stations, and prisms. Paths show movement behavior of objects through space and time (Kraak, 2003;Kwan, 1999;Li, Çöltekin, & Kraak, 2010). The stations indicate locations where people stay for longer moments (Andrienko, Andrienko, Mladenov, Mock, & Pölitz, 2012;Hadlak, Tominski, Schulz, & Schumann, 2010), and prisms representing space reachable within a given time budget (Miller, 1999;Yu, 2006). An example of the STC containing a space-time path and several stations is shown in Figure 2(b). Several (commercial) implementations of the STC for analytical purposes exist, among them GeoTime (Kapler & Wright, 2005;Proulx, Khamisa, & Harper, 2010). Our implementation has incorporated only those functions related to specific multimedia elements (photo, movie, document) that are of interest to the narrative.
The 2D map in Figure 2(a) shows the path followed by the authors plus four points of interest annotated by a photo. In Figure 2(b), the same information is represented in the STC, but the points of interest are annotated by a numeric identifier. The authors' trip started at 08:30, passed several points of interest and ended again at 11:30. The points of interest (stations) reveal themselves at those locations where the path is nearly vertical for a while. Especially at point 4, one can see the authors stayed at those locations significantly longer. The stop times are indicated as text labels in Figure 2(a) and are emphasized along the time axis in Figure 2 The 2D map provides a good overview of the trip, and the points of interest are clearly indicated. However, there is no notion of time, except for the stop labels along the photographs. The STC does give immediate insight into the relation between space and time and, as such, also shows how long the stops at the points of interest took. The cube's spatial overview might be a little less clear than the map view, but the overall character of the trip is better preserved because directions and time are implicit.

Historical event
During 1812, Napoleon and his Grande Armée undertook their fatal march on Moscow. This march was immortalized in Minard's famous map 'Carte figurative des pertes successives en hommes de l'Armée Française dans la campagne de Russie 1812-1813' of which a detail is seen in Figure 1(d) (Kraak, 2014). It is also known from Tolstoy's historical novel 'War and Peace'. Napoleon started the campaign because Tsar Alexander ignored the Continental System. This was established by the French to avoid all trade between continental Europe and England. After a long preparation, Napoleon crossed the Neman River near Kaunas in current Lithuania with over half a million troops on 24 June 1812. He tried to force the Russians into a big battle, but the Russians aware of Napoleon's capabilities were able to avoid this until September when the battle at Borodino took place. After that, Napoleon was able to occupy Moscow. However, after a month, he decided to return. At that time, his Grande Armée was only one-fifth of its original size. The retreat was dramatic under intensive cold and harsh winter circumstances. The crossing of the Berezina River was one of the last major combats of the Grande Armée during this campaign (see Figure 3).
Napoleon barely escaped the Russian troops and lost half of his remaining army during this crossing. Of the original half million troops, only about 10,000 returned. Figure 1(d) depicts this event in different map details. One of the author's family members Gerrit Janz was a soldier in the 125th Regiment of the IX Army Corps, who were sent to Russia to support Napoleon's retreat. On 27 November 1812, he fell during the battles at the banks of the Berezina River. His wanderings were the reason of the authors to visit the Berezina area as depicted in Figure 2. The combination of the 1812 and 2012 event is the core of the STC story told in the Main Map. Here, the cube allows an easy link and comparison between the two events 200 years apart.

Software: generating the maps
We used the open source GIS and Remote Sensing software package ILWIS (http://52north.org/communities/ ilwis) to construct the 2D maps and the STCs. The 2D map for the trip made in 2012 is generated from the OpenStreetMap, and the path, based on GPS files is plotted on top (see Figure 2(a)). This map is also used as base map in the cube. It can be moved along the vertical axis to follow the trajectory. The 2D map for the Berezina River crossing in 1812 is obtained from the Atlas to Alison's History of Europe (Johnston & Alison, 1848, plate 78), and the paths are based on the data derived from Smith (2002), and plotted on top of the scanned map. The map is georeferenced, and used as base map in the cube (see Figure 3).
In both 2D map and the cube, a path's qualitative and quantitative attribute information can be shown using different colors or varying the width of the path, respectively. Annotations, such as video recordings, texts, and drawings, can be linked to the paths. In the ILWIS software, the 2D map and the STC are linked (see Figures 2 and 3). The cube can be rotated to position it in any perspective to get the best view on the data.
For the cube in the Main Map, both 1812 and 2012 2D maps were exported to ArcGIS. In ArcScene, they were draped over a terrain model to offer a sense of the topography of surrounding area. In ArcScene, both were positioned such that the view angle would allow telling the story best. From ArcScene, the image of the terrain was exported to Omnigraffle. This package was used for the finishing touch of the all illustrations in this paper. From this production stage onwards, one can no longer manipulate the STC in three-dimensional space, the maps become static now. Omnigraffle, vector drawing software, was used to draw the cube's edges, as well as the vertical lines (stations) between points of interest in both time layers in the Main Map. Also here, the annotations and labels were added, as well as scale indicators for space and time and a north arrow.
Omnigraffle was also used to create the three submaps. For each of the three visual sample queries, the cube was masked to express the locational, attribute, or temporal query. The results, a topographic map, a thematic map, and a time diagram, respectively, were displayed next to the cube.

Design: application of the STC
The STC has a historical map draped on a terrain model at the bottom of the cube representing 1812. The main path of the French and Russian troops is marked by blue and green lines. The paths are annotated with historical scenes and documents from the French army archives. On top, an OpenStreetMap is draped on the same terrain model with the path of the authors in red, representing 2012. The path is annotated by photos of landmarks. Labeled vertical orange lines connect events and locations between 1812 and 2012. For instance, the location of the bridges built by the French to cross the river, and the location of the battle where Gerrit Janz died.
The STC integrated two stories, but via visual query, one can explore more details. The map shows three examples. To explore this historical event from the spatial perspective, and understand the complex situation around bridges spanning the river, one can select the relevant space and display details of a topographic map -(b) in main map.
To show specific attribute information, one can select, for instance, the troops and display the positions of the Russians and French in more detail, (c) in Main Map shows the composition of the French and Russian corps during the crossing of Berezina River.
To reveal temporal aspects of the events, one can select an interval on the timeline, and then choose an object as shown in the example of selected orange lines -(d) in Main Map. Here, the details of the family relation between soldier Gerrit Janz and one of the authors between 1812 and 2012 are revealed via the family tree. The family tree birth and death dates could timewise be displayed in the STC too, but the geography of the family members 'in between 1812 and 2012' has no link to the geography display in the cube.

Discussion and conclusion
Napoleon's Russian Campaign and the story of the fall of one of the author's ancestors, in combination with the authors' trip to the same area, were the motivation for the development of the annotated STC application. This case study depicting the individual events of 1812 and 2012 also allows a good comparison between the two. A few critical remarks have to be made. The application generating the STC is interactive and allows one to select alternative viewpoints required to tell selected parts of the story.
In this particular example, the exaggeration of time along the z-axis is limited because of the 200 years' duration of events displayed. In the cube, the 1812 and 2012 events both represent a discrete time interval (see Figures 2 and 3). To make it more engaging, a terrain model is used in both the 1812 and 2012 event maps. This also implies that the time scale along the z-axis is only continuous on a global scale (overview). If one would zoom in to for instance 2012, the path's heights follow the terrain (space) and not time as is usually the case in an STC. In different circumstances, the use of height and time along the same z-axis of the cube might potentially lead to some confusion. In theory, besides time, the z-axis could also be used to display attribute information. However, here, the objective is to show the relation between 1812 and 2012 which is emphasized by the vertical lines. These lines link today's geography with the past event using the STC concepts of stations.
Other options exist to compare the two events. The most straightforward option is to display two maps, each representing one of the events, next to each other. Another option is to display them on top of each other, and allow the user to slide one over the other (with a selected level of transparency). However, we opted for the STC because it is more engaging and visually attractive in the storytelling context.
In conclusion, the introduced map demonstrates the storytelling capabilities of the STC as an alternative next to more traditional map approaches. It presents the overview of the story but also allows detailed investigation of the narrative from location, attribute, and time perspective.

Disclosure statement
No potential conflict of interest was reported by the authors.