Neural Networks (SOM) Applied to INAA Data of Chemical Elements in Archaeological Ceramics from Central Amazon

ABSTRACT Artificial neural networks represent an alternative to traditional multivariate techniques, such as principal component and discriminant analysis, which rely on hypotheses regarding the normal distribution of the data and homoscedasticity. They also may be a powerful tool for multivariate modeling of systems that do not present linear correlation between variables, as well as to visualize high-dimensional data in bi- or trivariate structures. One special kind of neural network of interest in archaeometric studies is the Self-Organizing Map (SOM). SOMs can be distinguished from other neural networks for preserving the topological features of the original multivariate space. In this study, the self-organizing maps were applied to concentration data of chemical elements measured in archaeological ceramics from Central Amazon using instrumental neutron activation analysis (INAA). The main objective was testing the chemical patterns previously identified using cluster and principal component analysis, forming groups of ceramics according the multivariate chemical composition. It was verified by statistical tests that the chemical elemental data was not normally distributed and did not present homogeneity of covariance matrices for different groups, as requested by principal component analysis and other multivariate techniques. The maps obtained were consistent with the patterns identified by cluster and principal component analysis, forming two chemical groups of pottery shards for each archaeological site tested. Finally, it was verified the potential of SOMs for testing if failures in underlying hypotheses of traditional multivariate techniques might be critically influencing the results and subsequent archaeological interpretation of archaeometric data.


Introduction and theoretical background
Artificial neural networks (ANN) represent an alternative to traditional multivariate analysis methods, such as principal component and discriminant analysis. They provide a useful tool for multivariate modelling of systems with non-linear correlation among variables. Two main applications of ANN may be identified: function approximation by retropropagation algorithms, and grouping or classification of input vectors (Merdun, 2011).
The main applications of artificial intelligence to archaeometric studies regards the comparison of geochemical data classification by traditional algorithms, such as hierarchic cluster analysis (CA) and principal component analysis (PCA), and neural networks based methods. One main advantage is that the latter do not depend on any specific data distribution (Bell & Croson, 1998). A recurrent method for such a purpose is the application of self-organizing maps (SOMs) (Toyota, 2009), developed by Teuvo Kohonen. Those maps are based in AAN in a frequently nonsupervised leaning strategy, and do not need a previous set of test samples of an already known structure (Lopes-Molinero, et al., 2000). A graphical representation of a SOM is presented in Fig. 1. The method is partially based in the way sensorial information are processed in separate parts of the human brain cortex.
As in the case of principal components and discriminant functions, those maps are a useful tool for graphical visualization of high dimensional data in bi-or tridimensional structures. The maps are formed by nodes, or neurons, and every node has a weigh vector with the same dimension as the data, or inputs. The neuronal geometry may be hexagonal or rectangular, which influences the number of neighbors. SOMs can be distinguished from other ANNs by the preservation of the topological properties of the original space, by using neighborhood functions.
In the SOM method, an input vector is allocated in the neuron unit that has the most similar weigh vector, or the nearest, based in an adequate metric to measure the distances between vectors. The general idea is that the weigh vectors representative of each neuron are spatially correlated, so that near representative vectors in the grade are more similar between them when compared to distant vectors.
The SOM training consists in a competitive unsupervised learning, which assign input vectors to the most similar neuron, represented by its weigh vector. In this way, this method aids in identifying grouping patterns in the data set. As a new input vector is presented to the net, its Euclidean distance to every weigh vector associated to a neuron is calculated. The "winning" neuron, called best matching unit (BMU), and neighboring vectors have their weigh adjusted due to the input vector. The magnitude of the adjustment decreases over time and with the distance to the BMU, according the following updating equation (adapted from Lopez-Molinero, et al., 2000) W where t is the iteration or epoch of the algorithm, D(i) is the input vector (i is the sample size of the data set for training, varying from 1 to n), θ(v,t,d) is the neighborhood function (frequently Gaussian or Triangular) depending on the distance d to the BMU, and α(t) is the monotonically decreasing learning coefficient. The learning process initiate with large neighborhoods to the point where its definition turns more restrict, and the weigh vector of each neuron converge to local estimates in the map. An epoch of training is complete when all input vectors are presented once to the net for correction of neuronal weigh vectors. At the end of the procedure, the neurons, or output nodes, may be associated to the groups present in the multidimensional space of input by graphical visualization, with color scales representing the Euclidean distances between the neurons, stored in a U matrix.
The batch version of SOM algorithm, where all the input vectors are presented simultaneously to the net in a unique update is more recent. In the literature, there are still few works dealing with the application of self-organizing maps to archaeometric data (Lopez-Molinero, et al., 2000;Toyota, 2009).

Archaeological background
This work comprises the analysis of pottery shards data from two large and important archaeological sites in Central Amazon, namely Lago Grande and Osvaldo. The main archaeological objective is the verification of potential cultural and commercial exchange networks in the region. Once confirmed, it would have impact on the traditional theories about the occupation of pre-colonial Amazon, based on environmental determinism, which concluded that large sedentary settlements and hierarchical and complex human organizations could not emerge in the tropical rain forest environment (Meggers, 1996;Lima, 2008;Heckenberger & Neves, 2009 and references therein).
Previous archaeometric analysis of data from Lago Grande and Osvaldo led to the determination of two chemical groups of pottery for each archaeological site. They presented pairwise superposition, which were further explored seeking for archaeological correlations regarding territorial integration, commercial exchange and exogamic marriage between the inhabitants of the region (Hazenfratz-Marks, 2014). The statistical significance of the chemical groups was tested indirectly by comparison with variation coefficients found in archaeometric studies of pottery around the world (Harbottle, 1982;Bishop, 2003), by discriminat analysis and by multivariate analysis of variance. Table 1 present the average concentrations of nine chemical elements used in previous studies for each chemical group defined for the two archaeological sites. Fig. 2 presents a refined projection of the multivariate data in the two first principal components.

Methods
In this work, a study with self-organizing maps was developed for comparison with multivariate patterns identified by CA and PCA of geochemical data of pottery shards from two large archaeological sites in central Amazon. The results may be found in other publications (Hazenfratz-Marks, 2014;Hazenfratz, et al., 2016). The total data set comprises the concentrations of 9 chemical elements (Sc, Cr, Fe, La, Ce, Eu, Yb, Lu, Th) in 141 samples, measured by instrumental neutron activation analysis (INAA).
INAA is a nuclear analytical technique for elemental analysis. It comprises the bombardment of chemical elements in a sample with neutrons, producing artificial radioactive isotopes, which are identified and measured by the gamma radiation emitted in their decay. Many archaeometric studies employ INAA to determine the concentration of chemical elements in many materials due to its advantages, like the possibility of using small amounts of sample and high sensibility for trace elements (Guinn & Lukens, 1965;Bode, 1996;Glascock et al., 2004).
In a previous work, which generated the data set used in this paper, INAA was applied to measure 24 chemical elements in the pottery shards (As, K, La, Lu, Na, Nd, Sb, Sm, U, Yb, Ba, Ce, Co, Cr, Cs, Eu, Fe, Hf, Rb, Sc, Ta, Tb, Th, Zn). Fifteen elements were selected from the original data set by means of analytic quality control based on analysis of relative standard deviation, bias and tests with z-and u-scores (Hazenfratz-Marks, 2014), yielding the exclusion of Nd, Sb, Sm, U, Ba, Rb, Ta, Tb and Zn. Furthermore, sodium and potassium were excluded due geochemical contamination in the depositional context, probably due to enrichment by intemperism of feldspars as the main geochemical mechanism. Geochemical alterations due to potential diagenetic effects in the concentration profiles of As, Cs and Hf were also identified. Such alterations could hide geochemical patterns and influence the archaeological interpretation of data and were excluded from further statistical analyzes. The final variable set was comprised of the chemical elements Cr, Fe, La, Ce, Eu, Yb, Lu, Sc and Th.  In order to analyze the refined data set selected for archaeological pattern recognition, the computational packages Kohonen and CP-ANN (implemented in Matlab 7.6.0 R2008a with the function newsom) were employed to calculate the SOMs (Ballabio, et al., 2009;Ballabio & Vasighi, 2012). It was adopted the non-supervised algorithm so that the previous classifications of samples by CA and PCA might not interfere in the learning procedure. After the convergence of maps, comparisons between the classifications of samples were made. The parameters selected for the algorithm were (Lopez-Molinero, et al., 2000;Tanevska, et al., 2007;Toyota, 2009;Ballabio & Vasighi, 2012): ▪ Model: non-supervised Kohonen map ▪ Net geometry: hexagonal ▪ Number of neurons: 7 × 7 ▪ Number of epochs: 500 ▪ Neighborhood function: Gaussian ▪ Training algorithm: batch ▪ Initialization of neuronal weighs: linear (from data eigenvectors) ▪ Data transformation: base-10 logarithm ▪ Initial learning rate: 0.5 ▪ Final learning rate: 0.01

Results and discussion
In Fig. 3 and 4 the SOMs for Lago Grande and Osvaldo are presented, respectively. Fig. 3 shows that the SOM converged to a configuration where the samples from different chemical groups were allocated in different regions of the map. The white and hachured neurons represent approximately the different group domains.
The group g1 has a higher number of neurons associated to it, and it may be correlate of a higher geochemical variation in the samples. In fact, by comparison of Fig. 2 and 3, regarding the refined PCA results, it is possible to observe that the 95% confidence ellipse associated to the group A (called g1 here) is larger than the ellipse associated to group B (called g2 here), which may be interpreted as another indication of the same higher geochemical variability. The same interpretation may be claimed for Fig. 4, regarding the SOM for Osvaldo archaeological site. The higher chemical variability of group g1 in Lago Grande was interpreted as possible superposition of different clay recipes, and it was hypothesized that it may represent pottery shards from other archaeological sites in the region that might have participated in larger exchange networks than previously thought initially, and not only between Lago Grande and Osvaldo (Hazenfratz-Marks, 2014). Fig. 5 refers to the application of SOM algorithm for the samples of pottery shards from Lago Grande and Osvaldo in a combined fashion. It was observed that the map converged to a configuration where the pottery shards of group g1 of Lago Grande and group g3 of Osvaldo presented superposition and were allocated in a common region in the map (white hexagons). This region is distinct from the region where the pottery shards from groups g2 of Lago Grande and g4 from Osvaldo were allocated, presenting the same superposition observed for groups g1 and g3 (blue hexagons). The three neurons highlighted in orange represent a transition region, with different mixing of samples. However, they represent the smallest portion of the map.
The results of the data sets from Lago Grande and Osvaldo combined are in agreement with the combined CA and PCA analysis which identified at least two chemical groups of pottery shards for each archaeological site, with pairwise superposition, as it can be observed in Fig. 2.

Conclusion
The Kohonen maps, a type of neural network, were applied to INAA data regarding the concentration of chemical elements in pottery shards from Lago Grande and Osvaldo, two large and important archaeological sites in central Amazon. The chemical grouping patterns identified here agreed with the results of previous multivariate statistical methods using cluster and principal component analysis, which identified two chemical groups of pottery shards for each archaeological site, with pairwise superposition. It indicates that the archaeological community can rely on the chemical groups of ceramic artifacts defined for those archaeological sites so far, and can use them for archaeological interpretation and basis for planning further archaeometric research regarding the pre-colonial Amazonian occupation.
It was also verified the potential of the self-organizing maps for archaeometric data analysis and comparison with multivariate statistical methods which rely upon hypotheses regarding the normal distribution of data and the homogeneity of the covariance matrices. Such a comparison could indicate if the deviation of underlying hypotheses of more traditional multivariate analysis would affect the results in pattern recognition, used frequently to analyze archaeometric data sets with the objective of identifying chemical groups of artifacts and/or identifying provenance.