Digital Twin Applications in Urban Logistics: An Overview

Urban traffic attributed to commercial and industrial transportation is observed to largely affect living standards in cities due to external effects pertaining to pollution and congestion. In order to counter this, smart cities deploy technological tools to achieve sustainability. Such tools include Digital Twins (DT)s which are virtual replicas of real-life physical systems. Research suggests that DTs can be very beneficial in how they control a physical system by constantly optimizing its performance. The concept has been extensively studied in other technology-driven industries like manufacturing. However, little work has been done with regards to their application in urban logistics. In this paper, we seek to provide a framework by which DTs could be easily adapted to urban logistics networks. To do this, we provide a characterization of key factors in urban logistics for dynamic decision-making. We also survey previous research on DT applications in urban logistics as we found that a holistic overview is lacking. Using this knowledge in combination with the characterization, we produce a conceptual model that describes the ontology, learning capabilities and optimization prowess of an urban logistics digital twin through its quantitative models. We finish off with a discussion on potential research benefits and limitations based on previous research and our practical experience.


Introduction
Urban Logistics has been growing rapidly in recent years due to rising consumer demand and online shopping, among other trends relating to population growth and urbanization (Savelsbergh & Van Woensel (2016)).As a result, operational planning and policy-making in urban logistics has become increasingly complex.The associated challenges require the development of 'smart' technologies that can assist with planning and resource allocation in urban logistics (Büyüközkan & Ilıcak (2021)).Such technologies are often attributed with smart cities that enjoy a fortified technological infrastructure by which city data can be collected and processed to improve decisionmaking by stakeholders within the city.
Smart technologies are normally derivatives of Artificial Intelligence (AI), which has managed to acquire significant interest from research and industry with its promising capabilities.Specifically, AI has witnessed many industrial applications in urban logistics to deal with real-life planning challenges as discussed in Jucha (2021), Sonneberg et al. (2019) and Shi et al. (2019).Among the various AI-driven technologies presented to tackle urban logistics problems, there is one that we are particularly interested in, namely because of its holistic approach in combining knowledge from different computational models.More precisely, some of the most important AI techniques such as learning and optimization could be easily embedded in its framework.We refer to this as the Digital Twin (DT).
The DT term has been first proposed in 2003 at Michigan University by Professor Grieves to prescribe product life-cycle management as explained in Semeraro et al. (2021).Despite dating back almost two decades, there is still no standardized industrial or academic definition for what constitutes a DT.The term is generally used to refer to any virtual replica of a real-life model but provides no guidelines or technical requirements on the functionalities of this replica.This often leads to confusion among parties with regards to what a DT is and how it is distinct from many existing simulation/decision support systems.We believe that this confusion has presented an obstacle to the development of research in DTs, while this paper aims to overcome this confusion by providing a clear characterization of DTs in urban logistics.
In particular, the definition of what a DT is, its functional requirements and conceptual compo-nents were often determined by the application context (Semeraro et al. (2021)).For the large part, DT applications have largely been manufacturing-based as in Rosen et al. (2015) and Tao et al. (2019).On the other hand, there has been very limited focus on the area of logistics compared to manufacturing as Haße et al. (2019) mentions, despite its importance and emphasis on how it could benefit from Big Data analytics associated with DTs Pan et al. (2021).This benefit only grows with time as cities are becoming increasingly smarter and collect data from a multitude of sources White et al. (2021).Specifically, by using this data, DTs could be used to improve the quality of life, mobility and services of the inhabitants of a city Botín-Sanabria et al. (2022).
In order to define a framework for building DTs, we first have to provide a characterization of urban logistics operations in terms of the key factors that govern it.The human aspect plays an important role in the urban environment as determined by the stakeholders and their interactions (Lagorio et al. (2017)).This aspect is less significant in other domains such as manufacturing where robotic equipment operating in an exclusive environment is mainly responsible for decision-making and implementation which guarantees some consistency in input and output.Urban environments, on the other hand, are seen to be more complex (Rydin et al. (2012)).As a result, a specific characterization upon which urban logistics DTs could be built is needed.
In response to the aforementioned problems, we strive to deliver the following contributions through this study: 1. Characterize urban logistics operations in terms of defining factors for dynamic decisonmaking.We arrive at four major input components we take to be resources, stakeholders, KPIs and measures.
2. Summarize previous findings from literature on DTs in urban logistics in terms of definition, technical anatomy, functionalities and set-up methodology.To the best of our knowledge, there is no existing research that contains a holistic overview of all three topics as previous papers tend to focus on a subset of these topics.
3. Provide a framework on the conceptual anatomy of urban logistics DTs in terms of the AI methods employed.This refers to a three-part framework on knowledge base, machine learning and optimization.

Urban Logistics
We first start by providing a definition for DTs in urban logistics.To arrive at a domainspecific definition -as conventional in the research of DTs -we first need to establish the important aspects of an urban logistics environment that a DT ought to cover.The authors of Savelsbergh & Van Woensel (2016) defines urban logistics as the efficient and effective transportation of goods in urban regions.The scope of our research thus reduces to transportation problems only in contrast to other logistical problems that deal with warehousing, shift-scheduling etc.
To construct a DT model for urban logistics, it is imperative to identify key factors that characterize this spectrum of operations.Anand et al. (2012) provide an ontology for urban logistics whereby these factors are identified.We only consider a subset of the factors that we find relevant as input for dynamic decision-making in urban logistics.These are stakeholders, Key Performance Indicators (KPI)s, resources and measures.Note that decisions are also a key factor in this ontology.However, they are an output factor in response to the aforementioned input factors.We elaborate on the four input factors below.

Stakeholders
The government, businesses and citizens in the urban logistics supply chain are referred to as stakeholders.For a detailed survey on the roles of stakeholders in urban logistics, we refer to Lagorio et al. (2016).The authors of Lagorio et al. (2017) explain how stakeholders have an integral function in defining the ecosystem of urban logistics network through their interests, interactions and decisions.On the other hand,

KPIs
As explained above, stakeholders have interests.These interests translate to objectives which are measured using KPIs as stated in Morana & Gonzalez-Feliu (2015).KPIs could be used to guide optimization procedures for logistical operations as they can be used to represent objective functions.An example can be found in van Heeswijk et al. (2020) who use an agent-based simulation to verify routing schemes.The schemes are assessed by predetermined KPIs representing the objective functions.Sustainable urban logistics networks KPIs normally include -and are not restricted to -CO2 emissions, cost, lead time and delivery travel time as given in Sarraj et al. (2014).For an exhaustive list of some of the most popular KPIs for urban logistics, we refer to Griffis et al. (2007) and Gunasekaran & Kobu (2007).

Resources
Resources refer to all the available resources possessed by all stakeholders in the urban logistics network.Szmelter-Jarosz et al. (2020) explains how urban logistics resources fall into four categories, namely material, human, capital and information.Material resources include machines like trucks, IT platforms etc. Human resources refer to all the laborers involved in executing decisions in the urban logistics supply chain and the decision-makers themselves.Capital refers to the financial resources.Information refers to the intellectual resources such as knowledge and experience.

Measures
Lastly, the measures represent the rules and regulations under which the resources of the digital twins operate.Most often these measures include regulations imposed by policy-makers such as in Russo & Comi (2010) and Muñuzuri et al. (2005) and include restricting goods vehicle access to certain roads such as in heavily congested residential areas.Note that these rules correspond to constraints that are not embedded in the resources themselves, unlike the maximum capacity of a vehicle for example, but rather imposed by the rule of law.
Being practical constraints, the measures could be used to configure modelling constraints when setting up mathematical optimization models for urban operations.Resulting solutions ought to respect the constraints, so that the corresponding real-life decisions remain feasible.Conversely, quantitative approaches could also be adopted by policy-makers to study the effects of their proposed measures as explained in Cardenas et al. (2017).

Digital Twins
We provide a general definition for DTs based on a survey of definitions from several applications from Semeraro et al. (2021).We then inspect literature that specifically discussed DT applications in urban logistics and use this to assemble information on the technical anatomy, functionalities and set-up of an urban logistics DT.Other literature with different application scopes is disregarded here.

Definition
The abundant definition as per previous research initiatives goes along the lines of DTs being digital reconstructions of real-life physical systems that mimic the behavior of the systems and their integral components through real-time linkage with the systems.So far, the definition above resembles closely that of simulation in the sense that they are both virtual representations of physical entities.However, the part on real-time linkage bears huge importance in defining the distinction as explained in Semeraro et al. (2021).A DT is synched to the physical system -which is a city in our case -in the sense that the status of the twin always corresponds to the current real-life status, and is updated once the real-life status changes.The status of the twin should not depict anything that is not happening in real-time.For instance, traffic jams should not be depicted by the virtual model when traffic is not building up in reality, even when empirical traffic data implies otherwise.Nonetheless, the DT may visualize phenomena that are not currently happening in real-life through its simulation tools as will be explained below.
The distinction between the simulated visualizations and the status of the virtual model, however, ought to remain clear.
In particular, Semeraro et al. (2021) states that "The Digital Twin should evolve synchronously with the real system along its whole life cycle", thereby modifying its initial configuration to adapt to the current situation.That said, the DT does not only update its status but also the quantitative models it embeds.The autonomous procedure by which it does will be stressed below.The targeted outcome of this procedure is that the modelling accuracy of the DT steadily improves.
Returning to the distinction from simulation, the DT's dynamic replication of a physical system resembles emulation more closely as Semeraro et al. (2021) explains.The DT ensures that the integral components of the system it models are accurately represented through the data it collects about them rather than simply model the general behavior of the system.The increased level of representation offered by emulation serves a considerable benefit as it provides a closer replication of reality in contrast to the more static concept of simulation.
DTs also control the physical system through data that they transfer to it.A unidirectional data flow -from the physical to the virtual model -is not sufficient for a virtual model to be labelled a DT.Semeraro et al. (2021) state that the interaction with the physical system should be bidirectional as data collected from the physical space updates the virtual model, while the physical twin improves its operational performance by exploiting knowledge acquired from the virtual model's processing of the data.For instance, an analytic insight generated from the DT regarding traffic should be communicated to the logistic planners/traffic controllers to assist them with decision-making.This may not be the case with a simulation model, since it is not linked to the physical system and thus, the analysis it produces need not be relevant for the current situation.
To conclude their findings, Semeraro et al. (2021) provide the following definition of a DT: "A set of adaptive models that emulate the behavior of a physical system in a virtual system getting real time data to update itself along its life cycle.The digital twin replicates the physical system to predict failures and opportunities for changing, to prescribe real-time actions for optimizing and/or mitigating unexpected events observing and evaluating the operating profile system".The definition above is not specific to urban logistics indeed.Nonetheless, we feel that the literature is in pressing need for an established basis upon which other features of DTs could be built for different applications.

Technical Anatomy
From a software perspective, there are many components in a DT.Generally speaking, the literature refers to components such as Internet of Things (IOT), cloud-computing and Application Programming Interfaces (APIs) such as in Moshood et al. (2021).The software engineering aspect, however, is of minor interest to us.Instead, we adopt a conceptual approach explaining how AI capabilities could aid real-time decision making in an urban setting when integrated in the DT framework.For a detailed discussion on how software is integrated into the DT framework, we refer to Botín-Sanabria et al. (2022).
From a technical standpoint, Belfadel et al. (2021) propose a framework into the anatomy of twins.We present an illustration based on their anatomy in Figure 1.They explain that a typical DT model is composed of the following hierarchies: the top-level hierarchy which is known as the Physical World, the intermediate level known as the Data and Model Management System and the bottom level known as the Storage System.
The Physical World represents the external physical entities and sensors such as city operational data, IoT entities and sensors, and APIs.This level is linked to the Data and Model Management System that is composed of two main systems: Data Ingestion System (DIS) and Model Management System (MMS).The DIS aims to integrate contextual data entities to the system and keep the DT updated about the status of the physical system.The MMS manages the models library which is a set of software applications that provide analytic tools.Most importantly, the MMS contains the Model Library Manager (MLM) which holds an overview of all available models in the DT, their specifications and their relationships with KPIs.The Storage System stores information on the management of model libraries and managed data along with other general storage as simulation scenarios, their configuration, related models and data etc.
Returning to the architecture described above, this is the task of the Decision System in the MMS which is responsible for assessing scenarios in terms of how likely they may occur, examines the KPIs of the selected scenario and recommends the necessary interventions in the physical world through the DIS to achieve the predicted outcome.

Functionalities
The main application area of DTs that is relevant for our study is smart cities.The introduction of DTs is supposed to help overcome many issues that urban models generally suffer from.Nochta et al. (2019) provide examples of these issues such as a degree of simplification of urban processes, shortages in data requirements, complications involved in data collection and inadequately handling human behavior and its implications, among others.
From a computational perspective, Marcucci et al. (2020) mention that DTs in smart cities "describe, capture and simulate policy (both real and potential) implications of alternative solutions for optimizing them with respect to a given objective or set of them".Schislyaeva & Kovalenko (2021) survey functionalities pertaining to DTs of logistic networks on the higher-level, emphasizing on how they can dynamically prescribe and optimize the urban physical system.They also discuss how DT simulation models could be used for stress tests while their predictive analytics tools can be used to predict the state of their physical counterpart.Gutierrez-Franco et al. (2021) state that DTs can predict possible future scenarios and evaluate them to find appropriate responses for the most likely ones.
These functionalities are of utter relevance to urban logistics.For instance, Big Data from traffic movements could be used to calculate travel times which can in turn be used to set-up optimization models for vehicle routing and configure simulation models to verify the proposed routing plans.
Moreover, the analytics embedded in the twin can be used to predict possible vehicle failures judging by their working circumstances or other internal diagnostics that would otherwise be unobserved.Schislyaeva & Kovalenko (2021) confirm this by saying that DTs retain data that can not be (easily) obtained from the physical model.Gutierrez-Franco et al. (2021) mention that DTs can raise alerts when exceptional situations are detected so that controllers can intervene.Anomaly detection could be pressing in some situations such as when a vehicle has been observed to remain stationary for a prolonged period of time.This could be the result of an accident in a distant location that would have otherwise remained undetected.Exception handling is especially relevant when physical assets exist in unsafe environments as Moshood et al. (2021) suggests, which improves the safety protocols.Not to mention the typical operational disruption consequent to accidents that could be efficiently managed by DTs where the difficulty of real-time re-planning is better addressed by real-time data connectivity and great computational power, improving the responsiveness of a Logistics Service Provider (LSP).This is because DTs can not only predict if a problem might happen, but also propose a solution.
DTs also encompass large sets of KPIs that are generated from its diagnostics.Gutierrez- (2021) explains,"The decisive factor is how this data is processed further in order to offer real added value.In this context, the added value is created with the help of KPIs tailored precisely to the targeted application".Additionally, KPIs play an even more important role in the learning process of a DT, something that will be extensively discussed below.
There is an additional benefit brought by the visual effects of a DT, and that is supply chain visibility.Moshood et al. (2021) explain that supply chain visibility depends on an organization's ability to be transparent and clear about its internal and external processes of its supply chain.
To that end, organizations have to determine the logistics operations that are most affected by lack of transparency and clarity, and devising techniques by which data could be easily exchanged between all participants.The visibility factor is not only important because of the enhanced interpretability it allows by visually depicting operations, but it also plays a collaborative part due to the involvement of multiple stakeholders along the supply chain who can be easily informed in a standardized manner of expected outcomes to any process.Moshood et al. (2021) confirm the importance of this by stating that it is essential to provide as much information as possible at the higher-degrees of strategic decision-making that can impact the supply chain as a whole.Marcucci et al. (2020) also stresses the importance of DTs in translating complex ideas into more intuitive ones through visualization.Moshood et al. (2021) reinforce the technological prowess of DTs by stressing how they could automate monotonous tasks that could be subject to human error, which is a benefit that could only come with its real-time connectivity.The advantages of automated processes are countless and can be explored in manifold applications.Furthermore, Schislyaeva & Kovalenko (2021) mention that it can troubleshoot remote equipment and perform remote maintenance as an extension of its automation capabilities.
The concept of DTs is normally coined with a self-learning feature where the virtual system consistently tries to improve its modelling of the physical system through the data it retrieves from it.Gutierrez-Franco et al. (2021) mention "a learning process based on the KPIs" process, where the modelling parameters of the DT can be calibrated by comparing the actual outcome of the operations with results from the simulation and optimization models.Therefore, DTs can learn from daily operations using machine learning models hat facilitate the acquisition and accumula-tion of knowledge from the urban environment.In turn, accumulated knowledge can be used to make predictions about the outcomes of future operations when data about these operations is not available.
In continuation of this learning framework, Kalaboukas et al. (2021) presents a study about how 'cognitive' DTs in agile supply chains increase their knowledge base as they learn more from data obtained from the physical model.They state that DTs should try to relate their predicted outcomes to the actual observed ones by learning how Unpredictable Desired (UD: desirable outcomes that were not predicted) and Unpredictable Undesired (UU: undesirable outcomes that were not predicted) events affect the physical counterpart.By employing this framework, DTs can enhance their learning capabilities over time as it offers a guideline to train and improve DT models through knowledge gathered from past UD and UU events.
To sum up, and as Nochta et al. (2019) explains, the introduction of DTs compliments the 4th industrial revolution where "moving from a period of relative data scarcity to an era of 'digital abundance' may enable" the generation of more accurate models based on real-time Big Data of higher quality that can describe urban logistics processes on a greater level of detail than before.

Set Up & Illustration
In principle, setting up a DT is a complex process.Moshood et al. (2021) states that is a relatively new area of research and that its precise implementations are scarce.This goes in synergy with the findings of Belfadel et al. (2021), implying "that existing architectures are too generic for usage in logistics".For smart cities, there have been several partial implementations such as Cambridge Kalaboukas et al. (2021), Lyon Belfadel et al. (2021) and cities in the Netherlands as we will explain below.Botín-Sanabria et al. ( 2022) also surveys other studies of smart city DTs in Asia and Europe, of which a subset is dedicated to urban logistics.Some research initiatives such as Ivanov et al. (2020) propose the concept of a DT of a city from a governance perspective, with limited focus on urban logistics that involves other stakeholders such as LSPs.
As an emphasis on the challenge associated with building DTs, Moshood et al. (2021) stresses that "a completely integrated Digital Twins is a long-term approach that does not happen immediately insisting that it will be long before it can be used by industry".Much of the difficulty is attributed to the intense technological requirements such as Internet of Things Sensors, Cloud computing etc.Consequently, Moshood et al. (2021) proposes to start simple and focus on maintaining the accuracy of data while incrementally reducing the chance of human error.Marcucci et al. (2020) provides an example of a collaborative initiative between policy-makers in the contest of Living-Labs.They suggest that Living-Labs are the most up-to-date data-driven methodology tackling the problem of managing urban logistics.The major idea is to involve all potential stakeholders in the urban logistics network in the design of the DT to agree on common objectives and functionalities.The concept is being tested in cities like Gothenburg, London and Rome.There, Living Labs are developed to create efficient and shared solutions among stakeholders.Moshood et al. (2021) also states that knowledge ought to be exchanged among the multiple stakeholders, a factor catalyzed by visibility.An extensive analysis on sociotechnical interaction among government, industry and consumers is conducted in Nochta et al. (2019), who argue that there is a need to look beyond technological factors and incorporate a distinct societal aspect into the design and implementation of DTs if they were to make any resonating changes in the practical modelling of urban environments.
Another important aspect regarding the design of a DT is the modeling one.Marcucci et al. (2020) also explains that DTs should strive to provide a simplified version of the physical model as they should never replicate the physical system in every detail, as that would not make them models anymore that we could use to efficiently study urban environments.The purpose of the DT as a model is to abstract the complex environment of a city in a limited number of variables.This sets many implications on what factors to include in the model.More variables could be included in the future as the requirements of its user base expand.
For the learning part, Kalaboukas et al. (2021) say that it is necessary to create a data set for which observed behaviors of the virtual model can be categorized among UP and UU events.This process, however, involves considerable data-manipulation and cleaning.For instance, they mention how "removing unpredicted and undesirable behaviors" is necessary when configuring a DT model to ensure that the training data is undisturbed.This is subsequent to experimenting with different scenarios and concluding on a particular desired behavioral model, so that only behaviors of interest are included in the design.
There are other less technical factors that should also be carefully contemplated when designing a DT.These could include legal issues regarding data-sharing.Since the DT is a broad model that encompasses multiple stakeholders, some with conflicting objectives, it is important to govern how data is shared and used such that each and every party knows exactly what it needs to know.This is discussed in detail in Kalaboukas et al. (2021).Working in environments with multiple stakeholders often also requires the introduction of common operational norms, rules and objectives.The DT needs to be aware of those rules that reflect the priorities, policies and other terms of collaboration among stakeholders.
All factors considered, Gutierrez-Franco et al. ( 2021) provide a methodology on the steps taken to set-up an urban logistics DT for an LSP. Figure 2 provides an illustration based on their methodology.The 6-step procedure starts with data collection.The data types collected should be dictated by the availability of data and interests of LSP using the DT.In Step 2, and after the data has been collected, it has to be suitably processed for a particular purpose such as devising diagnostic statistics that prescribe the operational context and can then be fed to a mathematical model.The design proposed in Figure 2 is specific to an LSP in a developing city.An urban logistics DT in general could be utilized by different stakeholders such as policy-makers and consumers in different cities which may or may more developed in line with the Living Lab approach presented above.This poses new implications on the design of the urban logistics DT.For instance, different data types could be collected in Step 1 depending on the requirements of the stakeholder(s).Additionally, LSPs may deal with different problems for which they have different approaches that deviate from the sequential procedure in Gutierrez-Franco et al. (2021).To that end, different stakeholders may require different 'versions' of the DT.This is adhered in Schislyaeva & Kovalenko (2021) which states that "One object can have more than one twin, with different models created for different users and use cases".Therefore, it is necessary to come up with an ontology upon which the general design of an urban logistics DT could be established to accommodate for all these possible variations.Consequently, we propose a conceptual model in Section 4 that discusses such an ontology and the integration of AI tools in the urban logistics DT.

Conceptual Model
In this section, we present our own conceptual model of DTs based on AI methodology.Our model is identified by three components, namely a knowledge base, machine learning and mathematical optimization.

Knowledge Base
Using the characterization in Section 2 and the set-up methodology of Gutierrez-Franco et al.
(2021) in Section 3.4, we are able to devise an ontology expressed by means of a knowledge graph.This ontology is similar to the one used in Anand et al. (2012) in terms of knowledge and relation representation.To specify the high-level entities, we use a mix of the five key factors in Section 2 and define several others based on the set-up in Figure 2. Specifically, we have the following entities: stakeholders, resources, KPIs, measures, decisions, data, statistical analysis tools, mathematical optimization, simulation and machine learning where the former five are discussed in Section 2, while each of the latter five corresponds to a step in Figure 2.
We are yet to elaborate on the latter five.We deem the Data entity to be self-explanatory as it represents any data collected by the DT through sensors for example, so we do not expand it further here.Specifically, the other four entities below that use the data represent relevant AI tools that can be used to solve urban logistics related-problems.Statistical analysis of data has been used in  Referring to the technical architecture in Section 3.2, the four AI entities would be embedded in the MLM.The MLM is an integral component of DTs as it embeds the quantitative tools that express its analytical capabilities.In turn, human controllers should consistently strive to improve its content so as to equip the DT sufficiently to analyze very complex processes and bring added value through optimization.In order to arrive at a compact design of the ontology, we merge these four into a single high-level entity which we refer to as the AI component.This is because their relationships with other entities in the ontology and among each other are more or less identical.
At the high-level, our ontology is given in Figure 3 where all entities are defined as well as their associated relations.The feedback-loop constituted by the bidirectional data exchange to optimize current operations and learn from past ones to improve future ones is given by the red arrows.In particular, data is collected from resources and used as input to the AI component.
The AI component processes the data to support decision-making through optimization.Once decisions have been implemented, KPIs are generated which are collected again as data to evaluate the decisions and learn from them.At a secondary level, the ontology of the AI component itself is given in Figure 4.The four components interact regularly with each other to support their functionalities.The output of one component can be used as the input to another, in contrast to the ordered pattern introduced in Figure 2. Figure 4 could, thus, be viewed as a generalization of Figure 2. It can also be viewed as an arbitrary pipeline of models.The same logic holds the other way round as any pipeline of these models could be categorized under the ontology of Figure 4. To demonstrate this, we provide an example below for solving a Vehicle Routing Problem (VRP) variant.We explain how a pipeline could be constructed from an algorithmic set-up, how it integrates into the framework of the DT and how the DT uses its capabilities to improve the AI models through its real-time connectivity.
However, we first provide a brief overview of previous research on how the aforementioned AI tools complement each other in decision-making for urban logistics -with a focus on routing operations -in the following section.Our focus on routing stems from our observation of the academic interest surrounding problems such as VRP and the importance of solving VRP variants for the daily operations of LSPs.

AI Pipeline Construction
There are some parts of the AI component that we pay little attention to.For instance, we consider statistical analysis to be a more traditional method where the computing powers of AI are less relevant.For simulation, we provide a brief discussion as it remains a relevant tool for digitaltwins for scenario-assessment especially.We are most interested in the combination of machinelearning and optimization as dictated by the concepts of data-driven optimization which we regard as the primary computing asset of AI.
Starting with simulation and optimization, a comprehensive overview of the different methods for different simulators is given in Amaran et al. (2016).Rabe et al. (2020) use simulation optimization to decide on the number and location of parcel lockers to which consumer goods are delivered.Muñoz-Villamizar et al. (2013) use simulation optimization to solve a complex Location Routing Problem (LRP).Applications in VRP are also abundant as shown in Tripathi et al. (2009) for the case with stochastic demands, and Pérez-Rodríguez & Hernández-Aguirre ( 2016) with fixed time-windows.
As for the dense combination of machine learning, mathematical optimization and simulation, there exists a few publications -such as Gutierrez-Franco et al. ( 2021) -containing all three methods.Rijnen et al. (2019) proposes an approach for urban trailer management where a machine learning model is used as a surrogate model of the simulator to evaluate simulations prematurely and reduce the computational burden while carrying out the optimization with a genetic algorithm.James et al. ( 2019) employ deep reinforcement learning with neural combinatorial optimization -based on a graph neural network -to solve an online VRP.Moreover, they verify the solutions using simulation.Although the simulation is not directly involved in the optimization procedure, it is still an integral tool in solution generation.
Advancing our focus to learning and optimization where we believe most research has been done and most AI potential lies, papers like Lombardi & Milano (2018) and Bengio et al. (2021) survey the general integration of machine learning into combinatorial optimization for different learning mechanisms.Mazyavkina et al. (2021) survey a more detailed application of reinforcement learning to solve combinatorial optimization problems.Khalil et al. (2017) extend on this by providing applications in graph related problems and providing a framework on how heuristics for these problems could be learned.
Historically, the application of machine-learning to optimize VRP has been rewarding.Bai et al. (2021) provide a comprehensive survey of machine learning applications in solving VRPs including stochastic variants.They consider the usage of machine learning as both, a modelling tool and an optimization one.More specifically, Niu et al. (2021) and Niu et al. (2022)   The aforementioned papers compose a small sample of the numerous literature on applying machine learning to solve VRP whose purpose is illustrative.Surely enough, there are other publications that address machine learning in VRP and would be of great use to a dynamic decision making system such as a DT.A summary of the papers cited above, the VRP variants they address and the employed learning and optimization methods is given in Table 1.
Given the approaches prescribed above, we can easily construct a pipeline based on the AI models employed.To illustrate that, we make use of an approach that solves a stochastic VRP variant.
We consider the stochasticity aspect due to its significance in characterizing urban environments where the uncertainty is due to the complex interactions between countless entities.We refer to this problem as SVRP1.
Let the pseudo-code for solving SVRP1 be given by Algorithm 1.It is worth mentioning that Algorithm 1 and the associated pipeline are examplifications whose purpose is simply illustrative.
It takes as input a set of customer locations N with properties such as time windows and priority levels given in set C N .Furthermore, let T represent the travel time data repository of the DT where each observation t ij corresponds to a travel time between locations i and j.The output would be a set of routes R * representing a feasible planning.
Algorithm 1 makes use of a heuristic with a machine learning component that constantly interacts with a mathematical optimization model such as when reinforcement learning is applied to a local search heuristic to approximate the value-function of actions; see R de O da Costa et al.
(2021).The problem state S is prescribed by the current and best obtained solutions respectively, whereas the associated actions concern k-echange moves.
With algorithm 1, the pipeline in Figure 5 can be constructed to solve SVRP1.The pipeline illustrates the order in which entities from the AI component in figure 4 interact with one another to output a solution x = R.The ontology prescribed in Figure 4 provides a fundamental design for pipelines using the AI component.Since all relations among all entities in Figure 4 are bidirectional, any possible ordering of the entities in a pipeline is allowed.Furthermore, other pipelines using different approaches may omit some of the entities if they do not use them at all.
3: Determine distribution of travel times t ij through preprocessing of T using statistical analysis tools.4: Set up stochastic optimization model SVRP1 5: while termination criterion for optimization is not met do 6: Apply search heuristic H that has a machine learning component.

7:
Generate candidate solutions R and add them to R.

DT Integration with AI Methods
The pipeline prescribed by Figure 5 provide several important implications.Firstly, it is primarily dependent on the input parameters T , N and C N .That means that as soon as there is a significant change in any of these parameters, the output of the pipeline -which is a planning verified by simulation -may very likely cease to be relevant from an optimality perspective.Secondly, it makes use of previous knowledge as specified by previously collected data on the travel times t ij and the machine learning component which has been subjected to training using a data-set linking operational decisions with KPIs.Lastly, it outputs a feasible operational planning.

Model Specification
With regards to the first observation, the real-time connectivity of the system is crucial.Data necessary for real-time decision-making can be collected by the DT.To illustrate this, we present the following terms.Let P (t) be a high-dimensional vector representing the parameters of the underlying quantitative models of the DT at continuous time t.In that case, the model can be completely prescribed by {P (t), t > 0} at any arbitrary time t.Furthermore, let ∆ be some time interval after t so that D(∆) represents the data collected from the physical system in that interval.The virtual model evolution with the physical system can be characterized by the following  (1) where f (.) is a concatenation of vectors of functions responsible for updating the virtual model's parameters given some data intake and previous parameter estimates.For instance, f (.) could embody the series of machine learning algorithms in the DT.We elaborate more on this below.
The specification of the functions in f (.) along with the interval ∆ defines the control duty of the human expert in the DT where 'better' choices lead to better designs and better models.It is crucial to have a high data-updating frequency while taking latency into account as Marcucci et al. (2020) suggest.We emphasize more on the specifications of the function f (.) in Section 4.3.2.

Data-Driven Optimization
Here, we discuss the integration of DTs into AI pipelines to support decision-making for urban logistics.In particular, we refer to the pipeline in Figure 5 and the model specification in Section 4.3.1.For the technical aspect of integrating DTs with AI, we are not acquainted with many papers that address solving a particular-problem in urban logistics with the help of a DT except for Gutierrez-Franco et al. ( 2021) that deals with VRP on a high level.Other papers such as Xu et al.
(2022) address the detailed integration of DTs in Data-Driven Optimization (DDO), including the technical aspect, although for another application in crane-scheduling.Unfortunately, the methods do not extend naturally to other applications due to the complex structure of DTs, the physical systems they are coupled with and the relationships between them.
For our framework, we pay particular attention to the theory of machine learning and optimization as prescribed by the concepts of DDO to explain the integration process from a mathematical standpoint.Before we proceed, we propose an important categorization of DDO in a DT.We refer to DDO applications whose purpose is to amplify the virtual model's approximation of the physical system as descriptive.As for applications responsible for decision-making, we refer to those as prescriptive.
Starting with descriptive applications, the objective here is to mimic the physical system as much as possible.It is important to have an accurate representation of the physical system to guarantee correct data input when making decisions.In the example of SVRP1, having accurate estimates of the travel times between two points is important.Otherwise, certain routes may seem favorable by the optimization mechanism, while in reality they jeopardize operational efficiency.
If there is no standardized measure of the travel times, that means they have to be learned and estimated by means of some predictive model in the DT.
The utilization of predictive models to estimate variables in the physical system to provide a better representation in the virtual space has been discussed in papers like Wang et al. (2020).
There, the authors discuss how a supervised learning mechanism can be deployed in the context of Bayesian optimization to estimate air-flow rates in the DT of a data center.Although the application is quite distinct from urban logistics, the methodology is quiet suitable for the learning infrastructure of DTs given their reliance on surrogate models as Bárkányi et al. (2021) suggest and data storage capacity as Belfadel et al. (2021) explain.
In the general case, learning to estimate a continuous variable -such as travel times -is a regression problem that would be guided by some loss function of the form: aggregated over N data-points.Here, V (x n , ˜) w represents the output of the predictive model for the input features x n corresponding to the n th observation and learnable parameters w.Observe that w ∈ P (t).Lastly, V n is the actual target value for the n th observation.The function L(.) could represent some deviation criterion like the square function.
For the DT, the N data points would be recalled from the storage system where all the data that has been collected in intervals ∆ -as specified in Section 4.3.1 -is being stored.The parameters w that characterize the predictive algorithm are optimized such that: The procedure by which the parameters w are optimized depends on the predictive algorithm used.
For instance, in the case of a neural network, an iterative procedure is used whereby w are updated by means of a function f 1d (.) ∈ f (.) that could assume a specification as follows: with α being a scaling parameter and ĝ( w(m) ) being an estimate of the gradient of (2) at w(m) ; the estimate of w at iteration m.The aforementioned specification of the update function f 1d (.) is known as gradient descent and is one of many (gradient-based) optimization techniques.
To realize how the DT's real-time data connectivity aids with training, we resort to the theory of gradient estimation for neural networks.From N data-points, a batch of size B is normally used to estimate ĝ( w) such as in:  For prescriptive ones, supervised learning generally works provided some consistency is guaranteed.For instance, if we are faced with a similar set of operational conditions, decision alternatives and KPI valuations, we could use experience gained from supervised learning to make decisions for current operations.In reality, this is seldom the case.And while consistency in operational conditions is unlikely in complex environments like cities, the decision alternatives are numerous and corresponding KPIs are unclear at first hand.
In the context of SVRP1, there are many possible routing options as dictated by the ordering of customers.Furthermore, it is not clear what the resulting KPIs are from each routing decision due to the associated stochasticity.In such cases, live interaction with the environment is needed to evaluate decisions produced by a local search algorithm such as k-exchange and observe the resulting KPIs.However, with logistic operations the high-cost sensitivity forbids the required trialand-error procedure induced by reinforcement learning to generate a training data-set.To counter this issue, Xu et al. (2022) propose what is currently known as offline reinforcement learning as an alternative.There, interaction with the environment occurs offline through simulation logic and standard reinforcement learning techniques could be applied again to train the algorithm before it could be deployed live for operational decision-making.The configuration for the simulation environment is determined by P (t) that embeds a live representation of the status of the physical system.That said, the concepts presented by equations ( 2), ( 3), ( 4), ( 5) and

Future Research
In analogy with the points discussed above, there are many possible benefits and challenges related to DTs.The most obvious benefit is its provision of a methodology to optimize the logistic network in a city.This would be reflected in reduced pollution and congestion volumes, more efficient logistics operations and increased consumer satisfaction through higher service levels.
However, there are many costs that ought to be borne beforehand.explain the reluctance of some companies to invest in testing DTs.Many LSPs may be unwilling to enable the DT to control their resources due to cost and safety concerns.The absence of a link by which the virtual model can control the physical model for testing purposes poses a serious challenge to the credibility of current studies on DTs.
To counter this, some platforms already provide basic implementations based on expert knowledge and collaboration with industry.The Atlas Leefbare Stad DT by Logistics Community Brabant LCB (2022), which is a virtual replica of cities in the Netherlands, is one such example.It models relevant variables as dictated by the requirements of its user base of academic researchers and LSPs.
In Atlas, however, the transfer of data is unidirectional -from the physical to the virtual system only, in contrast to definition from Section 3.1.Marcucci et al. (2020) refer to such a virtual model as a Digital Shadow (DS).
While a fully comprehensive study on DTs could not be met with a DS, a partial study is still feasible.Marcucci et al. (2020) mention that "the primary function a DT addresses is descriptive in nature".Examples of its descriptive functions include anomaly detection, warnings, predictive tasks and even recommending optimization-derived solutions without implementing them.By comparing its descriptive output with actual outcomes as interpreted by expert knowledge, experts can form opinions about the usefulness of the virtual model.
There are other challenges associated with building a DT.Marcucci et al. (2020) mentions "that technological changes and strong attention towards global warming" may require more "radical changes in technology and policy" than the incremental approach guiding the design and development of DTs.This places pressure on the benchmarks the DT is expected to meet as the correctness of a fully functional DT may be too slow to realize any convincing gains in the short-term.
Furthermore, Marcucci et al. (2020) explain that relationships between variables is expected to change over the course of time due to external factors .The DT model, therefore, compels constant updates so that changes in relationships and knowledge are incorporated on time, otherwise its added value may be questionable.
On the other hand, with past data, DTs can explain the possible underlying causes of encountered phenomena.This real-time management and control of situations aids with the integration of short-term decision making with long-term strategies as Nochta et al. (2019) suggests.Therefore, the DT would provide a more suitable framework to achieve the sustainability goals than other contemporary methodologies.That said, we aspire that future research expands on the ontology we proposed.

4.
Specify potential opportunities and challenges in future research of urban logistics DTs based on previous literature and our practical experience.That said, the rest of this article is organized as follows.Section 2 provides the characterization of urban logistics (Contribution 1).Section 3 discusses previous literature on DTs in urban logistics (Contribution 2) in terms of definition, technical requirements & capabilities and set-up methodology.Section 4 proposes the conceptual framework (Contribution 3).Section 5 lists the potential benefits and limitations (Contribution 4).

Figure 1 :
Figure 1: General technical anatomy of an urban logistics DT based on Belfadel et al. (2021).

Franco
et al. (2021) mentions that DTs can support accurate calculation of performance indicators of logistics operations through their scenario prediction and assessment mechanisms.Moshood et al. (2021) also mentions how DT sensors could be used to generate new data types, which as Gutierrez-Franco et al. (2021) suggests could create even new relevant KPIs.As Belfadel et al.

For
the latter purpose in Step 3, a mathematical optimization model could be set up by which decisions are made.The planning then has to be verified by means of a simulation in Step 4 in a more representative setting where the behavior of the city is better captured than in a simplified optimization model.Once the solution has been verified, it is set to be implemented in Step 5, with KPIs on its actual performance in the city being generated in real-time.The realized KPIs from Step 5 are compared with the estimated ones in Step 4. Major deviations are corrected for in Step 6 through configured learning processes such as reinforcement learning in order to ensure more accurate modelling and better decision-making in the future.

Figure 3 :
Figure 3: Our proposed high-level urban logistics DT ontology.

Figure 4 :
Figure 4: Our proposed ontology for the AI component in Figure 3.
the case with stochastic customers,Joe & Lau (2020) apply reinforcement learning to approximate the value-function of actions from a genetic algorithm.Other applications of machine learning in solving deterministic VRP can be found inMorabit et al. (2021) who use supervised learning for column generation to solve a VRP with time windows.Furian et al. (2021) combine supervised learning with a branch-and-price approach to predict the value of binary decision variables in the optimal solution of an instance, and the branching scores for fractional variables for capacitated VRP.Cooray & Rupasinghe (2017) consider a different heuristicbased approach where unspervised learning is used to tune the parameters of a genetic algorithm for energy minimizing VRP.Reinforcement learning, as a method, has been popular in the literature as well.R de O da Costa et al. (2021) provide a general methodology for reinforcement learning of a meta-heuristic for standard VRP with actor-critic networks.Nazari et al. (2018) employ reinforcement learning to devise a parameterized stochastic policy with an actor-critic network for optimizing capacitated VRP.They also explain that their approach could be extended to other variants.Similarly, Hottung & Tierney (2019) learn a stochastic policy by reinforcement learning for capacitated VRP and split delivery VRP through an actor-critic model.However, they employ a novel concept where the training targets are defined by the objective of an infeasible solution so as to bridge the gap with the (best) feasible solution.Zhao et al. (2020) also use reinforcement learning with an actor-critic network to devise a stochastic policy whose output can be combined with a local search procedure to optimize standard VRP and VRP with time windows.
8: end while 9: for solution R in R do 10: verify R using simulation 11: if simulation objective of R is better than R * then 12: R * = R. 13: end if 14: end for 15: return R *

P
(t + ∆) = f (D(∆), P (t)) 5) so that ĝ( w) can be seen as an average of estimates of the gradient at data points x b with parameters w represented by ĝ(x b , w).Standard training procedures select B random sample from the N samples in the data-set.However, if the algorithm is to be retrained in such a way so as to incorporate recent structural changes (if any), we might be tempted to estimate ĝ(.) using B samples from the most recently collected data in D(∆) during the last interval ∆ since the model was updated.Previous knowledge would be incorporated in the values of w(m) , which is in turn used -alongside D(∆) -to determine w(m+1) which incorporates new knowledge.

Figure 6
Figure 6 abstractly portrays this training process.Here, we assume a predictive model for the travel times in a city that is based on factors like current traffic, time of the day and previous travel time estimates -assuming a time-series correlation -among other factors.The predictive model need not to be retrained at fixed intervals of data collection, but if the controllers believe that new data embed significant structural change in the relationship between variables, knowledge about this change ought to be incorporated in the virtual model.Structural changes in relationships between variables are referred to as "concept drifts".Lu et al. (2018) investigate learning under concepts drifts.With supervised learning, training is generally done offline before the predictive algorithm can be deployed by the DT to estimate the necessary parameters used as input for the formulation of mathematical optimization problems.This suffices for descriptive DDO applications.

Figure 6 :
Figure 6: Abstract depiction of the estimation of ĝ(.) using a batch B from data collected.
Figure 6 extend themselves naturally to the case with reinforcement learning.The distinction lies in the input representation and relation to succeeding inputs, while the output evaluates the routing decisionthrough a reward -based on the KPIs generated from the simulation environment.The frequency of training is also important as reinforcement learning requires (re-)training after a fixed number of actions/steps has been taken, while in supervised learning, the training requirement is invariant to the number of actions taken as actions are often only produced once training has concluded.
For a start,Botín-Sanabria et al. (2022) cite data security concerns and communication networkrelated obstacles.Additionally, the set-up costs of the technology-intensive DT are not negligible.

Table 1 :
use hypothesis generation to learn a genetic algorithm to solve a multi-objective VRP with uncertain demand.For Categorization of the learning and optimization methods in a sample of the VRP literature.