A Digital Twin Demonstrator to enable flexible manufacturing with robotics: a process supervision case study

ABSTRACT Manufacturing companies are facing different challenges. On the one hand, production is moving from mass-production to mass-customization and personalization. Reconfigurable-, adaptive- and evolving-factories are necessary to achieve the required flexibility. On the other hand, technology must be integrated with human skills to assist the operators in supervising and maintaining manufacturing plants with growing complexity. In line with the Industry 4.0 paradigm, a Digital Twin Demonstrator is proposed to support the supervision activity of the operator in the context of flexible manufacturing with robotics. The supervision is achieved through a Human-Computer-Machine Interaction (HCMI) enabled with the Digital Twin technology. The suggested demonstrator is implemented and validated using a lab case study where it is demonstrated how the proposed HCMI interaction enables ‘close-to-real-time’ supervision of the manufacturing system in its self-adaptation to production and environmental changes.


Introduction
Nowadays, production is moving from mass-production to mass-customization and personalization (lot-size-one) (Hu, 2013). Manufacturing companies are facing fierce pressure to cope with rapidly changing market demands for high product variety, small-lots of (mass-) customized products, and quick delivery requirements (Mindas & Bednar, 2016). Furthermore, the economic sustainability of manufacturing companies is based on the combination of high-performance and high-quality products with cost-effective productivity. Therefore, reconfigurable-, adaptive-, and evolving-factories are necessary to achieve small-scale productions in an economically viable way (Stump & Badurdeen, 2012).
Along with economic sustainability, social sustainability is considered a current challenge in modern manufacturing. One of the problems addressed by social sustainability is the one of integrating human skills with technology (Madonna et al., 2019). The achievement of this objective imposes several postulates on the role of technology in future manufacturing, such as assisting plant operators in supervising and maintaining production plants with growing complexity (Dotoli et al., 2019).
Among the different Industry 4.0 technologies (Lu, 2017;Di Nardo, 2020), Digital Twin (DT) may be utilized to address the aforementioned challenges. In traditional simulation, the digital representation of an existing physical object does not use any form of automated data exchange between the physical object and the digital one. In a DT, the data flow between an existing physical object and a digital one is fully integrated in both directions (Kritzinger et al., 2018). In this way, the digital model is synchronized with the status of the physical object and the results of the simulation can be directly implemented for optimizing the physical object. Due to the enhanced reactivity with respect to traditional simulation, DT enables the use of the simulation for the deployment of flexible and reconfigurable production systems for the manufacturing of personalized products (Kousi et al., 2019). In addition, since Virtual Commissioning (VC) constitutes the basis for the DT of manufacturing systems (Lechler et al., 2019), the visualization capability of the VC simulation can be used for supporting the operators to supervise and maintain modern production plants (Redelinghuys et al., 2018). Due to the lack of works in the topic, the novelty presented in this article is the demonstration of the capability of the DT technology to support the supervision activity of the operator in the context of flexible manufacturing with robotics.
In flexible manufacturing, changeover activities must be minimized to efficiently manufacture a large variety of products (Blecker & Abdelkafi, 2006); e.g. activities to change parts, fixtures, tooling, equipment programming from one product to another, etc. In this context, the manufacturing system must be robust enough to: • Production changes: within a mass-customizing company, different variants of products must be manufactured. Therefore, the production process should automatically adapt to fulfil different production orders with the available resources by minimizing the changeover activities (Hernandez et al., 2020) • Environmental changes: in flexible manufacturing systems, small environmental changes and noise are common due to the production of different variants (Simpson et al., 1998). However, the system should not arrest the manufacturing operation or decrease its performance due to occurrence of these scenarios. Examples of environmental changes are products manufactured from resources with different dimensions, and resources positioned in different configurations with respect to their ideal ones (Minhas et al., 2011).
In the context of flexible manufacturing with robotics, a Digital Twin Demonstrator is proposed in this work to support the supervision activity of the operator in monitoring the adaption of the manufacturing system to production and environmental changes. The supervision is achieved through a Human-Computer-Machine Interaction enabled with the DT technology, and is intended to enable the operator to control the production process. Next, we show how the proposed demonstrator allows the 'close-to-real-time' supervision of a robotics manufacturing system that can manufacture products with several variants, and obtained from resources with different dimensions positioned in different configurations with respect to their ideal ones. The article is structured as follows: Section 2 describes a state-of-the-art analysis of related works, and the proposed demonstrator is introduced in Section 3. Section 4 implements the demonstrator to a lab case study. Obtained results are discussed in Section 5 and finally, Section 6 presents the conclusions and set the directions for future work.

State of the art
Modern operators are faced with a large variety of jobs and tasks ranging from specification and monitoring to verification of production strategies. As the most flexible entity in the manufacturing system, operators are considered a key factor for achieving flexible production . Different researches in Human-Machine Interaction have focused on the design and use of computer technology for facilitating the operator supervision of the manufacturing system (Gorecky et al., 2014). Within the Industry 4.0 domain, a recent analysis of the state-of-the-art in human-machine interaction is shown in Krupitzer et al. (2020). However, the analyzed works only deal with Human-to-Machine (H2M) and Machineto-Human (M2H) communication without considering the role of the simulation in this interaction. Whereas, the development of the DT technology has enhanced the research topic of integrating the simulation within the human-machine interaction (Lu et al., 2020).
In manufacturing, DT technology has been utilized in different application domains with different purposes (Barricelli et al., 2019;Cimino et al., 2019;Kritzinger et al., 2018). For instance, DT has been employed to 'handle the flexibility' of modern manufacturing systems (David et al., 2018;Toivonen et al., 2018;Um et al., 2017;Yao et al., 2018). However, these studies do not consider applications in the robotics field.
Different works can be found in the literature concerning DT in robotics (Qi et al., 2021). On the one hand, researchers have focused on the definition of DT architectures without considering the business advantage and added value of using the DT technology within the process life-cycle. For instance, Kousi et al. (2019) proposed a DT architecture using the Robot Operating Environment (ROS) simulation software. Kuts et al. (2019) developed an immersive robotics environment that integrates DT and Virtual Reality (VR). Tavares et al. (2017) presented a DT simulator for industrial work cells.
On the other hand, different use cases have been developed, showing the benefits of DT in robotics. Burghardt et al. (2020) used an immersive robotics environment that integrates DT and VR to achieve the automatic programming of industrial robots. In the energy industry, Pairet et al. (2019) utilized DT simulators for training and testing human-robot collaboration scenarios in offshore platforms.
Concerning the supervision of robotics applications, Martins et al. (2020) proposed the use of the ABB RobotStudio software combined with the OPC UA standard for supporting the design, commissioning and supervision through the DT technology. However, the literature lacks studies on the use of DT for supporting the supervision activity of the operator in the context of flexible manufacturing with robotics.
Given the above, in this article we propose a DT demonstrator for flexible manufacturing reactive to both production and environmental changes. An HCMI architecture enabled with the DT technology is used to support the operator to verify whether the robot control logic fulfils the production requirements of the production scenario.

Digital Twin Demonstrator
Here we propose a Digital Twin Demonstrator that seamlessly integrates the Human (i.e. the operator), the Computer (i.e. the DT simulation), and the Machine (i.e. the production plant). The proposed HCMI solution applies to any production environment characterized by small-lots of customized products manufactured from resources that may have different properties and configurations. Examples of properties may be different geometric shapes and dimensions, while configurations may be different locations and orientations. Figure 1 illustrates the proposed DT demonstrator for supporting the supervision activity of the operator in the context of flexible manufacturing.
The process starts with the input from the operator, which specifies the production order and the properties of the resources available for conducting the manufacturing operation. Then, selected sensors perceive the environment to identify the configuration of the resources available for the production. Using the information input from the operator and obtained from the sensors, the controller automatically 'self-adapts' its control logic. 'Production-adaption' is performed to fulfill the production order with the available resources, while 'environmental-adaption' is performed to adapt the control logic to the actual production scenario. Once the controller has completed the operation, a DT model is updated to replicate the configuration of the production scenario. Then, the DT model is simulated using the control logic resulting from the self-adaption operation. The DT simulation reproduces the results of the manufacturing activity and is visually supervised by the operator for the validation. The operator verifies the fulfillment of the product specifications and the viability of the operation within the production scenario.
The verification by the operator will lead to two possible scenarios. In the first scenario, the manufacturing operation is viable, and the manufactured products fulfill the specifications defined in the production order. If that is the case, the operator triggers the production. In the second scenario, the product specifications are not fulfilled, or the manufacturing operation is not viable. In this case, the self-adaptation process has failed, and a manual-adaptation of the control logic is necessary. The DT simulation model is saved, along with the control logic code, the production order, the available resources, and the environmental perception. Next, this information will be used by a software engineer for the design of a new control logic (code) aiming for the fulfillment of the product specifications. Once the software engineer completes the development of the new control logic, a new validation cycle will start.
The information regarding the occurred failure would be useful for the manualadaptation operation. However, how to identify and send this information from the operator to the software engineer will be a matter of future work.

Case study
In this section, the Digital Twin Demonstrator proposed in Section 3 is first instantiated and then applied to a case study for its validation. A Universal Robot 1 (UR) simulating a cutting operation is used for this purpose. Here, the robot must cut a rectangular plate positioned in an arbitrary configuration for generating the number of rectangular sheets required from the operator.
The instantiation of the DT Demonstrator is shown in Section 4.1. Section 4.2 describes the algorithm used for the production-adaption, while Section 4.3 the one for the environmental-adaption. Finally, Section 4.4 utilizes the implemented demonstrator for performing the cutting operation.

Instantiation of the Digital Twin Demonstrator
The developed Digital Twin Demonstrator is shown in Figure 2. An UR3 robot is used as production plant. Polyscope software is utilized for programming the robot and Polyscope User Interface (UI) for the Human-Machine Interface (HMI). Using the URCaps plugin, Polyscope invokes a Python script that contains the algorithms for self-adapting the control logic to fulfill the production order within the actual production scenario. The DT simulation environment is built by interfacing the software URSim with Experior.
Polyscope, Polyscope UI and URSim are provided by the UR vendor, whereas Experior is a VC and DT simulation software developed by Xcelgo A/S. 2 URSim is responsible for simulating the robot movement, while Experior mirrors the robot configuration built in URSim for reproducing its interaction within the manufacturing environment. It is necessary to include the interface of both software since URSim only simulates the robot movement but not a virtual manufacturing environment.
Hardware-in-the-Loop (HiL) commissioning is selected for the DT simulation since this commissioning interface a virtual plant model and a real controller (Lee & Park, 2014). HiL enables the implementation of a 'seamless' HCMI architecture, where seamless (interoperability) refers to the meaningful information exchange between human, computer, and machine. Here, the simulation model of the physical plant is interfaced with the real controller since the UR controller is connected to a Personal Computer (PC), which runs URSim and Experior (see Figure 2). The UR controller and the PC are interfaced with an Ethernet cable and communicate through the TCP/IP protocol. The Real-Time Data Exchange (RTDE) 3 interface is also applied since it works with a standard TCP/IP connection and synchronizes the PC with the UR controller without breaking any real-time properties.

Production-adaption algorithm
In the bin packing problem, items of different volumes must be packed into a finite number of bins or containers. Each bin has a fixed given volume and the number of utilized bins must be minimized. In computational complexity theory, bin packing is a combinatorial NP-hard problem (Korte & Vygen, 2012). The cutting problem investigated in this case study can be dealt as a two-dimensional bin packing rectangular problem.
In order to achieve a feasible, fast convergent although non-optimal solution, a heuristic is utilized that integrates the 'Shorter Side Descending' pre-sorting, the 'Best Long-Side fit' heuristic-based algorithm, and the 'Guillotine' algorithm (Bothwell, 2019). The details of the utilized algorithms are available to the reader. 4 The flowchart of the algorithm utilized for our 2D cutting problem is shown in Figure 3. The input of the algorithm consists of the production order and the resources available for conducting the manufacturing operation. In our case study, the available resource is a rectangular plate with length L and width B, and the production order is defined with sheets of given dimensions ½l j ; b j � for j ¼ 1; . . . ; n. The pre-sorting is first implemented and then the 'Best Long-Side fit' and 'Guillotine' algorithms are iterated until all the items have been assigned or no sufficient space is available on the plate for producing the next item. The output of the algorithm is a sequence of points in Cartesian coordinates indicating the locations of the corners of each item. Each point is referred to the lower-left corner of the plate as shown in Figure 4.

Environmental-adaption algorithm
The Cartesian points obtained from the algorithm illustrated in Section 4.2 are relative to a plate aligned with the x-y axis of the robot coordinate system. However, the plate can be positioned in any arbitrary configuration as shown in Figure 5. In this case study, we consider that the z-coordinate of the plate remains fixed. Few geometric transformations must be implemented to adapt the identified Cartesian points to the actual plate configuration.
The actual location and orientation of the plate are captured with the robot Wrist Camera. 5 A 1 × 6 'Object-location' vector is generated containing the plate center point C position with respect to the robot R coordinate system, while its orientation is defined with respect to the workspace W coordinate system; see Figure 5.
A vector transformation is implemented to describe the Cartesian points with respect to center point C (see Figure 4): where r CP i is the point P i position with respect to the plate center point C, r OP i is the point P i position with respect to point O, and r OC is the plate center point C position with respect to point O.
The points P i must be rotated with respect to the z-axis of the C coordinate system in order to adapt their orientation to the actual plate configuration. Given β the angle between the z-axis of the C coordinate system and the one of the workspace W coordinate system, the following rotational matrix is applied: Eventually, a further vector transformation is implemented to represent the points P i with respect to the robot R coordinate system: where r RP i is the point P i position with respect to point R, r RC is the plate center point C position with respect to point R, and r CP i is the point P i position with respect to the plate center point C.
Doing this, all the points are represented with respect to the robot R coordinate system. Thus, the cutting operation can now be performed.

2D cutting problem
In this section, we use the DT demonstrator for the resolution of the 2D cutting problem. The DT demonstrator implemented with the software selected in section 4.1 is illustrated in Figure 6. The production order and the available resources are inserted by the operator through the robot HMI. Then, the UR controller receives the input parameters and automatically calculates the corners of each sheet; i.e. production-adaption. After that, the camera perceives the environment and the control logic adapts the previously calculated points to the actual plate configuration; i.e. environmentaladaption. Using these points, the robot calculates the trajectory and a pop-up is displayed on the HMI through which the operator will indicate the outcome of the self-adaptation operation. Next, the Experior environment is updated to replicate the actual production scenario; i.e. the virtual plate is positioned on the same configuration of the physical plate. After that, the UR controller triggers the URSim simulation that moves the virtual robot using the trajectory generated in the UR controller. The robot movement is mirrored in Experior, and the manufacturing operation is simulated. Meanwhile, the operator inspects the DT simulation and then selects an option on the previously generated HMI pop-up. If the virtual production was successful, the operator presses the 'CONTINUE' button and the physical robot implements the calculated trajectory. Otherwise, the 'STOP PROGRAM' button is selected and the DT simulation model, as well as the control code, the production order, the available resources, and the environmental perception are automatically sent to the software engineer.
Given above, the proposed DT demonstrator is applied to a robot, simulating a cutting operation in order to be validated. The experimental setup is illustrated in Figure 7 and the initial Experior model is shown in Figure 8. A board marker is inserted in the hand effector of a UR3 robot and the cutting operation is simulated by writing on a whiteboard. Then, the following production scenario is implemented:   whose center point C is located at x ¼ 15cm and y ¼ 30cm with respect to the robot R coordinate system and rotated by À 30deg ðÀ 0:52radÞ with respect to the z-axis of the workpiece W coordinate system.

Results and discussions
In this section, the results of the application of the DT demonstrator to the 2D cutting problem are shown, and then its implications with respect to academy and industry are described.
A video illustrating the implemented HCMI interaction is available to the reader. 6 First, the operator inserted on the HMI the specifications of the available plate and the sheets to be produced; i.e. human-machine interaction. Then, the UR-controller automatically performed the production-adaption. The robot control logic utilized the algorithm illustrated in section 4.2 to calculate the Cartesian coordinates of the corners of each sheet. The result of the production-adaption can be seen in Figure 9. In this figure, 'Ideal plate configuration' indicates the configuration in which the plate is aligned with the axes of the robot R coordinate system.
Next, the camera perceived the environment and the environmental-adaption was automatically performed. Using the geometric transformations illustrated in section 4.3, the robot control logic converted the Cartesian coordinates of the previously calculated points to the actual plate configuration. The result of the environmental-adaption is illustrated in Figure 10. In this figure, 'Actual plate configuration' indicates the plate configuration in the physical production scenario.
Finally, the Experior model was updated moving the plate from the ideal (Figure 9) to the actual configuration ( Figure 10). A DT simulation was run, and the operator verified the self-generated control logic by inspecting the outcome of the virtual manufacturing operation; i.e. human-computer interaction. The result of the DT simulation is shown in Figure 11.
Since the manufacturing operation was viable in the given case and the manufactured products fulfilled the specifications defined in the production order, the operator pressed the 'CONTINUE' button on the HMI and triggered the production. Once the 'CONTINUE' button was pressed, real commissioning was implemented.
The following outcomes are generated from the implemented DT demonstrator by analyzing the results of the case study: • Supervision of flexible production: the demonstrator supports the supervision activity of the operator through the implementation of an HCMI interaction.  The proposed demonstrator can work in the context of flexible manufacturing since the use of the DT simulation enables the generation of a system reactive to both production and environmental changes; i.e. in the investigated case study, manufacture products with several variants, and obtained from resources with different dimensions positioned in different configurations with respect to their ideal ones • Close-to-real-time supervision: the proposed demonstrator allows to quickly evaluate the system adaptation to production and environmental changes. This functionality enables close-to-real-time supervision of the operator • Seamless communication: the 'master' actor that manages the implementation of the DT Demonstrator is the UR controller. In fact, it receives the inputs from the operator through the HMI, self-adapts the robot control logic to the production scenario, and triggers the DT simulation through its communication with the PC. Eventually, it makes the robot perform the production after the confirmation from the operator. It can be noticed that the involved actors seamlessly communicate without any extra implementation over-head.
Finally, the proposed demonstrator presents several potentials from an academic and industrial perspectives. In education, the demonstrator may be utilized as a learning factory (Abele et al. (2017)): (i) to teach to engineering students different algorithms related to flexible production and robotics; (ii) to train industrial operators. In research, the demonstrator may be adopted to investigate different approaches to tackle the flexibility challenges introduced from the mass customization paradigm. Furthermore, our demonstration of the DT capability to support the supervision activity of the operator in the context of flexible manufacturing with robotics may inspire other researchers to investigate additional supervision functionalities; e.g. remote supervision for hazard production plants, etc. In industry, the approach may be applied for supporting the operators in the supervision of flexible production systems, such as the illustrated robot for manufacturing products with several variants using resources with different dimensions and configurations. The main limitation of the approach in its industrial application is that the algorithms for the self-adaption to production and environmental changes must be customized on the basis of the considered manufacturing system and its production scenarios. However, the implemented demonstrator may be utilized as a testbench for selecting and tuning the algorithms before their scaling in the industrial environment.

Conclusions and future work
This article proposes a Digital Twin Demonstrator to face the challenge from the operators to supervise modern manufacturing systems which are characterized by growing complexity and flexibility. The demonstrator is designed in the context of flexible manufacturing with robotics and implements a Human-Computer-Machine Interaction enabled with the DT technology. This interaction supports the operator to verify whether the robot control logic fulfils the production requirements of the production scenario. The proposed demonstrator is validated in a case study of a robot simulating a cutting operation.
The obtained results confirmed how the proposed demonstrator allows for seamless interaction among the operator, the computer and the machine (i.e. the robot). Additionally, the demonstrator enables close-to-real-time supervision of the flexible manufacturing system in its self-adaptation to production and environmental changes; i.e. manufacture products with several variants, and obtained from resources with different dimensions positioned in different configurations with respect to their ideal ones.
Furthermore, the following future works are identified: • Enhanced environmental perception: in this work, the environmental perception was performed with a camera. Future work should integrate a force sensor placed on the robot end-effector to improve the environmental perception functionality; e.g. automatic identification of the plate z-coordinate and dimensions, etc. • Manual-adaptation: when 'self-adaptation' is not achieved, the information concerning the occurred failure should be sent to the software engineer, along with the DT simulation model, the control logic code, the production order, the available resources, and the environmental perception. A solution for sending this information from the operator to the software engineer should be investigated • Remote monitoring: in the presented demonstrator, the operator needs to be in the production plant to supervise the production system. The process of sending the results of the environment perception to a monitoring station is a topic of future work in order to enable the remote monitoring of the production process.