The adoption of robotics in pack houses for fresh produce handling

ABSTRACT Fresh produce handling, particularly in final inspection and pack, is highly dependent on dextrous human labour. As part of a relatively low-profit margin industry, rising wage costs and labour shortages are impacting the viability of this sector and are a direct threat to global food security. Adoption of robotics is required to automate delicate handling tasks; this is a key goal for sustainable and profitable businesses that supply packed produce to consumers. This mini review considers the state of the art, as well as any developments required in robotic technology for the automation of inspection and packing of whole unprocessed fresh produce. There is a particular focus on robotic end effectors for the handling of fresh produce. We consider the role of soft robotics research in meeting hygiene and safety requirements, as well as the current limited range of end effectors for handling of highly varied and delicate produce types. Future directions are discussed based on the observation of available current technology available in research and the application to commercial practice.


Introduction
The fresh produce supply chain is a diverse global business distributing raw and processed product to consumers by retail and wholesale markets.In this review only whole, intact, fresh produce presented for sale by retail directly to consumers will be considered.The review will also only focus on the final stage of inspection and packing, which is primarily postbulk grading of produce.The reason for this focus is the particular dependence on human labour for this delicate handling and inspection step for a wide range of single, vine, and bunched produce.Rising wage costs and the inability to employ sufficient staff are a particularly acute problem in the UK (Rose & Bhattacharya, 2023), but one which also affects many other countries.One answer for a predominantly import business model is to pack more product at source, but mistakes and the breakdown of product in transit can be costly.Increasing the use of technology, such as robotics and automation to maintain control and drive down the costs of packing produce in the supply chain is a key developmental goal for the pack house sector (DEFRA, and Simon Pearson, 2022).This will ensure that sustainable and profitable businesses of the future can continue to supply safe and nutritious fresh produce to consumers.Significant exports of edible and non-edible plant products are sourced from southern Europe, equatorial and southern hemisphere countries; much of this produce is transported, packed, and sold in northern Europe and the UK.This supply chain creates significant logistical and technical challenges for handling highly perishable, fast-moving consumer goods.The focus of this review is to explore innovations in robotics and sensors that will increasingly be adopted in this critical added value step for the fresh produce supply chain.

Fresh produce handling
Fresh produce enters the supply chain journey at the point of harvest.Most produce at that stage has entered the ripening sequence and without careful management and handling, high levels of spoilage or waste can occur.Where supply chains have a cool chain, for example, then the supply chain loss is <4% but where there is no cool chain, losses can exceed 50% (Al-Dairi et al., 2022).At harvest, only produce free from defects is selected; product can be graded and packed on site into punnets or graded and bulk packed for onward grading and weighing.All products at harvest, and on arrival at pack houses, will undergo quality control and may require reworking to remove any defects that have developed during transport before they are handled for final inspection and packing (Al-Dairi et al., 2022).Defects and foreign objects present are specific to the commodity and in the UK must comply with legislation such as the Fresh Fruit and Vegetable Marketing Standards (DEFRA, 2022).
Because of fresh product scope, supply chains, storage, and shelf-life requirements are diverse.The differences in handling must be considered when approaching robotic systems for final inspection and packing.As an example of the complex differences, between otherwise mechanically similar produce, we can consider the supply chains of bunched table grapes, vine, and loose tomatoes.These fruit types have quite different ripening traits, grapes being nonclimacteric and tomatoes climacteric.Although grape berries are non-climacteric fruits and have low respiration rates, the grape pedicels and stalks behave in a climacteric manner having respiration rates 10 times that of the berries (Deng et al., 2007).Grape berries kept at 0°C can be stored for 60-120 days.Despite the complexities of tissue ageing, table grapes and tomatoes have quite different ripening profiles; grapes can be transported long distances at 0°C, in suitable packaging typically for 20 days and tomatoes, carried by, e.g.consolidated produce trailer, at 7°C for 1-4 days.Following transport, they have a similar shelf life for consumers from the point of sale, typically 7 days.Large volumes of table grapes and tomatoes are imported or transported significant distances within country from the field of origin.Invariably, the fruit needs to be inspected for defects that have manifested during transport.The proportion of defective fruits varies by batch and within or between seasons and it is standard practice to rework and present only class 1 fruit that is free of defects for final packing and direct retail.Fruit packed and labelled at this stage and dispatched is considered point of sale to the supply chain pre-retail.Picking and packing blemish-free produce into punnets to a specific weight and size is a part of the supply chain that has been designed for humans, as sight, touch, and dexterity are required.Automation requires an engineering solution, but the requirements are complex, and progress for many commodity types has been slow.Because of the delicate and precise nature of the work required at final inspection and pack, a process engineering-only solution has not emerged, and human labour is required to perform packing tasks with punnets such as topping up weight, reworking due to quality issues, and accurate product orientation for flow wrapping and labelling.The speed of these tasks is related to and ultimately constrained by the speed of human interaction.There is an initial need for robotics to match the precision and speed of the current processes, but there are significant potential benefits that automation can, in time, exceed these human levels of speed and precision (DEFRA, and Simon Pearson, 2022).The development and adoption of robotics also provides the capacity to build end-toend autonomous solutions for the pack house industry (Rose & Bhattacharya, 2023).

Key robotic system requirements and definitions
Robots have the potential to perform repetitive tasks accurately and at high speeds.To execute to commercially acceptable standards for pick and place, robotic systems comprise multiple sub-components such as a manipulator, end effectors, cameras, and other sensors for grasping but can be broadly divided into two major sub-systems: perception and manipulation [Figure 1].
Perception systems are used for individual produce recognition and recording metrics related to the produce.Over the last decade, there has been significant progress in the development of cameras, vision systems, and embedded sensory instrumentation which has contributed to the development of robotic capability.One of the most important improvements in these systems has been their detection capabilities, recognition, and localisation of three-dimensional objects in real working conditions (Czajewski & Kołomyjec, 2017).For fresh produce, the perception stage can involve recognising the individual fruits, segmenting separate bunches of fruit, disease detection, ripeness, as well as detecting defects or blemishes (Zujevs et al., 2015).Integration of both contact and noncontact chemical sensors may also allow for lifecycle monitoring with ripeness and quality assessment in the form of contactless machine olfaction (Manuela & Wilson, 2015;Wilson, 2013).In the last decade, deep learning techniques are also increasingly being implemented in many processes of object recognition, pose estimation and detection in agriculture (Andreas & Prenafeta-Boldú, 2018;Kusuma et al., 2019).
Object detection and property extraction use data for 2D and 3D recognition, modelling, and classification of individual units of fresh produce placed in front of vision systems, by assessing their colour, texture, and morphological features (Seema et al., 2015).Geometrical parameters such as length, breadth, and volume of produce may also be captured, which can be used in estimating weight and class for the finished product and informing decisions on packaging weight.
Object localisation is needed to calculate an object's cartesian co-ordinates with reference to the manipulator position, as well as the pose estimation needed for grasping.Several methods can be used such as monocular depth estimation or stereoscopic depth perception which can determine distances based upon triangulation of global cues (Masoumian et al., 2022).Localisation would also provide the punnet or packaging locations for targeting the placement of the produce.Localisation is strongly linked to the manipulator grasping action which is another major factor in the delicate handling and movement of fresh produce for packing operations.
Manipulation consists of path planning, grasping, and collision avoidance, and there are a variety of different industrial robot types and systems used for manipulation in pick and place operations.These include Robotic Arms, Cartesian robots, Delta robots, and SCARA robots (Granta Automation Ltd, 2018).A robotic arm for pick and place applications in one plane typically has 5 or 6 degrees of freedom (DOF) for complex operations.Cartesian robots move in three linear axes and are widely used for regular shaped and spaced objects and for these requirements, have a better position accuracy compared with a robotic arm (Bader & Rahimifard, 2020).Delta robots are formed with connected lightweight linkages, and they are used in many industries for fast picking of primarily regularly shaped objects.SCARA robots, also called 'fast pick' robots, have 4 DOF and have similar applications to Delta robots (Maw, 2018;Riyadi et al., 2017).
End Effectors are devices connected to the end of a robotic arm which directly interact with objects and the surrounding environment.Within the context of fresh produce handling, this would take the form of a fresh produce gripper or probe.Fresh produce can be difficult to handle because it is soft, perishable, and is irregularly shaped; multiple end-effector designs are required because of the diverse range of shapes and properties.

Collaborative robots
Pack houses which have been designed around human workstations may constrain robot operability, especially where large enclosures are required for safe operation at speed.However, collaborative robots (cobots) are designed for safer human-robot interaction in close proximity (Gualtieri et al., 2022).Increasingly, cobots are being used in production areas where a mixture of robots and humans can markedly improve productivity (Vasconez et al., 2019).Cobots are being increasingly used in pick and place, assembly, and material handling to automate simple repetitive tasks alongside human workers (Javaid et al., 2021;Simões et al., 2020).There are some examples of cobot usage within the food and catering industries such as those reported by Accorsi et al. (2019) and Emmanouil et al. (2023).However, few examples of human-robot interaction in agricultural produce handling (Contreras & Florez, 2022); a possible reason for this is the developed process engineering techniques for simple tasks within pack houses and complexity and requirements of remaining human tasks for automation.Avoiding any potential collision with human workers is of critical importance for safe operation; cobots have built-in safety features which can adjust the robot speed or stop based on the detection of unexpected objects within the path of the arm and end-effector (Chemweno et al., 2020).An example is 'safe skin' which allows for dynamic perception of any intrusive entity within a certain range of each part of a robot arm, and the skin can be tuned up to 15 cm with a response time of 0.01 s and an emergency stop time of 0.1 s (Pang et al., 2018).Currently, collaborative robotic arms with 6-7 DOF can provide position accuracy at speed but can prove to be highly expensive to implement.Commercially available cobots include solutions for path planning and collision avoidance to an acceptable range, but they are often unsuitable for complex challenges such as trimming and removal of defective articles amongst irregularly shaped and delicate packaged fresh produce (Emmanouil et al., 2023).

Motion planning
Motion planning is a crucial aspect of handling fresh produce as delicate objects are likely to be grouped in crates or punnets without uniform orientation.It is important that manipulators move around this uncertain environment, without damage to untargeted produce in the same area or container.It is also required that the speed of the operation itself does not impact highly delicate produce.Motion/path planning must account for produce integrity and the requirement for some objects to maintain a particular orientation such as table grapes and vine tomatoes.Motion planning is highly dependent on the chosen manipulator, end effector, and the technique adopted for the grasping (Farber, 2017).Developments in sampling-based motion planning (Elbanhawi & Simic, 2014) and the use of deep learning in motion planning (L.Chen et al., 2022) can be applied in the fast handling of delicate produce that will often be presented in uncertain conditions such as unorganised crates or punnets.Algorithms for reaching goal positions efficiently while avoiding the obstacles present in a robot's environment are being built upon with varied techniques (Llopis-Albert et al., 2018).Additionally, the need to understand the orientation of the items of produce at rest, which are to be targeted and transported during a pick and place operation, is known as 'pose estimation' (Joffe et al., 2019).It is recognised in (Joffe et al., 2019) that natural fresh products may have high variability and deformability, and that assumptions made in pose estimation must be adapted.Efforts to model the orientation of non-uniform fruits and vegetables are developing for harvesting applications such as with peppers (Hao et al. 2018;Eizentals & Oka, 2016) and with apples (Sarabu et al., 2019).

Object grasping
The most difficult challenge in pick and place for delicate fresh produce is achieving a precise, yet delicate grasp.The grasping force needs to be low enough to avoid object damage and impacting shelf life, but it must also be sufficient to avoid slippage and to complete pick and place movements at a high speed without dropping the object.There are a wide range of diverse, vegetables, fruit, and fungi shapes; therefore, handling can involve either direct contact with the surface of an article or manipulation of a pedicel or vine if applicable.Particular consideration of this has been applied in the context of harvesting (Navas et al., 2021).Studies that focus on the optimal handling strategies for fruit and vegetables reveal a large variety of proposed approaches and technologies but few developed solutions (Wang et al., 2022;Zhang et al., 2020).
There are multiple factors which affect grasping success as well as methods to improve grasping.Firstly, using perception, planning, and visual sensing abilities, a robot can identify and visualise grasps available for an object based on its orientation.Several methods of reinforcement learning (RL) are being used for pick and place of unknown objects to improve grasp selection and accuracy (Lobbezoo et al., 2021).Approaches such as (Gualtieri, Pas and Platt, 2018) use deep reinforcement learning (DRL) to solve pick-and-place and re-grasping problems where the exact object geometry is unknown.Deep learning is being used broadly to improve the performance of robotic grasp detection with several approaches (Caldera et al., 2018;Lenz et al., 2015).
Sensors also play a vital role in informing the grasping of delicate objects.Methods of slip detection and control in the handling of fruits and vegetables include use of a piezo resistor-based slip sensor in grasping of apples and tomatoes (Tian et al., 2018) and a combination of visual sensing and force sensing to improve grasping of tomatoes and sweet peppers in Xie et al. (2020).Many studies lack shelf life testing and evaluation of produce damage or grasp damage rate.Chen et al. (2022) recorded the damage rate for apples picked with and without a force feedback system with notable improvement from 20% damaged produce to 0%.Another method of improving grasping success for irregularly shaped, delicate objects is to use compliant or hybrid-soft grippers.Using a gripper with soft materials such as a suction cup or soft elastomer fingers can vastly reduce the need for control accuracy in grasping because compliant, deformable grippers can be thought of as having a built-in level of tolerance (Shintake et al., 2018).Some of the necessary grasping motions and force limits can be included in natural dynamics of an 'underactuated' gripper design.Reducing the requirements of control with mechanical design is known as 'embodied intelligence'; generally, this concept allows for greater diversity in objects being picked up without changing of the end effector (Cianchetti, 2021;Wang et al., 2020).

Robotic end effectors
The lack of suitable end effectors is considered one of the main factors that hinders the rapid introduction of robots into the fresh produce sector (Wang et al., 2022).Any end effector must be able to pick up delicate produce without impacting the quality and shelf life of the product, as well as being hygienic and food safe (European Hygienic Engineering & Design Group, 2018).They must contain as few mechanical components as possible to avoid sharp contamination in often punneted and sealed products (Fantoni et al., 2014).In addition, end effectors must also adapt to the fruit topography which can include wet and sticky surfaces.Where produce is bunched or on a vine, the ability to support the whole product is important to avoid loss of individual articles during any pick and place movements.There are many different types of robotic gripper and varied approaches and technologies for gripping in agriculture (Bader & Rahimifard, 2020;Zhang et al., 2020).Examples range from finger and pincer-based grippers to vacuum suction cups and other soft mechanisms [Figure 2].For the handling of fresh produce, the grippers proposed and tested are as diverse as the wide range of targeted fruits and vegetables (Carlos et al., 2011;Lien, 2012).
Traditional approaches to pick and place of delicate objects often involve adding soft material pads to pincer and finger grippers such as Russo et al. (2017) [Figure 2(a)] or using vacuum suction cups (Morales et al., 2014) [Figure 2(e)].Suction cup grippers such as Morales et al. (2014) are very popular in pick and place applications with benefits such as mechanical simplicity for hygiene and cleaning.However, suction cups may be limited to fresh produce due to requirements for target surfaces to be relatively flat, smooth, and dry.They may also leave a bruised defect on softer produce surface after handling (Wang et al., 2022).There is also a tendency in many studies, to test grippers with fresh produce that have similar properties of tough, relatively flat and smooth skin such as tomatoes, bell peppers, apples, oranges, and potatoes.In Koivikko et al. (2021), a suction cup gripper is improved with controllable magnetorheological fluid to better adhere to different shaped objects such as bananas and mangos.Some studies also combine grasping and suction cups to mitigate these effects when appropriate (Sam & Nefti, 2010;Wang et al., 2020) [Figures 3(e) and 2(h)].A particular challenge for fresh produce end effectors is hygiene and potential for transferring of disease between articles of produce (European Hygienic Engineering & Design Group, 2018).Robotics designed for food handling industry focus on system simplicity for ease of cleaning and minimising areas for bacterial propagation (Wang et al., 2020).In Pettersson et al. (2011) [Figure 2(g)], we see an extreme example of mechanical simplicity and design for hygiene safety.Fast switching or hot-swapping of gripper tooling is another option for maintaining packing line operation whilst regularly cleaning end effectors.
The end effectors currently used or proposed in areas of fresh produce handling are increasingly using soft robotic concepts such as compliant, deformable materials (Liu et al., 2018) [Figure 2(c)], as well as alternative soft methods of actuation such as pneumatics (Blanes et al., 2014) [Figure 2(f)] and magnetorheological fluids (fluids which change properties when a magnetic field is applied) (Tsugami et al., 2018) [Figure 3(g)].Interacting with fragile and complex shaped objects without applying excessive contact forces is a challenge for traditional rigid robot grippers.To address this, soft robotic grippers have been proposed in agriculture for their abilities to apply small forces and for conforming to complex nonuniform objects passively and easily (Chauhan et al., 2022;Elfferich et al., 2022;Navas et al., 2021).Soft robotics is a growing branch of robotics which specifically utilises the properties of soft materials and  (Gafer et al., 2020) Quad spatula gripper tested with plum tomatoes and olives.(c) (Liu et al., 2018) hybrid underactuated gripper tested on various fruit including grapefruit (d) (Blanes et al., 2015) pneumatic actuated gripper for assessment of eggplant firmness.(e) (Morales et al., 2014) suction cup for handling of onions and artichokes (f) (Blanes et al., 2014) 3D printed rigid finger gripper actuated with pneumatics tested with potatoes.(g) (Pettersson et al., 2011) pincer gripper with hygienic design for carrots/tomatoes (h) (Sam & Nefti, 2010) Dual mode gripper with fingers and suction cup pictured with strawberry and raspberry.
structures in grippers, as well as methods of actuation with compliance and deformability (Hughes et al., 2016).PneuNets, also known as fluidic elastomer actuators (FEAs), are pneumatic actuators which use air pressure to inflate chambers inside deformable materials to achieve motion.For pick and place of fresh produce, PneuNets are extremely popular (Chen et al., 2018;Low et al., 2022;Park et al., 2019;Wang et al., 2020;Y) [Fig 3c, 3d, 3e, 3 h], all use three or four PneuNet fingers for gripping fruit.These structures are highly compliant and safe for delicate interaction as well as having extreme simplicity for hygiene due to mechanisms of actuation being located far away at the base of the manipulator arm (Wang et al., 2020).Other than PneuNets, pressure-based end effectors such as vacuum membrane grippers (Krahn et al., 2017;Li et al., 2019) [Fig 3a and 3b] achieve more conformity to produce than a traditional suction cup with thin compliant membrane structures.The grippers adapt to varied produce shapes such as pears and sweet peppers and use negative pressure mechanisms for jamming of the membranes and resulting grasping.Soft actuators using air pressure and vacuum-based systems are more likely to be fully deformable and compliant compared to traditional actuators using motors and mechanisms located within the end effector, and one exception is the work of Pettersson et al. (2010) which uses a novel deformable Bernoulli-principle gripper for adhesion to fresh produce shapes.However, although the Bernoulli-principle gripper could be considered soft robotic due to its actuation and deformable surface, the overall gripper is still rigid and could not safely interact with grouped produce.
Commercial examples of robotic grippers for delicate pick and place are becoming more common in the food industry with higher value and processed goods.As competition increases, the cost of such robotic systems will be driven downwards, and technology advances will make the initial investment required more worthwhile for pack house operation.The main challenge is whether these food grippers which are used with fresh produce types in specific structured packaging scenarios can be applied or adapted to the highly dextrous tasks beyond packaging, which are required for final inspection and weight adjusting.
Soft end effectors are rapidly emerging to cater to food products for safe handling and hygiene purposes.PneuNet-based grippers such as the mGrip® from Soft Robotics Inc (Whitesides, 2018) [Figure 4 (Krahn et al., 2017), soft membrane vacuum gripper for various produce types (b) (Li et al., 2019), vacuum origami gripper shown with an apple (c) (Low et al., 2021), Reconfigurable PneuNet gripper tested with various produce types (d) (Park et al., 2018), soft PneuNet gripper with rigid plastic spacers tested with apples.(e) (Wang et al., 2020), Dual mode soft gripper with PneuNet fingers and suction cups tested with an Orange.(f) (Pettersson et al., 2010), Bernoulli effect-based gripper tested with an apple.(g) (Tsugami et al., 2017) systems for different shapes such as the Gripwiq range of SofTouch Suction grippers [Figure 4(h)] which includes many configurations for pick and place of small and delicate, to heavy and robust food items.Another Piab product is the piSOFTGRIP® vacuum actuated finger gripper, [Figure 4(c)] it is made with one piece of silicone rubber for increased food safety and is pictured in a lime packaging scenario.
Although moving and sorting fresh produce is the main fresh produce handling task for end effectors, another activity requiring automation is manual firmness assessment of produce, used by experienced human operators to assess ripeness and to reject defective produce.Efforts to specifically gather this data with end effectors include eggplant and cherimoya firmness (Blanes et al., 2015;Ortiz et al., 2019) [Figure 2(d)], and assessment of mango firmness and ripeness in Carlos et al. (2015).

Conclusion
The fresh produce pack house sector is labour intensive but has many opportunities for automation.There are numerous opportunities to automate beyond the production line engineering, but there are many complexities and challenges for robotics to replace the dextrous and intricate tasks beyond pick and place that humans currently perform.It is ultimately these tasks, such as adjustment of punnet weights, and the inspection and removal of spoilt produce which remain as challenges for automation.Product lines have been developed around human processing of fresh produce.Robotics have been introduced for stand-alone parts of the process, but pack house automation requires integrated systems for end-to-end solutions that will create fully autonomous lines, with the potential to exceed the capabilities of lines involving humans.
For many product types, the innovation in soft handling end effectors will be critical for end-to-end solutions that can safely pack and remove fruit to precise specifications and adjust punnet weights, without 'giveaway' or the inclusion of excess fruit.There is much commercial and pre-commercial development of soft end effectors for pick and place of food items, but the current barriers to implementation for finalstage inspection and pack include the extra level of control and dexterity required and the investment cost of such robotics.
Advancements in vision and AI technologies such as deep learning will also be crucial for the fresh produce sector for recognising spoilt, defective, and damaged produce at the same level as human experts in the automation of inspection and quality control.Deep learning is a rapidly developing area of research, and the improvements brought to perception systems will also improve the ability to grasp produce more safely, as well as requirements for packaging and defining the orientation of produce quickly.Currently, there is considerable investment and activity towards the automation of harvesting fresh produce (Zhou et al., 2022).However, postharvest fresh produce handling and packaging tasks are potentially more straightforward to automate, because of the more structured nature of pack houses, punnets, and conveyers in comparison with the unstructured, complex environments represented by polytunnels, glass houses, and farms.Ultimately, the potential gains often promoted in automated harvesting research, such as 24-hour picking of fruit, cannot be fully realised without equal automation of pack houses and produce inspection.

Figure 1 .
Figure1.The core robotic system requirements needed for effective fresh produce handling.