Real World Bayesian Optimization Using Robots
to Clean Liquid Spills
Isobel Voysey1, Josie Hughes1, Thomas George Thuruthel1, Kieran Gilday1,
and Fumiya Iida1
Bio-Inspired Robotics Laboratory, Department of Engineering, the University of
Cambridge, UK {iv256,jaeh2,tg444,kg398,fi224}@cam.ac.uk
Abstract. Developing robots that can contribute to cleaning could have
a significant impact on the lives of many. Cleaning wet liquid spills is
a particularly challenging task for a robotic system, and has several
high impact applications. This is a hard task to physically model due
to the complex interactions between cleaning materials and the surface.
As such, to the authors’ knowledge there has been no prior work in
this area. A new method for finding optimal control parameters for the
cleaning of liquid spills is required by developing a robotic system which
iteratively learns to clean through physical experimentation. The robot
creates a liquid spill, cleans and assesses performance and uses Bayesian
optimization to find the optimal control parameters for a given size of liq-
uid spill. The automation process enabled the experiment to be repeated
more than 400 times over 20 hours to find the optimal wiping control
parameters for many different conditions. We then show that these so-
lutions can be extrapolated for different spill conditions. The optimized
control parameters showed reliable and accurate performances, which in
some cases, outperformed humans at the same task.
Keywords: Robotic cleaning · Control Optimization · Service Robotics.
1 Introduction
In the coming years service robots are expected to have significant impacts on
our daily lives [2]. The kitchen is one place where robotics has the potential to
make a significant impact and to improve quality of life for many individuals [20].
One particular kitchen task which is widely disliked is cleaning, yet it is key for
maintaining the cleanliness and food hygiene standards in a kitchen [4]. One of
the most challenging cleaning tasks is cleaning wet surfaces. Such surfaces are
also one of the most high risk zones of the kitchen with regards to the spread of
unwanted disease [6]. Such robots could improve users quality of life and assist
with enabling the elderly to live independently in their homes for longer [19].
In addition, the ability to clean liquid spills could enable robots to clear up
hazardous liquid waste, assisting in tasks such as nuclear decommissioning or in
high risk lab environments [17].
2 I. Voysey et al.
Fig. 1. Summary of the physical iterative experimentation approach to identify the
optimal control parameters for cleaning a given volume of liquid.
Cleaning has been a long term goal for many research projects due to the
challenge it poses for path planning, control strategies, manipulation, and sens-
ing [8]. Autonomous cleaning robots have been fairly successful in structured
environments when dealing with mess which can be modeled as solid parti-
cles [12]. In particular robot vacuum cleaners have been very successful and
widely adopted [16]. However, many approaches are designed specifically for a
single application like vacuuming or wiping, or for a specific cleaning tool [7].
Learning-based control strategies have been introduced as viable methods to
solve the complexities involved with the problem of robotic cleaning by manipu-
lation [10]. However, to make this a viable solution, the problem must be cleverly
formulated to make the learning tractable, safe, and general purpose. Learning
by imitation of humans has also been proposed for generating effective cleaning
trajectories [9], but has limited generality. To move towards developing robots
that can clean liquids,new approaches are required as the problem cannot be
easily modeled. The liquid on a surface is complex and secondly the interaction
between the liquid and the cleaning device (for example) a cleaning sponge is also
very hard to model. Whilst there has been some examples of cleaning liquids, for
example with window cleaning robots [11], there has been limited exploration of
the optimal cleaning trajectories for liquids.
Thus, to develop cleaning algorithms and address this complex physical prob-
lem, we propose a new approach for developing control solutions for the cleaning
of liquid spills. In this method, a robot has been developed which can physically
iterate over the problem, and monitor its own success, allowing the system to
learn and optimize the control parameters through physical experimentation of
the cleaning progress. This approach is summarized in Fig. 1 showing that for a
given volume of liquid the experimental process is fully automated. From this,
we can extract and extrapolate rules for cleaning liquid spills of different sizes.
This novel framework for developing the control approaches for cleaning liquid
spills is fully automated allowing a single cleaning experiment to be performed
in around 3 minutes. The learning and optimization process is shown to achieve
control solutions which achieve performance that matches or exceeds human
performance. In particular, in this paper we make the following contributions:
Learning to Clean 3
– A base algorithm for liquid cleaning, with adjustable parameters
– An autonomous experimental framework for continuous exploration and op-
timization of liquid cleaning trajectory
– Experimental evidence demonstrating the validity of the system, and show-
ing the ability to find controllers for different hardware solutions through
iterative physical experimentation
This work presents an original framework for robotic liquid cleaning, with the
methods given in Section II. Following this, details of the experimental setup are
given. The experimental results are given in Section IV, followed by a discussion
and conclusion on the wider role of this approach for the development of control
strategies for physical tasks which are hard to model.
2 Methods
As shown in Fig. 1, physical experimentation is used to obtain the relevant con-
trol parameters, an approach which has previously been applied to other control
problems [15, 14, 3]. A base cleaning algorithm is optimized using feedback which
is extracted automatically using computer vision. This section presents this it-
erative method.
2.1 Cleaning Algorithms
A template, parameterized cleaning path has been devised which is then opti-
mized through the iterative process. This parameterized path has been developed
from human intuition and experience of performing the cleaning action. Parame-
ters have been chosen which are believed to have the biggest influence on cleaning
behaviors. The path is parameterized by three key variables. Firstly the space
step, d, the distance between points on the edge of the liquid spill, measured
parallel to the major axis. Secondly, h, the height by which the sponge is com-
pressed. The final parameter, v, is the velocity of movement along the path. The
cleaning path is centered around the major axis of the spill which is identified
for every liquid spill. The robot end-effector is kept parallel to the long axis of
the liquid spill.
Fig. 2 demonstrates these parameters with the range of values considered is
given in Table 1. Thus, each cleaning attempt, ci, can be described by the vector
[ d, h, v]. Human intuition is necessary to develop a base cleaning algorithm to
make this a tractable problem. Although this does limit the search space, it still
allows for a wide variety of different cleaning strategies to emerge. The control
and parameters were chosen to be as generic as possible, however, there may
need to be some adaptation for liquids with a significantly different viscosity or
foaming properties.
2.2 Optimization
To identify the optimal cleaning parameters, c, for a given volume V of liquid spill
an optimization algorithm must be used. This task requires the optimization of a
4 I. Voysey et al.
Table 1. Parameters of the cleaning algorithm showing the minimum and maximum
values and also the parameters chosen for the general wipe.
Property Variable (%)
Min
Value
Max
Value
General
Wipe
Velocity (m/s) v 0.05 0.5 0.3
Space step (cm) d 1 10 5.5
Compression
height (cm)
h 0 3 1.5
Fig. 2. Diagram of path planning and parameterization of the path. Inset showing how
the compression height is defined.
complex dynamic system with environmental interactions which are challenging
to models. Thus, an optimization algorithm that does not rely on an accurate
physical model is needed [18]. Bayesian optimization is such a method and can
sample efficiently from black-box functions without requiring gradient informa-
tion. It is a method employed to minimize the number of iterations required
to find the maximum of an objective function f(θ), where θ is the parameter
vector. Thus the cleaning parameters to be next tested, cn+1, are determined by
the objective function, in this case the liquid removal performance, P :
cn+1 = argmax
c
P (c) (1)
We employ a typical Matern kernel (with smoothness parameter ς = 2.5) as
defined in [13]. The exploration parameter (κ) of the acquisition function has
been tuned based on the Gaussian process Upper Confidence Bound with a value
of κ = 80 used to favor exploration. The optimization was implemented using the
BayesianOptimization library for Python1. To ensure the optimization process
explored the full area, the optimization was initialized by testing points where
two parameters were held at the mid range and the other was at 25% or 75%
of the range. Performance was evaluated by comparing the initial area of liquid
on the surface to the area remaining after cleaning and taking the percentage
reduction in area.
1 https://github.com/fmfn/BayesianOptimization
Learning to Clean 5
Fig. 3. Vision pipeline. (a) Original image, (b) cropped image, (c) enhancing contrast,
(d) thresholding, (e) closing mask, (f) external contours selection.
2.3 Visual Identification of Liquids
This approach relies on the ability to correctly identify the location and area
of the liquid spill, but visually identifying liquids is a challenging problem due
to the reflection, refraction, and variability in presentation [5]. It is important
that the area is estimated accurately as the area evaluation is used to form the
acquisition function of the optimization process.
Fig. 3 shows the vision pipeline which has been developed to accurately
identify the area and location; it has the following stages:
– Cropping : The image is cropped to a predetermined region of the surface.
– Contrast enhancement : Contrast-limited adaptive histogram equalization is
applied to enhance the definition of the edge of the liquid.
– Thresholding : Adaptive mean thresholding with an inverted output is applied
to account for varying lighting conditions.
– Mask closing : Erosion followed by dilation is applied to close the mask. This
ensures a continuous contour running around the edge of the liquid.
– Contour selection: Only external contours are selected to account for the
hollow nature of the mask and are converted to area. Contours smaller than
40 square pixels (1.3 cm2) are discarded as noise.
Weak coffee is used in the experiments as the base liquid. Using this pipeline it
is possible to identify the area of liquid before cleaning, a0, and the area after
removal, ar. Thus the performance for each cleaning process can be determined:
pi =
(
1− ari
a0i
)
.100% (2)
The accuracy of the vision system has been tested for 20 liquid spills by com-
paring that found with the pipeline to the ground truth, which was found using
manual thresholding with the software ImageJ and a 30-centimeter ruler as a ref-
erence object [1]. The results (Fig. 4) show a high accuracy with approximately
less than 1% error.
6 I. Voysey et al.
Fig. 4. Accuracy of the vision system, showing the estimated accuracy against a ground
truth for a range of areas of liquid.
3 Experimental Setup
An experimental setup has been developed to automate the physical tests. This
allows for many experiments to be carried out and for conditions to be kept
constant, so successive experiments can be compared. The experimental setup is
shown in Fig. 5, with the experimental process detailed in Fig. 5 . Mounted on
the end-effector of a UR5 arm is a tube connected to a custom designed syringe
pump, which has limit switches which determine the volume of liquid that will be
dispensed. The pump is connected to a large liquid reservoir. After dispensing a
given volume of liquid, the overhead camera is used to identify the area of liquid
present on the kitchen work surface, a0i. Using the parameters chosen by the
optimizer, the robot carries out the cleaning process using the sponge mounted
on the end-effector.
After cleaning, the camera is used again to detect the amount of liquid re-
maining, ari, and hence evaluate the success of the cleaning process. To allow the
experiment to be repeated, the setup must then be returned to the original state.
To empty the sponge, a liquid removal system has been developed. The cleaning
device (in this case a sponge) is pressed onto the central disc of the liquid re-
moval setup, with the compression forcing liquid out of the material. The robot
end-effector tilts forwards and backwards to remove more liquid. When liquid
flows out of the material it drains through the mesh into the container beneath.
The dry sponge is then used to perform a full grid-wise sweep of the surface to
remove any liquid that may remain. The parameters were chosen manually for
this general wipe process based on trial and error (see Table 1 for values). To
ensure the surface was fully clean, this process was repeated twice. After wiping,
the sponge is emptied and the next experiment could begin.
This setup has been shown to be highly reliable at delivering a liquid spill of
a given volume and providing high reproducibility across tests. For a sequence
of tests (n=12) where 28 ml was deposited on the surface, the shape was consis-
tently circular and the area of the liquid patch averaged 496.4 cm2 (s=12.9 cm2).
Learning to Clean 7
Fig. 5. Left) The experimental setup in a kitchen setup, showing the key parts: the
syringe pump for dispensing liquid, the liquid removal system, sponge and liquid dis-
penser. Right) Flowchart of the full experimental procedure for obtaining optimal con-
trol parameters.
On average, it takes around 3 minutes for each experiment to be undertaken,
allowing the system to rapidly test and iterate.
4 Results
4.1 Optimization for a Fixed Volume of Liquid
The results of the automated optimization of cleaning parameters for a fixed
volume, in this case, 28 ml, are shown in Fig. 6. To contextualize the performance
of the system, the effectiveness of human cleaning and a naive algorithm was
measured and have been added for comparison. The human was subject to only
two constraints on cleaning: a maximum of 15 seconds to clean (the average
time of the robot movement) and that they could not remove liquid from the
sponge during cleaning. The naive baseline used a side to side wiping pattern
with non-optimized parameters using the general clean parameters specified in
Table 1, thus providing a non-optimized robot clean for comparison.
The results show that after 35 iterations (approximately 2 hours of experi-
ments) the optimal parameters are found. The velocity settles the quickest, and
thus successful cleaning at this volume is least sensitive to the velocity of the
path. The compression height, h, is the last variable to be optimized, to a height
of around 2 cm. The spacing of the path optimizes at a similar speed to com-
pression height with an optimal value of around 4 cm. The performance appears
to be sensitive to small changes in control parameter.
8 I. Voysey et al.
Fig. 6. Left)Bayesian optimization of the cleaning process for a liquid spill with a
volume of 28 ml. Showing the removal success and the exploration of the parameters.
Right) The performance of the optimization process for different volumes of liquid
throughout the optimization process.
The optimal performance that is found shows the potential to exceed that
of human performance. When cleaning, the human approaches this as an open
loop process and could return to unsuccessfully cleaned patches whereas the
robot is closed loop. To approach, and indeed exceed human performance shows
the effectiveness of this approach. It shows that the optimization process finds
specific control parameters with a degree of accuracy and precision in execution
which a human would struggle to identify and perform.
4.2 Optimal Control Parameters for Varying Volume Spills
The automated optimization process has been run on three additional volumes
of liquid to understand how the optimum control parameter varies for different
volumes of liquid. Fig. 6 shows the performance throughout the optimization
process for the four different volumes of liquid explored. Again, for each volume of
liquid tested, the human success rate and that of a naive algorithm are presented
for comparison. In general, the optimization process takes longer for the larger
volumes of liquid. For the largest spill it took over 70 experiments, which equates
to nearly 4 hours of experiments. In all cases the optimized performance exceeds
or shows near to human-level performance. However, the robot system output
performs humans better for larger volumes of liquid.
Fig. 7 compares the optimal control parameters for different size of liquid
spills. For larger spills, a larger space step provides better performance with an
Learning to Clean 9
Fig. 7. The optimal parameters that were found for different volumes of liquid spill,
and the overall performance that was achieved.
approximately linear relationship between liquid spill volume and space step. The
effect of velocity is more complex; for larger spills a higher velocity is found to be
optimal, but prior to this, decreasing velocity is found to improve performance.
The optimal compression height is found to be similar across all volumes of
liquid. In summary, these results show the complexity of the cleaning problem.
To test the sensitivity of the performance on the cleaning parameters on
different volumes, the optimal parameters for a given volume, cv,opt, were tested
on a range of other volumes of liquid spill. Fig. 8 shows the results; it can be
observed that there is a significant decrease in performance when the optimal
parameters are not used. Thus, using the optimal parameters for a given volume
of liquid is necessary to achieve optimal performance. This validates the necessity
of this process, and that the optimal parameters found for the different volumes
provide the best performance despite the lack of obvious trends in the selection
of the control parameters with respect to the volume of liquid.
4.3 Parameter Generalization
Using the optimal control parameters for different volumes of liquid, parameters
can be predicted for other volumes using regression. Linear regression was used
for the spacing size of the path, compression height is assumed to be a constant,
and cubic regression is used for the velocity of the sponge. Fig. 9 shows the
cleaning performance when parameters are chosen using regression in comparison
to those chosen through the optimization process. These are also compared to
cleaning using the parameters found for 35 ml.
Using this regression based approach to predict the optimal in most case,
exceeds human performance. The predicted parameters consistently out perform
the naive algorithm and that using the 35 ml parameters. In many cases, in
10 I. Voysey et al.
Fig. 8. The optimal parameters for a given volume of liquid and applied to other
volumes of liquid to test the sensitivity.
Fig. 9. Performance of the parameters predicted by regression for previously untested
volumes in comparison to the 35 ml parameters.
particular around the middle of the range of volumes tested, the performance
exceeded the human baseline considerably.
4.4 Comparison of Different Cleaning Methods
This approach can be shown to generalize to other cleaning approaches. The
cleaning process was also optimized for a J-cloth. Fig. 10 shows a comparison of
the optimization for a J-cloth and the sponge for a 28 ml liquid spill. The J-cloth
performance was found to be more sensitive to the input parameters, especially
the compression height (Fig. 10). However, the optimal performance is higher
than that of the sponge. This experiment demonstrates how this setup could be
used to allow comparison between different cleaning techniques, and to enable
the optimal technique to be found for a given volume of liquid.
Learning to Clean 11
Fig. 10. Optimization of two different cleaning materials (sponge and J-cloth) for a
liquid spill with a volume of 28 ml.
5 Conclusions
This approach of iterative finding the optimal control parameters for the complex
physical process of cleaning liquids has been shown to find parameters which
can in some cases outperform humans. By automating the process it has been
possible to carry out over 400 experiments to allow the optimal parameters to be
found for given volumes of liquid. The use of regression to generalize the results
found for a wider range of volumes was shown to work well, and enable this
approach to be practically useful on variety of different sized liquid spills. This
work focused on cleaning water-based liquid spills as it is both a widely occurring
liquid spill type and challenging cleaning tasks due to the low viscosity of water.
In future work, other liquids with varying properties should be investigated,
alongside different cleaning materials.
This approach of using robotics to automate the finding of optimal control
parameters has been shown to effective. Within the cleaning domain this could
be expanded to optimizing for different cleaning devices or for more complex
cleaning scenarios. It is also more widely applicable in scenarios which are chal-
lenging to model by developing continuous robotic exploration approaches.
Acknowledgment
This work was supported by BEKO PLC and Symphony Kitchens. We are es-
pecially thankful for the valuable inputs from Dr Graham Anderson and Dr
Natasha Conway.
References
1. Abra`moff, M.D., Magalha˜es, P.J., Ram, S.J.: Image processing with imagej. Bio-
photonics international 11(7), 36–42 (2004)
2. Alonso, I.G., Ferna´ndez, M., Maestre, J.M., Fuente, M.d.P.A.G.: Service robotics
within the digital home: applications and future prospects, vol. 53. Springer Science
& Business Media (2011)
3. Brodbeck, L., Hauser, S., Iida, F.: Morphological evolution of physical robots
through model-free phenotype development. PloS one 10(6), e0128444 (2015)
12 I. Voysey et al.
4. Bugmann, G., Copleston, S.N.: What can a personal robot do for you? In: Confer-
ence Towards Autonomous Robotic Systems. pp. 360–371. Springer (2011)
5. Eppel, S., Kachman, T.: Computer vision-based recognition of liquid surfaces and
phase boundaries in transparent vessels, with emphasis on chemistry applications.
arXiv preprint arXiv:1404.7174 (2014)
6. Flores, G.E., Bates, S.T., Caporaso, J.G., Lauber, C.L., Leff, J.W., Knight, R.,
Fierer, N.: Diversity, distribution and sources of bacteria in residential kitchens.
Environmental microbiology 15(2), 588–596 (2013)
7. Kabir, A.M., Langsfeld, J.D., Shriyam, S., Rachakonda, V.S., Zhuang, C., Kaipa,
K.N., Marvel, J., Gupta, S.K.: Planning algorithms for multi-setup multi-pass
robotic cleaning with oscillatory moving tools. In: 2016 IEEE International Confer-
ence on Automation Science and Engineering (CASE). pp. 751–757. IEEE (2016)
8. Kemp, C.C., Edsinger, A., Torres-Jara, E.: Challenges for robot manipulation in
human environments [grand challenges of robotics]. IEEE Robotics & Automation
Magazine 14(1), 20–29 (2007)
9. Kim, H., Lee, H., Chung, S., Kim, C.: User-centered approach to path plan-
ning of cleaning robots: analyzing user’s cleaning behavior. In: Proceedings of
the ACM/IEEE international conference on Human-robot interaction. pp. 373–
380 (2007)
10. Kim, J., Mishra, A.K., Limosani, R., Scafuro, M., Cauli, N., Santos-Victor, J.,
Mazzolai, B., Cavallo, F.: Control strategies for cleaning robots in domestic ap-
plications: A comprehensive review. International Journal of Advanced Robotic
Systems 16(4), 1729881419857432 (2019)
11. Miyake, T., Ishihara, H.: Mechanisms and basic properties of window cleaning
robot. In: Proceedings 2003 IEEE/ASME International Conference on Advanced
Intelligent Mechatronics (AIM 2003). vol. 2, pp. 1372–1377. IEEE (2003)
12. Prassler, E., Ritter, A., Schaeffer, C., Fiorini, P.: A short history of cleaning robots.
Autonomous Robots 9(3), 211–226 (2000)
13. Rasmussen, C.E.: Gaussian processes in machine learning. In: Summer School on
Machine Learning. pp. 63–71. Springer (2003)
14. Rosendo, A., von Atzigen, M., Iida, F.: The trade-off between morphology and
control in the co-optimized design of robots. PloS one 12(10), e0186107 (2017)
15. Saar, K.A., Giardina, F., Iida, F.: Model-free design optimization of a hopping
robot and its comparison with a human designer. IEEE Robotics and Automation
Letters 3(2), 1245–1251 (2018)
16. Sahin, H., Guvenc, L.: Household robotics: autonomous devices for vacuuming and
lawn mowing [applications of control]. IEEE Control Systems Magazine 27(2), 20–
96 (2007)
17. Samseth, J., Banford, A., Batandjieva-Metcalf, B., Cantone, M.C., Lietava, P.,
Peimani, H., Szilagyi, A.: Closing and decommissioning nuclear power reactors.
UNEP Year book pp. 35–49 (2012)
18. Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine
learning algorithms. In: Advances in neural information processing systems. pp.
2951–2959 (2012)
19. Van Rensbergen, G., Pacolet, J.: Instrumental activities of daily living (i-adl) trig-
ger an urgent request for nursing home admission. Archives of public health =
Archives belges de sante publique 70(1), 2 (2012)
20. Zachiotis, G.A., Andrikopoulos, G., Gornez, R., Nakamura, K., Nikolakopoulos,
G.: A survey on the application trends of home service robotics. In: 2018 IEEE
International Conference on Robotics and Biomimetics (ROBIO). pp. 1999–2006.
IEEE (2018)