The algorithm “understands” how the process of cooking is arranged.
Researchers at the Massachusetts Institute of Technology (MIT) created thePizzaGAN neural network for making pizza. Based on the image of the dish, the algorithm is able to “understand” the order in which to add the ingredients in order to prepare the same pizza.
To do this, the neural network conducts “reverse cooking” pizza. The algorithm decomposes the image into separate layers, highlighting the elements of the filling, for example, mushrooms or meat, and gives a step-by-step recipe of the dish.
To train the pizza neural network, the researchers used only at the first stage 5.5 thousand images of pizza in the style of clip art. According to scientists, it saved time and made it possible to separate the filling from the pizza base. After that, the team added to the data set 9.2 thousand images of real pizzas collected on Instagram, and 12 different fillings, including bacon, broccoli, corn, mushrooms, olives and more.
As a result, the neural network learned to “understand” pizza. The algorithm can show pizza images before and after cooking, as well as the result of adding or removing certain ingredients. For example, you can show a neural network a snapshot of a dish and ask for mushrooms and onions to be removed — the system will show what a modified pizza looks like.
PizzaGAN so far works with only one type of food, but can bring the machine closer to cooking in the real world. The system is trained to collect “tasty” pizza and, in theory, is able to do it from beginning to end.
The researchers also concluded that the models used in PizzaGAN might be useful in other areas. They expect that the algorithm will work with other food consisting of layers, for example, burgers, sandwiches and salads. In addition, the system can be used for digital style consultants who create virtual combinations of clothing layers.