JP2021139775A - Mass estimation device and food packaging machine - Google Patents

Abstract

To provide a mass estimation device and the like which can estimate a mass accurately from an image.SOLUTION: A mass estimation device comprises: a position specification marker 42 which is provided along a conveyance direction of a conveyance unit 10; an imaging unit 20 which captures an image of an estimation object WK existing in a partial region of a conveyance surface 11 so as to include the position specification marker 42; a position specification unit 32 which specifies a prescribed region on the basis of the position specification marker 42 from the image including the position specification marker 42 imaged by the imaging unit 20; an image processing unit 31 which extracts the estimation object WK included in the prescribed region specified by the position specification unit 32 from the image including the position specification marker 42 imaged by the imaging unit 20; a mass estimation unit 33 which estimates a mass of the estimation object WK at a position specified by the position specification unit 32 with respect to the extraction image of the estimation object WK extracted by the image processing unit 31; and an output unit 34 which outputs the mass of the conveyed estimation object WK by accumulating the mass of the specific region for each region estimated by the mass estimation unit 33.SELECTED DRAWING: Figure 1

Description

The present invention relates to a mass estimation device using an image used in a food packaging machine or the like, and a food packaging machine using the mass estimation device.

When selling products such as foods, medicines, and parts, the mass is measured. For example, foods packaged in fixed quantities are in great demand, such as supermarkets, convenience stores, restaurants with chain stores such as family restaurants and beef bowls, and home delivery food services such as co-ops. Such packaging is performed in a food processing factory, but a food packaging machine is often used to wrap a large amount of food in a short time. Food packaging requires accurate packaging to a defined volume, or mass. Therefore, the food packaging machine is provided with a scale for measuring mass, and packaging is performed after weighing in advance so as to package with a predetermined mass. For example, in a packaging machine that transports and wraps products by a conveyor belt, an electronic scale such as a load cell is used to stop or decelerate the conveyor belt to perform bagging when the mass reaches a set value.

However, depending on the product to be packaged, weighing may not be performed smoothly. For example, when a product containing a large amount of water, such as cut vegetables, is placed on the load cell, a part of the product remains attached to the load cell. In this case, from the viewpoint of accurate mass measurement, hygiene, etc., it is necessary to clean the load cell every time the load cell is weighed, which causes a problem that it takes a lot of time and effort.

In addition, if the conveyor belt is temporarily stopped or decelerated each time weighing or cleaning is performed, bagging is hindered and the tact time is reduced. Especially at the production site, from the viewpoint of improving productivity, not only the accuracy of weighing but also the processing speed is required.

JP-A-2019-020343

The present invention has been made in view of such a background, and one of the objects thereof is to provide a mass estimation device and a food packaging machine capable of accurately estimating mass from an image.

Means for Solving Problems and Effects of Invention

According to the mass estimation device according to the first aspect of the present invention, it is a mass estimation device for partially estimating the mass of a large number of estimation objects scattered on the transport surface, and is distributed on the transport surface. A transport unit that transports the sprinkled estimated object, a position identification marker provided along the transport direction of the transport unit, and an estimated object that is fixed above the transport unit and transported by the transport unit. Among them, the position is obtained from an image pickup unit that captures an image of an estimated object existing in a part of the transport surface including the position identification marker and an image including the position identification marker captured by the imaging unit. From the position specifying unit that specifies a predetermined region based on the specific marker and the image including the position specifying marker imaged by the imaging unit, an estimation object included in the predetermined region specified by the position specifying unit is selected. An image processing unit to be extracted, a mass estimation unit that estimates the mass of the estimated object at the position specified by the position specifying unit with respect to the extracted image of the estimated object extracted by the image processing unit, and the mass estimation unit. An output unit that accumulates the mass of a specific region estimated by the unit and outputs the mass of the transported estimated object can be provided. With the above configuration, it is possible to estimate the mass of the estimated objects scattered on the transport surface. Further, since it is possible to specify in which region of the transport surface the mass is measured by the position specifying marker, it becomes easy to accumulate the partially measured mass and cut out the required amount of the estimated object.

Further, according to the mass estimation device according to the second side surface, in addition to the above configuration, the transport portion is a conveyor belt, and the position identification marker is provided along at least one end edge of the conveyor belt. It can be displayed on the marker display surface. With the above configuration, while the object is conveyed by the conveyor belt, the image of the estimated object is imaged by the imaging unit provided at a fixed position to sequentially estimate the mass, and the mass of the conveyed estimated object is accumulated. It will be possible to obtain it.

Further, according to the mass estimation device according to the third side surface, in addition to any of the above configurations, a wall portion provided between the transport surface of the transport portion and the marker display surface is further provided. Can be done. With the above configuration, it is possible to avoid a situation in which the estimated object loaded on the transport surface protrudes to the marker display surface and covers the position specifying marker.

Furthermore, according to the mass estimation device according to the fourth aspect, in addition to any of the above configurations, a display belt arranged adjacent to the conveyor belt and running in parallel with the conveyor belt is provided. The position identification marker can be displayed on the display belt. With the above configuration, by providing the display belt that runs in parallel with the conveyor belt of the transport unit, it becomes easy to provide the wall portion between the conveyor belt and the display belt.

Furthermore, according to the mass estimation device according to the fifth aspect, in addition to any of the above configurations, the color of the conveyor belt can be colored differently from the color of the estimation target object. With the above configuration, the contrast difference between the conveyor belt and the estimation target can be increased to improve the accuracy of edge detection, and thus the accuracy of mass estimation can be improved.

Furthermore, according to the mass estimation device according to the sixth aspect, in addition to any of the above configurations, the illuminating unit further includes an illuminating unit that illuminates the estimated object to be conveyed by the illuminating unit. The transport unit is arranged on the side opposite to the surface on which the image pickup unit is arranged with respect to the transport surface, and the illumination light from the illumination unit is transmitted through the transport surface to illuminate the estimated object. It is light, and the height of the estimation target can be estimated from the image captured under the illumination by the transmitted illumination light. With the above configuration, it is possible to estimate the mass of the estimation target by taking into account not only the two-dimensional image but also the upper part in the height direction.

Furthermore, according to the mass estimation device according to the seventh aspect, in addition to any of the above configurations, the mass estimation unit performs deep learning in advance on the extracted image of the estimation target object extracted by the image processing unit. Based on the trained model in which the relationship between the image and the mass is trained, the mass of the estimation target at the position specified by the position specifying unit can be estimated.

Furthermore, according to the mass estimation device according to the eighth aspect, in addition to any of the above configurations, the mass estimation unit learns the relationship between the image and the mass by machine learning using a convolutional neural network. A finished model can be used.

Furthermore, according to the mass estimation device according to the ninth aspect, in addition to any of the above configurations, the trained model can be any of VGG14 to VGG18.

Furthermore, according to the mass estimation device according to the tenth aspect, in addition to any of the above configurations, the trained model can be VGG16.

Furthermore, according to the mass estimation device according to the eleventh aspect, in addition to any of the above configurations, the trained model can be made by reducing the number of layers from VGG.

Furthermore, according to the mass estimation device according to the twelfth aspect, in addition to any of the above configurations, the estimation target can be a cut vegetable.

Furthermore, according to the food packaging machine according to the thirteenth aspect, it is a food packaging machine that wraps the estimated objects in predetermined masses, and transports the estimated objects scattered on the transport surface. A part of the transport surface of the unit, the position identification marker provided along the transport direction of the transport unit, and the estimated object fixed above the transport unit and transported by the transport unit. An image pickup unit that captures an image of an estimation target object existing in the image including the position identification marker, an image processing unit that extracts an estimation target object from an image including the position identification marker imaged by the imaging unit, and the above. From the image including the position specifying marker captured by the imaging unit, deep learning is performed in advance for the position specifying unit that specifies the position of the transport surface of the transport unit and the extracted image of the estimation target object extracted by the image processing unit. Based on the trained model in which the relationship between the image and the mass is trained by The output unit that accumulates the mass of each region and outputs the mass of the transported estimated object and the estimated object whose mass has been estimated by the mass estimation unit are transported and held by the transport unit. A stocker and a packaging unit for packaging the estimated object held in the stocker when the accumulated mass of the estimated object reaches a predetermined value based on the output of the output product can be provided. With the above configuration, it is possible to estimate the mass of the estimated objects scattered on the transport surface based on deep learning. Further, since it is possible to specify in which region of the transport surface the mass is measured by the position specifying marker, the partially measured mass can be accumulated and the required amount of the estimated object can be easily packaged.

It is a schematic diagram which shows the mass estimation apparatus which concerns on Embodiment 1. FIG. It is a schematic diagram which shows the food packaging machine using the mass estimation apparatus. It is a block diagram which shows the function of the computer of FIG. It is a schematic diagram which shows the mass estimation system which concerns on Embodiment 2. It is a schematic diagram which shows the mass estimation system which concerns on Embodiment 3. It is a perspective view which shows the position identification marker of a transport part. It is a cross-sectional perspective view which shows the transport part which concerns on the modification. FIG. 8A is an image diagram showing a captured image, and FIG. 8B is an image diagram showing an input image obtained by performing a projective transformation on the captured image of FIG. 8A. FIG. 9A is an image diagram showing an image of the mounting surface on which the estimation target is placed, FIG. 9B is an image of the area of the estimation target region of FIG. 9A, and FIG. 9C is an image diagram showing a boundary image. It is a graph which shows the relationship between area and mass. It is a graph which shows the relationship between a boundary and a mass. It is a schematic plan view which shows the state of setting the area of the band-shaped section image on the transport surface. 13A is a photographed image, FIG. 13B is an input image obtained by projecting and transforming FIG. 13A, FIG. 13C is a band-shaped section image of band width × 1, FIG. 13D is a band-shaped section image of band width × 2, and FIG. 13E is a band-shaped section of band width × 5. It is an image diagram which shows each section image. 14A to 14F are graphs showing the estimation accuracy of the estimation from the fifth region with the bandwidth 1. 15A to 15F are graphs showing the estimation accuracy of the estimation from the sixth region with the bandwidth 1. It is an image diagram which shows an example of extracting an estimation object from a transmitted illumination image. 17A to 17D are image diagrams showing a state of data expansion. It is a schematic diagram which shows the network structure using Embodiment 5. It is a graph which shows the mean absolute error in the cross-validation of 5 divisions. It is a graph which shows the average percent error. It is a schematic diagram which shows the structure of VGG16. It is a graph which shows the result of the cross-validation of 5 divisions in VGG16. It is a graph which shows the percent error of cross-validation of 5 divisions in VGG16. It is a graph which shows MAE of cross-validation of 5 divisions in VGG19. It is a graph which shows the percent error of the cross-validation of 5 divisions in VGG19. It is a graph which shows MAE in the cross inspection of 5 divisions in Resnet50. It is a graph which shows the percent error of the cross-validation of 5 divisions in Resnet50. It is a graph which shows the MAE of the cross-validation of 5 divisions in Inception V3. It is a graph which shows the percent error of the cross-validation of 5 divisions in Inception V3. It is a graph which shows MAE of cross-validation of 5 divisions by Xception. It is a graph which shows the percent error of the cross-validation of 5 divisions by Xception. It is a schematic diagram which shows the model which fine-tuned only FC Block. It is a schematic diagram which shows the model which fine-tuned FC Block and Block 5. It is a schematic diagram which shows the model which fine-tuned FC Block and Blocks 5 and 4. It is a schematic diagram which shows the original VGG16. It is a schematic diagram which shows the model which removed Block 5 from VGG16. It is a schematic diagram which shows the model which removed Blocks 5 and 4 from VGG16.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments shown below are examples for embodying the technical idea of the present invention, and the present invention is not specified as the following. Further, the present specification does not specify the members shown in the claims as the members of the embodiment. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the constituent members described in the embodiments are not intended to limit the scope of the present invention to the specific description unless otherwise specified, and are merely explanatory examples. It's just that. The size and positional relationship of the members shown in each drawing may be exaggerated to clarify the explanation. Further, in the following description, members having the same or the same quality are shown with the same name and reference numeral, and detailed description thereof will be omitted as appropriate. Further, each element constituting the present invention may be configured such that a plurality of elements are composed of the same member and the plurality of elements are combined with one member, or conversely, the function of one member is performed by the plurality of members. It can also be shared and realized. In addition, the contents described in some examples and embodiments can be used in other embodiments and embodiments.

In the present specification, mass estimation for a single estimation object is intended. In other words, mass estimation for multiple objects with different densities is not covered. However, those having the same density are the subject of the present invention.
[Embodiment 1]

The mass estimation device 100 according to the first embodiment of the present invention is shown in FIG. The mass estimation device 100 includes a transport unit 10 having a transport surface 11 for transporting the estimation target WK, an image pickup unit 20 fixed above the transport unit 10, and a calculation unit 30 connected to the image pickup unit 20. .. The mass estimation device 100 can partially estimate the masses of a large number of estimation objects WK scattered on the transport surface 11. The mass estimation device 100 is used by being incorporated in, for example, a packaging machine.

As an example of the packaging machine, the food packaging machine 1000 is shown in FIG. In the food packaging machine 1000, foods such as cut vegetables and fried tofu, which are WKs whose mass is to be estimated, are piled up in bulk on the conveyor belt 12 and conveyed toward the shooter 61 side. A packaging portion 62 is provided at the lower part of the shooter 61. With the amount of food to be packaged, that is, a predetermined mass of food accumulated in the shooter 61, the bottom surface of the shooter 61 is opened and the food is sent to the packaging unit 62. Specifically, a certain amount of food scattered on the transport surface 11 is transported and stopped, and the partial mass of the food on the transport surface 11, here, within the imaging field of view that can be imaged by the imaging unit 20. Then, the mass of the food contained in the predetermined region defined by the position identification marker 42 (details will be described later) is estimated. The food having an estimated mass is charged into the shooter 61, the mass charged into the shooter 61 is added, and when the mass reaches a certain level, the food is sent to the packaging unit 62, and the food is packaged and discharged. As a result, the transportation of a certain amount of food is realized.

Further, the shooter is not always indispensable, and the packaging unit 62 may be configured to directly input the estimated object WK such as food. Further, the mass of the estimated object WK is accumulated without stopping the operation of the conveyor belt 12, and when the mass reaches a certain amount, the partition or stopper for partitioning the packaging portion 62 is opened for packaging. , The conveyor belt 12 can be packaged while being continuously operated, and workability is improved.

The food packaging machine is an example, and it is also possible to wrap a transported object other than food. That is, in the present invention, the estimation target for estimating the mass is not limited to foods, but also includes, for example, powder amount, granules, tablet-like chemicals, machine parts, and the like. Further, the mass estimation device 100 according to the present embodiment is not limited to the packaging machine, and can be incorporated into a system that is required to measure a predetermined amount. In the following example, an example of applying to food as an estimation object for estimating mass will be described.

The transport unit 10 is provided with a transport surface 11 on the upper surface, and transports the estimated object WK scattered on the transport surface 11 in a constant transport direction. An endless track such as a conveyor belt 12 can be preferably used for the transport unit 10. In the example of FIG. 1, the transfer unit 10 is connected to the calculation unit 30, and the calculation unit 30 switches the movement and stop of the transfer surface 11 and controls the delivery amount. Here, in order to facilitate imaging by the imaging unit 20, the transfer surface 11 is moved by a certain amount, then stopped, waiting for imaging by the imaging unit 20, and then the transfer surface 11 is moved and stopped again. I'm repeating. The transport unit 10 and the calculation unit 30 do not necessarily have to be connected. For example, as a control on the food packaging machine side, the transport unit repeatedly stops and moves the transport surface at a predetermined timing, and the calculation unit outputs the estimated mass according to this timing and whether or not it can be input to the packaging unit 62. May be configured to indicate.

Further, a position specifying marker 42 is provided along the transport direction of the transport unit 10. The position identification marker 42 is displayed on the marker display surface 41 provided along at least one end edge of the conveyor belt 12. The position identification marker 42 is a display in which a plurality of marks are displayed in succession along the transport direction, and the marks are displayed so as to be different from each other. As a result, the calculation unit 30 can grasp which position of the conveyor belt 12 on the endless track is being imaged.

The imaging unit 20 captures an image of the estimated object WK existing in a part of the area of the conveyed surface 11 among the estimated objects WK conveyed by the conveyed unit 10, including the position identification marker 42. As the image pickup unit 20, a camera using an image sensor such as a CCD or CMOS can be preferably used. Further, as the imaging unit 20, in addition to the two-dimensional image, an imaging unit that can obtain height information may be used. For example, a depth camera using an infrared compound eye, a stereo camera using an illuminance difference stereo method, a light cutting method, or the like can be used. Further, the imaging unit 20 sends the captured input image to the calculation unit 30.

Further, as the illumination light for the estimation target WK, natural light may be used, or illumination may be separately arranged. Further, it is preferable that the color of the conveyor belt 12 is different from the color of the estimation target WK. As a result, the contrast difference between the estimated object WK and the conveyor belt 12 becomes large, and it becomes easier to extract the contour of the estimated object WK. At this time, it is also preferable to adjust the direction of the illumination light and the color of the illumination light so that the outline of the estimation target WK easily emerges.

The calculation unit 30 acquires an input image from the imaging unit 20, estimates the mass, and outputs the image. As shown in the block diagram of FIG. 3, the calculation unit 30 includes an image processing unit 31, a position specifying unit 32, a mass estimation unit 33, and an output unit 34.

The position specifying unit 32 identifies a predetermined processing area to be mass-estimated based on the position specifying marker 42 included in the input image captured by the imaging unit 20. Here, the position of the transport surface 11 of the transport unit 10 included in the field of view of the image pickup unit 20 is specified by the position identification marker 42. That is, the input images sequentially sent from the imaging unit 20 are distinguished so that the processing areas do not overlap.

Further, the image processing unit 31 extracts the estimation target WK included in the processing area specified by the position specifying unit 32 from the image including the position specifying marker 42 captured by the imaging unit 20. That is, image processing such as removing foreign matter contained in the processing area or extracting only the outline of the estimation target WK is performed.

Further, the mass estimation unit 33 estimates the mass of the estimation target WK at the position specified by the position identification unit 32 with respect to the extracted image of the estimation target WK extracted by the image processing unit 31. Mass estimation is completed in about 1 second.

The output unit 34 accumulates the mass of the specific region estimated by the mass estimation unit 33 for each region, and outputs the mass of the conveyed estimated object WK. For example, in the example of the food packaging machine, the mass estimated sequentially is sent out to notify the timing of putting the mass into the packaging unit 62.

The arithmetic unit 30 that realizes such a plurality of functions can be configured by, for example, a CPU, an MPU, a GPU, a TPU, or the like, a processor or a microcomputer such as an FPGA, an ASIC, or an LSI, or a chipset such as a SoC or an MCU.

The calculation unit 30 that functions as described above makes it possible to estimate the mass of the estimation target WK scattered on the transport surface 11. For example, the mass estimation unit 33 estimates the mass of a cut vegetable placed on a flat surface such as a belt conveyor or a table only from an image taken from above. The estimation algorithm preferably uses a machine learning-based system that automatically constructs an estimation formula from past data. A convolutional neural network is used for machine learning. Further, since the position specifying marker 42 can specify which region of the transport surface 11 the mass is measured, the partially measured mass can be accumulated and the required amount of the estimated object can be easily cut out.

Further, the mass estimation unit 33 specifies the extracted image of the estimation target object extracted by the image processing unit 31 by the position specifying unit 32 based on a learned model in which the relationship between the image and the mass is learned in advance by deep learning. It can be configured to estimate the mass of the estimated object at the given position. Any of VGG14 to VGG18 can be preferably used for this trained model.

As described above, the background to the demand for a system that easily estimates the mass from an image instead of directly measuring the actual amount of the object to be estimated is the demand for higher speed than the refinement of measurement. In fields where freshness is required, such as food production, the required amount is produced every day, but it is difficult to spend a large amount of time on production in order to maintain freshness. For this reason, when mass production is required, it is necessary to construct large-scale equipment capable of parallel production, but it is often difficult for small and medium-sized enterprises and local companies to make large-scale capital investment. In the production of food, weighing work is often performed before packaging, but precise weighing work using a scale takes time. Weighing with a load cell is often used in automatic weighing, but it takes at least several seconds to wait until the agitation caused by the impact of contact between the food falling from the top and the load cell stabilizes.

In view of such a background, the present inventors have made high-speed processing possible by simply estimating the mass from an image captured by using an imaging unit such as a camera as described above. Specifically, machine learning is performed by a convolutional neural network, and the weighing result for an unknown image is predicted and estimated from the past image and the weighing result data. This made it possible for the arithmetic unit 30 to predict the mass in a few milliseconds to a few tens of milliseconds.

The connection between the imaging unit 20 and the calculation unit 30, or the connection between other peripheral devices or devices such as an external server is not limited to a wired connection, but may be a wireless connection. For example, data communication can be performed by a standardized communication method such as WiFi, Bluetooth, or a form thereof, BLE (Bluetooth Low Energy), Zigbee (all are product names or service names).
[Embodiment 2]

Further, in FIG. 1, an example in which mass estimation is performed using a stand-alone computer as the calculation unit 30 has been described. However, the present invention is not limited to this configuration, and for example, the image data may be transmitted to the outside by data communication or the like, and the mass may be estimated externally. As an example, a mass estimation system by cloud computing is shown in FIG. 4 as a second embodiment. In the mass measurement system shown in this figure, an image captured by the image pickup unit 20 of the mass estimation device 100'provided at a certain location is sent to a server by data communication via a network line. The server estimates the mass of the received image and outputs it. The estimation result is transmitted to the mass estimation device 100'side via the network. Alternatively, it may be configured to send to a client in another location and perform necessary processing.
[Embodiment 3]

In the example shown in FIG. 1, a natural image of the estimated object WK arranged on the transport surface 11 taken from above by the imaging unit 20 is used. In this case, the image captured from the upper part is a two-dimensional plane image, and the area to be imaged can be grasped, but the difference in mass with respect to the apparent unit area from the upper part due to the difference in thickness and overlap cannot be identified. be. Therefore, it is conceivable to reduce the estimation error by using the volume information that takes into account not only the area but also the height information. For example, the illumination unit 50 is arranged on the opposite side of the transport surface 11 with respect to the image pickup unit 20 to perform transmitted illumination that projects illumination light from the back surface, and the difference in brightness of the transmitted light is used as the difference in thickness, that is, height information. It is conceivable to use it. An example of such a case is shown in FIG. 5 as a mass estimation device 200 according to the third embodiment. In the mass estimation device 200 according to the third embodiment, the same members as those described above are designated by the same reference numerals, and detailed description thereof will be omitted as appropriate.

The mass estimation device 200 shown in FIG. 5 includes an illumination unit 50. The lighting unit 50 is a member for illuminating the estimated object WK transported by the transport unit 10. The illumination unit 50 is arranged on the side of the transport unit 10 opposite to the surface on which the image pickup unit 20 is arranged with respect to the transport surface 11. The illumination light emitted by the illumination unit 50 is transmitted illumination light that passes through the transport surface 11 and illuminates the estimated object WK. Control of the illumination unit 50, for example, switching of the amount of light and ON / OFF of the illumination, etc. may be performed from the calculation unit 30. Further, for the transport unit 10, a material that easily transmits illumination light, for example, a translucent flexible resin material, a non-woven fabric, paper, or the like can be used.

The calculation unit 30 can estimate the height of the estimation target WK from the image captured under the illumination by the transmitted illumination light. With such a configuration, it is possible to estimate the mass of the estimation target WK by taking into account not only the two-dimensional image but also the upper part in the height direction.
(Position identification marker 42)

As the position identification marker 42, known AR markers, symbols and the like can be used. Bar codes, two-dimensional codes, etc. can be used as symbols. In particular, standardized symbols can be preferably used. Examples of the two-dimensional code include QR code, micro QR code, tQR, iQR, SQRC (QR code with security function), CP code, Data matrix (Data code), Veri code, and Aztec code. code), PDF417, MicroPDF417, Maxi code, GS1Data Matrix, GS1Data Bar14, Limited, Stacked, Expanded + Composite, UPC / EAN / GS1-128 / ITF / Codabar, etc. ). There are two types of two-dimensional code, stack type and matrix type, and both can be used. In the following example, the ArUco marker, which is an AR marker, was used as the position identification marker 42. The ArUco marker is a binary square reference marker that can be used to estimate the attitude of the camera included in OpenCV. A lightweight AR library based on OpenCV, the ArUco marker can be read from a long distance in real time and is reliable compared to other symbols such as QR codes that have a large number of cells and need to focus on the camera. You can get advantages such as high price.

An example of the position identification marker 42 is shown in FIG. In this example, the conveyor belt 10 is the conveyor belt 12, and the position identification marker 42 is displayed along one end edge of the conveyor belt 12. With such a configuration, while the object is conveyed by the conveyor belt 12, the image of the estimated object WK is imaged by the imaging unit 20 provided at a fixed position to sequentially estimate the mass, and the conveyed estimated object is conveyed. It is possible to accumulate the mass of WK.
(Wall 44)

In the example of FIG. 6, an example in which the position identification marker 42 is provided on the right side (lower side in the drawing) with respect to the traveling direction of the conveyor belt 12 has been described, but the present invention is not limited to this configuration and is left side with respect to the traveling direction of the conveyor belt. May be provided with a position identification marker. When the position identification marker is displayed on the transport surface of the conveyor belt, it becomes invisible when the estimated object is loaded on the transport surface. Therefore, the transport surface 11 and the marker display surface 41 provided with the position identification marker 42 are distinguished from each other. It is preferable to do so. In the example of FIG. 6, the wall portion 44 is upright at the boundary portion between the transport surface 11 and the marker display surface 41. The wall portion 44 projects in a posture that intersects the transport surface 11 and the marker display surface 41, preferably in a posture that is orthogonal to the transport surface 11. By interposing such a wall portion 44, it is possible to avoid a situation in which the estimated object WK loaded on the transport surface 11 collapses and spreads over the marker display surface 41 to cover the position specifying marker 42.
(Display belt 43)

The marker display unit is preferably displayed on a display belt 43 separate from the conveyor belt 12. The display belt 43 is arranged adjacent to the conveyor belt 12 and runs in parallel with the conveyor belt 12. The display belt 43 is preferably made of a material having the same elongation rate as the conveyor belt 12, and is driven by the same drive shaft as the conveyor belt 12, so that the movement amounts of the conveyor belt 12 and the display belt 43 are matched. Is possible. By making the conveyor belt 12 and the display belt 43 separate members in this way, it becomes easy to provide the wall portion 44, which is a fixing member, between them.

However, the present invention is not limited to this configuration, and a transport surface and a marker display surface may be provided on a common belt. As an example, in the mass estimation device 300 according to the modified example shown in the cross-sectional perspective view of FIG. 7, the wall portion 44'is fixed to the upper surface of the conveyor belt 12', and the transport surface 11 and the marker are fixed by the wall portion 44'. The display surface 41 is partitioned. With this configuration, it is possible to avoid a situation in which the estimated object WK on the transport surface 11 protrudes from the marker display surface 41 while using a common belt. The wall portion 44'may be in the form of rubber integrally molded with the conveyor belt 12', or the wall may be formed of a separate member and fixed on the conveyor belt.
(Mass estimation algorithm)

As an algorithm for estimating the mass of the object to be estimated by the mass estimation unit 33, a known algorithm for estimating the mass from an image can be appropriately used. In this embodiment, machine learning is used. Specifically, a neural network and a non-neural network are used.

As the non-neural network method, linear regression, nonlinear polynomial regression, and support vector nonlinear regression can be used. Since feature extraction is required in these non-neural network methods, the image processing unit 31 extracts the feature amount. As the feature amount, the object area area (area), the object boundary area (edge), the luminance information, and a combination of these can be used.

On the other hand, in the neural network method, it is preferable to use a convolutional neural network. As the convolutional neural network, deep learning (deep neural network), pre-learned convolutional neural network, VGG convolutional neural network delamination model and the like can be used.
[Embodiment 4: Non-Neural Net]

Hereinafter, each method will be described in order. First, a procedure for mass estimation using non-neural network methods such as linear regression, non-linear polynomial regression, and support vector non-linear regression will be described as a mass estimation method according to the fourth embodiment. As an example, an example of correcting the angle and orientation of the captured image captured by the imaging unit 20 will be described with reference to FIGS. 8A to 8B. Here, cut carrots are used as the estimation target. FIG. 8B shows the result of performing the projective transformation cropping to a rectangle on the captured image of FIG. 8A. This gives an input image to the prediction model.

Next, an example of preprocessing for converting the input image by the image processing unit 31 will be described with reference to FIGS. 9A to 9C. Here, a corrugated cardboard chip is used as an estimation target. In these figures, FIG. 9A is an input image obtained by imaging the mounting surface on which the estimation object is placed, FIG. 9B is an image showing the area of the estimation object area of FIG. 9A, and FIG. 9C is the boundary of the estimation object. Images showing (edges) are shown respectively. As shown in FIG. 9A, the color of the conveyor belt 12 is selected to have a large contrast difference with respect to the estimated object. Here, a white conveyor belt is selected for the ocher color of the corrugated cardboard chip. As a result, the input image is binarized to obtain the binarized image of FIG. 9B. Further, the edge image shown in FIG. 9C can be obtained by differentiating the binarized image.
(Relationship between preprocessed image and mass)

Scatter plots obtained by measuring and plotting the relationship between the preprocessed image obtained as described above and the actual mass are shown in FIGS. 10 to 11. In these figures, FIG. 10 shows the relationship between the area and mass of the estimated object area of the captured image from above, and FIG. 11 shows the relationship between the estimated object boundary (edge) and mass of the captured image from above. ing. Here, a corrugated cardboard piece imitating a cut vegetable was used as the object to be measured. As shown in these figures, it can be seen that the area and boundary of the captured image from above show a roughly proportional relationship with the actual mass.
(Continuous measurement)

Next, a method will be described in which the transport surface 11 is sequentially fed, the region on the transport surface 11 is divided into strips, and the mass of the image of these strip sections is estimated quasi-continuously. Here, as shown in the plan view of FIG. 12, the transport surface 11 is moved downward in the drawing while the imaging unit 20 is fixed, and the strip-shaped section image shown by the broken line is relatively displayed at time t-1 and time t. Consider an example of moving it upward in the figure. In the strip-shaped section image, the width (the strip width corresponding to the rectangular shape in the vertical direction in the figure) is x pixels. The band width of the strip-shaped section image matches the transport direction of the conveyor belt 12 which is the transport unit 10, and can be determined by the feed amount or the transport speed of the conveyor belt 12. Further, as shown in FIG. 13A, the band width can be distinguished by the position specifying marker 42 provided on the right side of the conveyor belt 12. For example, with respect to the input image of FIG. 13B obtained by projecting the captured image of FIG. 13A, the band width × 1 is shown in FIG. 13C, the band width × 2 is shown in FIG. 13D, and the band width X5 is shown in FIG. 13E, respectively. In this way, the input image is decomposed into band-shaped section images, the mass is estimated by the mass estimation unit 33 for each band-shaped section image, the obtained minute mass is integrated, and the conveyor belt 12 is conveyed until the target mass is reached. do. This makes it possible to obtain a measurement object having a desired mass.

The estimation accuracy thus obtained is shown in FIGS. 14A to 14F. Here, the band width × 1 is estimated from the region. In these figures, FIGS. 14A to 14C show the relationship between mass and APE [%], and FIGS. 14D to 14F show the relationship between area (area) and mass [g]. The linear regression (AREA5) 14A and 14D are FIGS. 14B and 14E are nonlinear polynomial regression (area5, degree = 5), Figure 14C and Figure 14F is support vector nonlinear regression ^{(area5, gamma = 2 -21,} C = 2 ^5, epslion = 2 ^-3), respectively. The results of linear regression, non-linear polynomial regression, and support vector non-linear regression (SVR) are shown in Table 1 from FIGS. 14A to 14F.

Similarly, the estimation accuracy is shown in FIGS. 15A to 15F. Here, the band width × 5 was estimated from the region. Also in these figures, FIGS. 15A to 15C show the relationship between mass and APE [%], and FIGS. 15D to 15F show the relationship between area (area) and mass [g]. 15A and 15D are linear regression (area6), FIGS. 15B and 15E are nonlinear polynomial regressions (area6, degree = 5), and FIGS. 15C and 15F are support vector nonlinear regressions (area6, gumma = 2 ^-17 , C). = 2 ^7, a epslion = 2 ^-2), are shown, respectively. Table 2 shows the results of linear regression, non-linear polynomial regression, and support vector non-linear regression (SVR) from FIGS. 15A to 15F.

（機械学習による予測モデルの生成）

(Generation of prediction model by machine learning)

In the above example, an example of mass estimation using a non-neural network has been described. Next, an example in which the mass estimation unit 33 performs mass estimation by machine learning will be described. In order to estimate the mass of the object to be estimated from the image including the position identification marker 42 captured by the imaging unit 20, the input image by the imaging unit 20 and the weighing output by the scale are learned in advance by supervised machine learning, and the image is taken from the image. Create a mass prediction model.

First, the imaging unit 20 captures a captured image of the estimated object. This procedure is almost the same as the case of the non-neural network described above. Specifically, a method of fixing the imaging unit 20, for example, a position specifying marker 42 for specifying a region so that stable and robust prediction can be performed even if the distance or angle between the estimation target and the imaging unit 20 changes slightly. Is provided. As described above, the position identification marker 42 is displayed on the display belt 43 arranged on the side of the transport surface 11. The image pickup unit 20 captures an image including the position identification marker 42 and sends it to the calculation unit 30. The calculation unit 30 acquires an input image from the image pickup unit 20, projects (registers) it into a rectangular area so that the image is viewed from the front, and cuts out the area by the position identification marker 42. The order of projective transformation and cutting out does not matter. For example, the cutout may be performed first and then the projective conversion may be performed, or these may be performed at the same time.

Further, when the area to be cut out is a rectangle, the position identification marker 42 is placed at the four corners of the rectangle, for example. In this case, display belts 43 displaying the position identification markers 42 are arranged on both sides of the transport surface 11. As a result, it is possible to easily cut out a rectangle having the position specifying marker 42 included in the input image as the four corners. However, the present invention is not limited to this configuration, and the position identification marker 42 is placed only at three points in the corner of the rectangle to correct only the distance and the angle, and conversely, to correct complicated surface distortion. You can also place more than a point.

Next, the image is extracted (cropped) in order to avoid various problems caused by including an object other than the estimation target in the cut out image. For example, if the cut-out image contains a foreign substance, it may take extra time for the learning to converge or it may be difficult for the learning to converge. Therefore, as a preprocessing, the estimation target is extracted from the image. An example of the image subjected to this preprocessing is shown in FIG. In this example, the image obtained by the transmitted illumination of the third embodiment is shown.

After acquiring the input image as described above, machine learning is performed by the mass estimation unit 33. Here, estimation by deep learning will be described. First, an example using the three-layer CNN model will be described as the fifth embodiment.
[Embodiment 5: Estimate by deep learning using a 3-layer CNN model]

The three-layer CNN model includes two convolutional layers (CNN) and one fully connected layer (FC, for recognition). With this three-layer model, the mass of carrots on the conveyor belt is estimated using CNN. Here, as a target data set, 946 images of 570 × 330 taken by a camera of cut carrots randomly sown between about 0 to 550 g and taken from a distance of 50 cm were prepared. The imaging field of view on the conveyor belt was 100 cm × 20 cm. The number of these images was quadrupled by performing mapping conversion in the XY directions, and a total of 3784 images were prepared. Here, the original image data as shown in FIG. 17A is transferred in the X-axis direction (inverted in the vertical direction) as shown in FIG. 17B, and transferred in the Y-axis direction as shown in FIG. 17B (horizontal). Data was expanded to increase the number of image data by preparing a data transferred in both the X-axis and the Y-axis (inverted both vertically and horizontally) as shown in FIG. 17D. In addition, Mean Absolute Error (MAE) was used as the evaluation method. In addition, cross-validation of 5 divisions was also performed. The network structure used in the fifth embodiment is shown in the schematic diagram of FIG. The network structure shown in this figure is composed of a conversion layer (Conv. Layer), a pooling layer (Pooling layer), a conversion layer, a pooling layer, an FC layer (FC layer), and an output layer (Output layer). Furthermore, in the CNN model, the batch size was 20, the learning epoch was 100, and MSE was used for the loss function.

The obtained mean absolute error is shown in the graph of FIG. In MAE in 5-fold cross-validation, 1fold was 18.0, 2fold was 17.1, 3fold was 18.4, 4fold was 24.2, and 5fold was 17.7. As a result of such cross-validation of 5 divisions, the average MAE was 19.1 g, the standard deviation was 2.59, and the learning time was 1 hour and 23 minutes.

Further, the graph of FIG. 20 shows an average percentage error indicating how much error there is for how much mass. The percentage error in cross-validation of 5 divisions was 0.41 for 1fold, 0.59 for 2fold, 0.27 for 3fold, 0.30 for 4fold, and 0.41 for 5fold. These average percent errors were 39%.

As a result of the above, in the estimation by deep learning using the 3-layer CNN model, the MAPE (mean percentage error) was about 40%, and it was stable at 200 g or more (about 20% of MAPE). On the other hand, those having a weight of 50 g or less had poor accuracy.
[Embodiment 6: Application of trained multi-layer model]

Next, an example of applying the trained multilayer model as the sixth embodiment will be described. Here, transfer learning was performed, and only the FC layer was relearned. Transfer learning is a method of adapting this learning model to a task of another related domain when there is a trained model trained for the task of one domain. The advantage is that less training data is required. Since the number of image data handled in this embodiment is small, it is considered that this transfer learning is suitable. We also performed fine tuning to relearn a part of the model. Here, the mass of food was estimated using five models, VGG16, VGG19, Resnet50, InceptionV3, and Xception, as trained models. As an example, the structure of VGG 16 is shown in FIG. Here, it is composed of blocks 1 to 5 and FC blocks.

In addition, the scope of application of fine tuning was limited to FC block, and only the full coupling was reconstructed for mass estimation and relearned. In the example of VGG16 shown in FIG. 21, only the “FC block” shown at the right end was relearned.

Next, the verification result of VGG16 will be described. First, the graph of FIG. 22 shows the result of cross-validation of 5 divisions. The mean absolute error (MAE) in cross-validation of 5 divisions was 18.9 for 1fold, 18.2 for 2fold, 18.9 for 3fold, 17.6 for 4fold, and 18.6 for 5fold. The above average MAE was 18.4 g and the standard deviation was 0.48. The learning time was 3 hours and 42 minutes.

Next, the percentage error in cross-validation of 5 divisions in VGG16 is shown in the graph of FIG. Here, it was 0.52 at 1 fold, 0.33 at 2 fold, 0.27 at 3 fold, 0.27 at 4 fold, and 0.54 at 5 fold. The average percent error was 41%.

On the other hand, for VGG19, the result of obtaining MAE in cross-validation of 5 divisions is shown in the graph of FIG. 24. Here, 1 fold was 31.2, 2 fold was 23.6, 3 fold was 23.8, 4 fold was 23.8, and 5 fold was 25.1. The average MAE was 25.5 g, the standard deviation was 2.88, and the learning time was 3 hours and 56 minutes.

Also, the percentage error of VGG19 in the cross-validation of 5 divisions is shown in the graph of FIG. 25. Here, 1 fold was 0.51, 2 fold was 0.48, 3 fold was 0.54, 4 fold was 0.5, and 5 fold was 0.68. The average percent error was 54%.

Further, for Resnet50, the MAE in 5-fold cross-validation is shown in the graph of FIG. Here, 1 fold was 192.3, 2 fold was 193.1, 3 fold was 176.6, 4 fold was 165.7, and 5 fold was 174.9. The average MAE was 180.5 g, the standard deviation was 10.6, and the learning time was 4 hours and 28 minutes.

Further, the percentage error in the cross-validation of the 5 divisions of Resnet 50 is also shown in the graph of FIG. 27. Here, 1 fold was 2.6, 2 fold was 4.5, 3 fold was 3.2, 4 fold was 6.3, and 5 fold was 4.6. The average percent error was 420%.

On the other hand, for Inception V3, the MAE in 5-fold cross-validation is shown in FIG. 28. Here, 1 fold was 154.8, 2 fold was 147.6, 3 fold was 141.9, 4 fold was 135.1, and 5 fold was 146.9. The mean MAE was 145.2 g and the standard deviation was 6.6.

Further, FIG. 29 shows the percentage error in the cross-validation of the five divisions of Inception V3. Here, 1 fold was 1.58, 2 fold was 1.64, 3 fold was 1.98, 4 fold was 1.84, and 5 fold was 2.56. The average percent error was 192%.

On the other hand, FIG. 30 shows MAE in cross-validation of Xception's 5-fold cross-validation. Here, it was 171.3 for 1 fold, 176.8 for 2 fold, 191.1 for 3 fold, 188.0 for 4 fold, and 182.7 for 5 fold. The average MAE was 181.9 g, the standard deviation was 7.2, and the learning time was 3 hours and 2 minutes.

Further, FIG. 31 shows the percentage error in the cross-validation of Xception in 5 divisions. Here, 1 fold was 2.67, 2 fold was 2.01, 3 fold was 2.11 and 4 fold was 2.54, and 5 fold was 1.46. The average percent error was 215%.

From the above, VGG16 showed the best result. Although it is generally considered that the performance is higher due to the multi-layered structure, it is impossible to construct an estimation model for ResNet, Inception, etc., which are multi-layered than the VGG16. It was also found that transfer learning by ImageNet is not effective for mass estimation.
[Example 7: Reduction of trained model]

Further, an example in which the trained model is reduced in layer will be described below as the seventh embodiment. Here, we examined six models that reduce the number of layers based on VGG16. The stratified CNN model is the following six stratified CNN models constructed based on VGG16.
(1) Fine-tuned model of FC Block only (2) Fine-tuned model of FC Block + Block5 (3) Fine-tuned model of FC Block + Block5, 4 (4) Original VGG16
(5) Model with Block 5 removed from VGG 16 (6) Model with Block 5 and 4 removed from VGG 16

Of the above, (1) to (3) have an image, and (4) to (6) have no image.
(1) Fine-tuned model of FC Block only

A model in which only FC Block is fine-tuned is shown in FIG. Here, Blocks 1 to 5 were fixed with the weights learned by Imaget, and only FC Block (far right in the figure) was relearned.
(2) Fine-tuned model of FC Block + Block 5

A model in which FC Block and Block 5 are fine-tuned is shown in FIG. 33. Here, Blocks 1 to 4 were fixed with the weights learned by Imaget, and only FC Block and Block 5 were relearned.
(3) Fine-tuned model of FC Block + Block 5 and 4

Further, FIG. 34 shows a model in which FC Block and Blocks 5 and 4 are fine-tuned. Here, Blocks 1 to 3 were fixed with the weights learned by Imaget, and only FC Block and Blocks 5 and 4 were relearned.
(4) Original VGG16

The original VGG16 is shown in FIG. Here, all layers are learned from 1 and there is no Image weight.
(5) Model with Block 5 removed from VGG 16

A model in which Block 5 is removed from VGG 16 is shown in FIG. Again, we are learning from 1 and there is no Image weight.
(6) Model with Blocks 5 and 4 removed from VGG16

Finally, FIG. 37 shows a model in which Blocks 5 and 4 are removed from VGG16. Again, we are learning from 1, and there is no Image weight.

Evaluation is performed using the above trained model. Here, as a data set, 946 images of cut carrots on the white conveyor belt shown in FIG. 1, 559 images of cut carrots on the blue conveyor belt, and 672 images of cut onions. Furthermore, from the back side of the translucent conveyor belt shown in FIG. 2, 100 images of cut carrots are placed on the mounting surface with 10-color LEDs lit, and 100 images of cut onions are placed for each color. Similarly, 110 sheets were prepared for each color. As the evaluation method, the mean percent error (MAPE) shown by the following equation was used.

In the above equation, N is the number of images, Ti is the true mass, and Ei is the estimated mass. In the part where the true mass is small, the MAPE becomes high and becomes an outlier. Here, the limit was set to 0 g or more, 10 g or more, 20 g or more, and so on, and the MAPE was calculated.

The results of using each of the above trained models are shown for each conveyor belt. First, when natural light (reflected light) from above is used as the illumination light shown in FIG. 1, the estimation result of the mass of the image captured on the white conveyor belt is shown in Table 3.

Table 4 shows the results of mass estimation for the image of the cut carrot mounted on the blue conveyor belt.

Similarly, Table 5 shows the estimation results of the mass of the image of the cut onion placed on the blue conveyor belt.

Next, in the case of using the transmitted illumination of FIG. 5, among the mass estimation results for the image of the cut onion placed on the translucent conveyor belt, 0 g or more of MAPE is shown in Table 6.

Further, Table 7 shows MAPE of 10 g or more among the mass estimation results for the image of the cut onion placed on the translucent conveyor belt.

Furthermore, Table 8 shows MAPE of 20 g or more among the mass estimation results for the image of the cut onion placed on the translucent conveyor belt.

Furthermore, Table 9 shows the MAPE of 0 g or more among the mass estimation results for the image of the cut carrot mounted on the translucent conveyor belt.

Similarly, Table 9 shows MAPE of 10 g or more among the mass estimation results for the image of the cut carrot mounted on the translucent conveyor belt.

Similarly, Table 11 shows a MAPE of 20 g or more among the mass estimation results for the image of the cut carrot mounted on the translucent conveyor belt.

Table 12 shows a MAPE summarizing the learning results of all color data with respect to the image of the cut onion placed on the conveyor belt using the above transmitted illumination.

Similarly, Table 13 shows a MAPE summarizing the learning results of all color data for an image of a cut carrot placed on a conveyor belt using transmitted illumination.

From the above, it was found that the 12-layer model is preferable for the white conveyor belt for the image using the reflected light.
(Convolutional neural network)

The above results are summarized. It is preferable to use a convolutional neural network when performing machine learning by the mass estimation unit after acquiring the input image as described above. As the method of constructing the convolutional neural network, a natural image set ImageNet that is different from the information about food is trained. The machine learning methods used in this embodiment are (1) non-neural network, (2) deep learning (deep neural network), (3) pre-learned convolutional neural network, and (4) VGG16 convolutional neural network reduction. It is a layer model. Since (1) the non-neural network method requires feature extraction, the reflected light conveyor belt image of the first embodiment features the object area area, the object boundary area (edge amount), and the combination of the above two. Used as a quantity. Further, in the transmitted light image of the third embodiment, the luminance information of the object region is used as the feature amount. Furthermore, as machine learning methods, (1) for non-neural networks, linear regression, nonlinear polynomial regression, and support vector nonlinear regression were used. In (2) deep learning, five layers of convolutional neural networks and a pre-learned convolutional neural network were used as machine learning methods. In the pre-learned convolutional neural network, VGG16, VGG19, ResNet50, InceptionV3, and Xception are used. Further (3) In the pre-learned convolutional neural network re-learning layer, only the fully connected layer (FC) was re-learned, and FC + block 5 and FC + blocks 4 and 5 were used as the machine learning method. Finally, in (4) VGG16 convolutional neural network layer reduction model, block 4 layer reduction (12 layers) and block 4, 5 layer reduction (9 layers) were used as machine learning methods. First, (2) the results of deep learning are shown in Table 14.

Here, using a blue conveyor belt as the transport unit 10 in FIG. 1, a deep learning model comparison was performed using carrots cut as an estimation target. Good performance was not obtained for ResNet (residual net) of 40 to 50 layer class and Google's Inception, which are generally said to be able to obtain high performance, and good results were obtained for VGG16 and 5-layer models. .. However, there is a judgment error of about 1.5 times.

Therefore, based on VGG16, it was estimated by a model in which several layers were deleted and the layers were reduced so as to change the re-learning layer, and it was found that better performance could be exhibited. The results are shown in Table 15. Here, a white conveyor belt and a blue conveyor belt are used as the transport unit 10 of FIG. 1, and the transmitted light of FIG. 5 is further applied to compare the mass estimation deep learning model of carrots and onions. Further, in linear regression, polynomial regression, and support vector regression, 1 to 9 g or more (bands 1 to 9) are measured, and the values in Table 2 show the most accurate band width).

In the whole data (0 to 600 grams), the error was large in deep learning, especially when the mass was small, while the error in the regression model was a dozen percent. When this is estimated only in the case of 10 g or more (carrot) or 5 g or more (onion), the error is 5 to 6% for the reflected light of the first embodiment and 4 to 8% for the transmitted light of the third embodiment. became. It was found that the effect of transmitted light can be improved in accuracy because the thickness can be estimated depending on the type of the object to be estimated.

When the conveyor belt and the estimation target are easily separated from the mass estimation of the vegetables cut in this way, the error can be estimated to be about 10% by machine learning, and 5 to 6% if limited to 10 g or more. It was confirmed that. It was also confirmed that when transmitted light is used as the illumination light, the accuracy can be further improved to about 4% by reflecting the thickness of the estimated object by the brightness difference.

In the above example, a mode in which the transport surface is moved by a certain amount and then stopped to capture a still image with the imaging unit has been described. With this method, it is possible to accumulate the estimated weight, stop the transport unit when the weight reaches a predetermined mass, and perform processing such as packaging. However, the present invention may be configured such that the image pickup unit captures a moving image while the transport surface is continuously moved, and the calculation unit acquires the moving image to continuously analyze the mass of the estimated object. good. With this method, since the transport surface is not stopped, continuous processing is possible, the number of times of switching between stop and movement at the transport portion can be reduced, the mechanical load can be reduced, and the tact time can be reduced.

In addition, since the estimated weight can be continuously accumulated, it is possible to perform processing such as stopping and packaging the transport unit when the predetermined mass is reached, or packaging without stopping the transport unit, and estimating the mass. Finer timing enables more accurate packaging.

The mass estimation device according to the present invention can be suitably used for a packaging machine that estimates and packs a mass using an image, particularly a food packing machine. Among them, cut vegetables such as carrots and onions, which have a large amount of water, were placed on a load cell and the mass was measured. As a result, some of the vegetables adhered to the load cell and required cleaning, which was troublesome. On the other hand, by using the mass estimation device according to the present invention, it is not necessary to put it on a load cell or the like, and it is not necessary to clean it, so that it can be used efficiently. It can also be applied to processed foods such as cut fried tofu. Furthermore, it can be used not only for foods but also for surveying chemicals such as tablets and parts such as screws.

1000 ... Food packaging machine 100, 100', 200, 300 ... Mass estimation device 10 ... Conveyor unit 11 ... Conveyor surface 12, 12'... Conveyor belt 20 ... Imaging unit 30 ... Calculation unit 31 ... Image processing unit 32 ... Position identification unit 33 ... Mass estimation unit 34 ... Output unit 41 ... Marker display surface 42 ... Position identification marker 43 ... Display belt 44, 44'... Wall part 50 ... Lighting unit 61 ... Shooter 62 ... Packaging unit WK ... Estimated object

Claims (13)

It is a mass estimation device for partially estimating the mass of a large number of estimation objects scattered on the transport surface.
A transport unit that transports the estimated objects scattered on the transport surface, and a transport unit that transports the estimated objects.
A position identification marker provided along the transport direction of the transport unit, and
Imaging that captures an image of an estimated object that is fixed above the transport unit and that exists in a part of the transport surface among the estimated objects that are transported by the transport unit, including the position identification marker. Department and
From the image including the position identification marker captured by the imaging unit, a position identification unit that identifies a predetermined region based on the position identification marker, and a position identification unit.
An image processing unit that extracts an estimation target included in a predetermined region specified by the position identification unit from an image including the position identification marker captured by the image pickup unit, and an image processing unit.
With respect to the extracted image of the estimation target object extracted by the image processing unit, a mass estimation unit that estimates the mass of the estimation target object at the position specified by the position identification unit, and a mass estimation unit.
An output unit that accumulates the mass of a specific region estimated by the mass estimation unit for each region and outputs the mass of the transported estimated object.
A mass estimation device including. The mass estimation device according to claim 1.
The transport unit is a conveyor belt.
A mass estimation device in which the position-specific marker is displayed on a marker display surface provided along at least one end edge of the conveyor belt. The mass estimation device according to claim 2, further
A mass estimation device including a wall portion provided between the transport surface of the transport portion and the marker display surface. The mass estimation device according to claim 3, further
A display belt arranged adjacent to the conveyor belt and running in parallel with the conveyor belt is provided.
A mass estimation device in which the position identification marker is displayed on the display belt. The mass estimation device according to any one of claims 2 to 4.
A mass estimation device in which the color of the conveyor belt is colored differently from the color of the object to be estimated. The mass estimation device according to any one of claims 1 to 5, further comprising:
A lighting unit for illuminating an estimated object transported by the transport unit is provided.
The illumination unit is arranged on the side of the transport unit opposite to the surface on which the image pickup unit is arranged with respect to the transport surface.
The illumination light from the illumination unit is transmitted illumination light that passes through the transport surface and illuminates the estimated object.
A mass estimation device capable of estimating the height of an object to be estimated from an image captured under illumination by the transmitted illumination light. The mass estimation device according to any one of claims 1 to 6.
The mass estimation unit is specified by the position identification unit based on a trained model in which the relationship between the image and the mass is learned in advance by deep learning with respect to the extracted image of the estimation target object extracted by the image processing unit. A mass estimation device configured to estimate the mass of an object to be estimated at a position. The mass estimation device according to claim 7.
A mass estimation device in which the mass estimation unit uses a trained model in which the relationship between an image and mass is learned by machine learning using a convolutional neural network. The mass estimation device according to claim 7 or 8.
A mass estimation device in which the trained model is any one of VGG14 to VGG18. The mass estimation device according to any one of claims 7 to 9.
A mass estimation device in which the trained model is VGG16. The mass estimation device according to claim 7 or 8.
A mass estimation device in which the trained model is a VGG with a reduced number of layers. The mass estimation device according to any one of claims 1 to 11.
A mass estimation device whose estimation target is cut vegetables. A food packaging machine that wraps estimated objects in predetermined masses.
A transport unit that transports the estimated objects scattered on the transport surface, and a transport unit that transports the estimated objects.
A position identification marker provided along the transport direction of the transport unit, and
Imaging that captures an image of an estimated object that is fixed above the transport unit and that exists in a part of the transport surface among the estimated objects that are transported by the transport unit, including the position identification marker. Department and
An image processing unit that extracts an estimation target from an image including the position identification marker captured by the imaging unit, and an image processing unit.
From the image including the position specifying marker captured by the imaging unit, a position specifying unit that specifies the position of the transport surface of the transport unit and a position specifying unit.
Based on a learned model in which the relationship between the image and the mass is learned in advance by deep learning for the extracted image of the estimated object extracted by the image processing unit, the estimated object at the position specified by the position specifying unit. Mass estimation unit that estimates the mass of
An output unit that accumulates the mass of a specific region estimated by the mass estimation unit for each region and outputs the mass of the transported estimated object.
A stocker that transports and holds an estimation target whose mass has been estimated by the mass estimation unit, and
When the cumulative mass of the estimated object reaches a predetermined value based on the output of the output, the packaging unit for packaging the estimated object held in the stocker and the packaging unit.
Food packaging machine equipped with.

Images