JP7425032B2 - Object determination device and object determination system - Google Patents

Description

The present disclosure relates to an object determination device and an object determination system.

Systems are known for determining the type of object being transported. For example, the type of object is determined by using a sensor to read an identification code provided on the object to be transported or on the transport mechanism of the object (see, for example, Patent Document 1).

Japanese Patent Application Publication No. 11-049331

However, in the prior art, it may be difficult to determine the type of object due to dirt on the part where the identification code is installed or shaking of the object being transported.

An object of the present disclosure is to provide an object determination device and an object determination system that can highly accurately determine the type of an object to be transported.

An object determination device according to the present disclosure includes an extraction section and a determination section. The extraction unit extracts a plurality of regions of interest from the photographed image of the object to be transported. The determination unit is configured to use a learning model that outputs the type of the target object from the plurality of regions of interest, which is learned using the plurality of regions of interest included in the learning image that is the photographed image. The type of the object is determined from the plurality of regions of interest. The target object is a vehicle, and the plurality of regions of interest are regions including mutually different parts of the vehicle.

According to the object determination device and object determination system according to the present disclosure, it is possible to determine the type of the object to be transported with high accuracy.

FIG. 1 is a schematic diagram of an object determination system according to an embodiment. FIG. 2 is a hardware configuration diagram of the object determination device and the sequencer. FIG. 3 is a functional block diagram of the object determination device. FIG. 4 is a schematic diagram of a photographed image. FIG. 5 is an explanatory diagram of the learning image. FIG. 6 is a flowchart showing the flow of information processing.

Embodiments of an object determination device and an object determination system according to the present disclosure will be described below with reference to the accompanying drawings.

FIG. 1 is a schematic diagram showing an example of an object determination system 1 according to the present embodiment. The object determination system 1 includes an object determination device 10, a sequencer 20, an imaging section 30, and a transport mechanism 40.

The object determination system 1 is a system that determines the type of object to be transported.

The target object is an object whose type is to be determined. The target object may be any object that is transported by the transport mechanism 40. The target object is, for example, an object such as a vehicle V, a part constituting an object or a building, food such as fruit, and the like. In this embodiment, an example in which the object is a vehicle V will be described.

The transport mechanism 40 is a mechanism that transports the object. In this embodiment, an example will be described in which the transport mechanism 40 is a transport mechanism for a painting process, which is a process included in the production line of the vehicle V.

The transport mechanism 40 includes, for example, a guide rail 40A, a support member 40B, and a support member 40C. The guide rail 40A is a member extending along the transport direction X. The support member 40B is provided movably along the guide rail 40A, and supports the vehicle V via the support member 40C. The support member 40C supports the vehicle V. The support member 40C may be referred to as a truck. Note that the support member 40C may be any member that can support the vehicle V, such as a member that supports the vehicle V by hanging it, a member that supports the vehicle V by gripping it, or a member that supports the vehicle V by placing it thereon. It may be any member. In this embodiment, the support member 40C is a member that supports the vehicle V by suspending it, as an example.

The transport mechanism 40 transports supporting members 40B arranged at predetermined intervals along the transport direction X in the transport direction X by controlling a drive unit (not shown) by the sequencer 20. By conveying the support member 40B in the conveyance direction X, the vehicle V supported by the support member 40C via the support member 40B is conveyed in the conveyance direction X.

Note that the transport mechanism 40 may be any mechanism that can transport the vehicle V along the transport direction X, and is not limited to the configuration shown in FIG. 1 .

In this embodiment, a tag 60 is provided on the support member 40C. The tag 60 is a device that wirelessly communicates stored information. The tag 60 is, for example, an RF (Radio Frequency) tag. For example, a cart ID (identification information) that uniquely identifies the support member 40C is stored in the tag 60 in advance.

The sequencer 20 is a control device that controls the transport of the transport mechanism 40. The sequencer 20 is communicably connected to each of the transport mechanism 40, the coating device 50, and the object determination device 10.

The sequencer 20 controls the transportation of the vehicle V supported by the support member 40C, and transmits a determination start signal to the object determination device 10 every time the vehicle V is transported to the predetermined position X1. The determination start signal is a signal that instructs to start determining the type of vehicle V. The sequencer 20 may determine that the vehicle V being transported has reached the predetermined position X1, for example, using the detection result of an infrared sensor or the like.

Next, the photographing section 30 will be explained. The photographing unit 30 photographs the transport space in which the vehicle V is transported. The photographing unit 30 is arranged to be able to photograph a predetermined photographing range R in the transport space in which the vehicle V is transported. The photographing range R is adjusted in advance so as to include, for example, a specific part P of the vehicle V transported to the predetermined position X1. The photographing unit 30 is adjusted in advance to photograph a fixed photographing range R in real space. The fixed imaging range R in real space means that the position, size, and range in real space are fixed.

That is, the photographing unit 30 is installed so that an area including a specific part P in the vehicle V is reflected in the photographed image 70 taken when the vehicle V transported by the transport mechanism 40 reaches the predetermined position X1. The position, shooting angle of view, etc. are adjusted in advance.

Part P is a member that constitutes vehicle V. In this embodiment, the photographing range R of the photographing section 30 is adjusted in advance so that at least a plurality of mutually different parts P, ie, a component P1 and a component P2, are reflected. That is, the photographing range R is adjusted in advance so that the plurality of parts P of the vehicle V that have arrived at the predetermined position X1 due to transportation are reflected. The part P may be any member that can identify the type of vehicle V. Further, the number of parts P included in the photographing range R may be plural, and is not limited to two parts P.

The photographing unit 30 obtains a photographed image 70 in a photographing range R by photographing. For example, the photographing unit 30 obtains moving image data composed of a plurality of photographed images 70 that are consecutive in time series. Each photographed image 70 constituting the moving image data is, for example, a still image. In the following description, moving image data may be referred to as a moving image, and still image data may be referred to as a photographed image.

In the present embodiment, an example will be described in which the imaging unit 30 starts capturing a moving image of the transport mechanism 40 when the transport mechanism 40 starts to be driven. Driving the transport mechanism 40 means transporting the vehicle V supported by the support member 40C included in the transport mechanism 40 in the transport direction X.

For example, when the photographing unit 30 receives a photographing start signal by a user's instruction to operate the photographing start button of the photographing unit 30, etc., the photographing unit 30 starts photographing the photographing range R. When the transport mechanism 40 starts to be driven, the user only has to issue an instruction to operate the photographing start button of the photographing section 30 or the like.

Note that the photographing unit 30 may start photographing a moving image in the photographing range R when detecting that the transport mechanism 40 is driven. In this case, the photographing section 30 and the drive section of the transport mechanism 40 may be communicably connected, and the transport mechanism 40 may transmit a photographing start signal to the photographing section 30 when driving is started. Further, the photographing unit 30 may start photographing in response to a photographing start signal received from the object determination device 10 or the sequencer 20.

Next, the object determination device 10 will be explained. The object determination device 10 is an information processing device that determines the type of object transported by the transport mechanism 40.

The type of object is information representing each group in which objects are classified into a plurality of groups according to predetermined classification criteria. As described above, in this embodiment, a case in which the target object is a vehicle V will be described as an example. When the object is a vehicle V, the type of the vehicle V is, for example, the type of the vehicle V, the model of the vehicle V, the model year of the vehicle V, etc., but is not limited to these. The type of vehicle V represents, for example, a regular car, a small car, a special purpose car, or the like. The vehicle type of vehicle V is a vehicle name specified by a vehicle manufacturing company or the like. In this embodiment, the description will be made assuming that the type of vehicle V is a car name.

First, the hardware configurations of the object determination device 10 and the sequencer 20 will be explained.

FIG. 2 is a hardware configuration diagram of an example of the object determination device 10 and the sequencer 20.

The object determination device 10 and the sequencer 20 have a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I/F 11D, etc. connected to each other by a bus 11E, and are configured as a normal computer. This is a hardware configuration that uses .

The CPU 11A is a calculation device that controls the object determination device 10 and the sequencer 20 of this embodiment. The ROM 11B stores programs and the like that implement various processes by the CPU 11A. The RAM 11C stores data necessary for various processing by the CPU 11A. I/F11D is an interface for transmitting and receiving data.

The program executed by the object determination device 10 and the sequencer 20 of this embodiment is provided by being pre-installed in the ROM 11B or the like. Note that the program executed by the object determination device 10 and the sequencer 20 of this embodiment is a file in a format that can be installed or executable in the object determination device 10 and the sequencer 20, and is stored on a CD-ROM, a flexible disk (FD), The information may be provided by being recorded on a computer-readable recording medium such as a CD-R or a DVD (Digital Versatile Disk).

Next, the functional configuration of the object determination device 10 will be explained.

FIG. 3 is a functional block diagram of an example of the object determination device 10. FIG. 3 also shows configurations other than the object determination device 10 included in the object determination system 1.

The object determination device 10 includes a processing section 12, a communication section 14, a storage section 16, and a display section 18. The processing unit 12, communication unit 14, storage unit 16, and display unit 18 are communicably connected via, for example, a bus 19.

The communication unit 14 is communicably connected to each of the sequencer 20, the tag 60, and the photographing unit 30. That is, the communication section 14 is a communication interface for communicating with each of the sequencer 20, the tag 60, and the photographing section 30. The storage unit 16 stores various information. In this embodiment, the storage unit 16 stores the learning model 16A in advance. Details of the learning model 16A will be described later.

The display unit 18 displays various information. The display unit 18 is, for example, a display that displays images. Note that the object determination device 10 may be configured to further include a speaker that outputs audio. Further, the object determination device 10 may be configured to further include an input device such as a keyboard or a pointing device that receives operation input from a user.

The processing unit 12 executes various information processing. In this embodiment, the processing unit 12 includes a reception unit 12A, a tag determination unit 12B, an acquisition unit 12C, an extraction unit 12D, a determination unit 12E, a learning unit 12F, and an output control unit 12G. Some or all of the reception unit 12A, tag determination unit 12B, acquisition unit 12C, extraction unit 12D, determination unit 12E, learning unit 12F, and output control unit 12G, for example, cause a processing device such as a CPU to execute a program. That is, it may be realized by software. Further, some or all of these functional units may be realized by hardware such as an IC (Integrated Circuit). Further, some or all of these functional units may be realized using a combination of software and hardware.

At least one of these functional units may be installed in an external information processing device that is communicably connected to the object determination device 10 via a network or the like. Alternatively, at least part of the data stored in the storage unit 16 may be stored in an external information processing device that is communicably connected to the object determination device 10 via a network or the like.

The reception unit 12A receives a determination start signal from the sequencer 20. As described above, the sequencer 20 transmits a determination start signal to the object determination device 10 when the transported vehicle V reaches the predetermined position X1. The reception unit 12A receives a determination start signal from the sequencer 20 via the communication unit 14.

The tag determination unit 12B reads the cart ID from the tag 60 via the communication unit 14 when the reception unit 12A receives the determination start signal. The tag determination unit 12B reads the cart ID stored in the tag 60 through wireless communication between the communication unit 14 and the tag 60. The tag determination unit 12B determines the type of vehicle V supported by the support member 40C identified by the read truck ID. For example, the storage unit 16 stores in advance cart management information in which the cart ID and the type of vehicle V are associated with each other. The tag determination unit 12B determines the type of vehicle V by reading the type of vehicle V corresponding to the truck ID received from the tag 60 from the truck management information. In the trolley management information, information that matches the type of vehicle V actually being transported through the transport mechanism 40 and the trolley ID of the support member 40C that supports the vehicle V is registered in advance according to an operation instruction by the user. It is assumed that there is

When the tag determination unit 12B successfully determines the type of vehicle V using the trolley ID stored in the tag 60, it outputs the determination result representing the type of vehicle V to the output control unit 12G, which will be described later.

Here, an abnormality in reading the trolley ID may occur due to dirt on the tag 60, shaking of the tag 60, shaking of the support member 40C, communication error, or the like. Further, the type of vehicle V corresponding to the bogie ID read from the tag 60 may not be included in the bogie management information. In such a case, the tag determination unit 12B fails to determine the type of vehicle V using the trolley ID stored in the tag 60. If the determination of the type of vehicle V using the trolley ID fails, the tag determination unit 12B outputs a determination start signal to the acquisition unit 12C.

When the acquisition unit 12C receives the determination start signal from the tag determination unit 12B, the acquisition unit 12C acquires the photographed image 70 photographed by the photographing unit 30. The acquisition unit 12C acquires the photographed image 70 photographed by the photographing unit 30 when the reception unit 12A receives the determination start signal from the sequencer 20. Specifically, the acquisition unit 12C selects one of the plurality of captured images 70 included in the moving image captured by the imaging unit 30 that was captured by the imaging unit 30 when the reception unit 12A received the determination start signal from the sequencer 20. A photographed image 70 is acquired.

In addition, in this embodiment, the processing part 12 of the object determination apparatus 10 demonstrates the structure provided with the tag determination part 12B as an example. However, the processing section 12 may have a configuration that does not include the tag determination section 12B. In this case, the acquisition unit 12C may acquire the photographed image 70 photographed by the photographing unit 30 when the reception unit 12A receives the determination start signal from the sequencer 20.

As described above, the photographing range R of the photographing unit 30 is adjusted in advance so as to include the specific part P of the vehicle V that has arrived at the predetermined position X1 due to transportation. Further, the sequencer 20 transmits a determination start signal to the object determination device 10 when the vehicle V reaches the predetermined position X1 due to transportation. Therefore, by acquiring the photographed image 70 photographed by the photographing section 30 when the determination start signal is received from the sequencer 20, the acquisition section 12C obtains the photographed image 70 of the vehicle V being transported. A photographed image 70 of a photographing range R including a plurality of parts P whose types can be specified is acquired.

The extraction unit 12D extracts a plurality of regions of interest from the captured image 70 acquired by the acquisition unit 12C.

FIG. 4 is a schematic diagram of an example of a photographed image 70. FIG. 4 shows an example of a captured image 70 acquired by the acquisition unit 12C. The extraction unit 12D extracts a plurality of regions of interest 72 from the photographed image 70.

The region of interest 72 is an area in which the type of vehicle V included in the photographed image 70 can be specified. The extraction unit 12D extracts a plurality of regions of interest 72, at least some of which are non-overlapping, from the photographed image 70.

It is only necessary that a member that can identify the type of vehicle V is reflected in each of the plurality of regions of interest 72 . In detail, it is preferable that the plurality of regions of interest 72 include mutually different parts P of the vehicle V reflected in the photographed image 70. The part P is a member that allows the type of the vehicle V to be determined, as described above.

In the present embodiment, an example will be described in which the extraction unit 12D extracts a first region of interest 72A including the component P1 and a second region of interest 72B including the component P2. The part P1 and the part P2 are different parts P from which the type of the vehicle V can be determined. Note that the extraction unit 12D only needs to extract a plurality of regions of interest 72, and is not limited to extracting two regions of interest 72.

The extraction unit 12D stores in the storage unit 16 in advance, region-of-interest information indicating the position and size of each of the plurality of regions of interest 72 in the photographed image 70. In the present embodiment, the extraction unit 12D stores the first area of interest information and the second area of interest information in the storage unit 16 in advance as the area of interest information of the first area of interest 72A and the second area of interest 72B.

Specifically, for example, the extraction unit 12D stores first region of interest information indicating the position and size of the first region of interest 72A in the captured image 70 in the storage unit 16 in advance. Further, the extraction unit 12D stores second region of interest information indicating the position and size of the second region of interest 72B in the captured image 70 in the storage unit 16 in advance. Then, the extraction unit 12D extracts the position and size range represented by each of the first region of interest information and the second region of interest information read from the storage unit 16 in the captured image 70 acquired by the acquisition unit 12C. Then, the first region of interest 72A and the second region of interest 72B are extracted from the photographed image 70. That is, each time the acquisition unit 12C acquires a new captured image 70, the extraction unit 12D focuses on an area having the same position and size in the image among the plurality of captured images 70 sequentially acquired, based on the region of interest information. It is extracted as area 72.

Returning to FIG. 3, the explanation will be continued. The determination unit 12E uses the learning model 16A to determine the type of vehicle V from the plurality of regions of interest 72 (first region of interest 72A, second region of interest 72B) extracted by the extraction unit 12D.

The learning model 16A is a learning model that outputs the type of vehicle V included in the captured image 70 from a plurality of regions of interest 72 included in the captured image 70. In other words, the learning model 16A is a learning model that uses the plurality of regions of interest 72 as input and the type of vehicle V as output.

The learning unit 12F previously learns the learning model 16A by machine learning using teacher data. The learning unit 12F uses the captured image 70 acquired by the acquisition unit 12C as a learning image for learning, and uses the plurality of regions of interest 72 included in the learning image as teacher data.

FIG. 5 is an explanatory diagram of an example of the learning image 80. The learning section 12F uses, as the learning image 80, the photographed image 70 of the photographing range R of the vehicle V that has reached the predetermined position X1 by transportation, which is photographed by the photographing section 30. The learning unit 12F then learns the learning model 16A using the plurality of regions of interest 72 and the plurality of processing regions of interest 74 included in the learning image 80.

Specifically, the learning unit 12F uses the first region of interest 72A and the second region of interest 72B included in the learning image 80 as input data of teacher data consisting of input data and correct answer data.

Specifically, the learning unit 12F extracts a region in the learning image 80 whose position and size are defined by the first region of interest information as the first region of interest 72A. Furthermore, the learning unit 12F extracts an area in the learning image 80 at a position and size defined by the second area of interest information as a second area of interest 72B. The learning unit 12F then uses the pair of the first region of interest 72A and the second region of interest 72B extracted from the learning image 80 as input data of teacher data.

Further, the learning unit 12F uses the types of vehicles V reflected in the first region of interest 72A and the second region of interest 72B as correct answer data for the input data of the teacher data.

Further, the learning unit 12F extracts, as a processed region of interest 74, a region in which at least one of the position and angle of the region of interest 72 is changed for each of the plurality of regions of interest 72 included in the learning image 80.

Specifically, the learning unit 12F extracts one or more first processed regions of interest 74A in which at least one of the position and angle of the first region of interest 72A included in the learning image 80 is changed. In FIG. 5, a first processing region of interest 74A1 to a first processing region of interest 74A2 in which the position of the first region of interest 72A is changed, and a first processing region of interest 74A3 in which the angle of the first region of interest 72A is changed are extracted. An example of this is shown below. The learning unit 12F may extract one or more first processing regions 74A in which at least one of the position and angle of the first region of interest 72A is changed, and may extract three first processing regions 74A. Not limited.

The learning unit 12F determines the first point of interest in the range of the position and angle deviation of the vehicle V in real space with respect to the photographing range R of the photographing unit 30, which is caused by shaking of the vehicle V that has been transported and reached the predetermined position X1. A plurality of first processing target regions 74A are extracted by changing the position and angle of the region 72A. However, the learning unit 12F adjusts the extraction position and angle so that each of the plurality of extracted first processing regions 74A becomes a region including the part P1.

The learning unit 12F also extracts one or more second processed regions of interest 74B in which at least one of the position and angle of the second region of interest 72B included in the learning image 80 is changed. In FIG. 5, a second processing region of interest 74B1 to a second processing region of interest 74B2 in which the position of the second region of interest 72B is changed, and a second processing region of interest 74B3 in which the angle of the second region of interest 72B is changed are extracted. An example of this is shown below. Note that the learning unit 12F only needs to extract one or more second processing regions 74B in which at least one of the position and angle of the second region of interest 72B is changed, and may extract three second processing regions 74B. Not limited.

Note that the learning unit 12F determines the second focus in the range of the position and angle deviation of the vehicle V in real space with respect to the photographing range R of the photographing unit 30, which is caused by shaking of the vehicle V that has been transported and reached the predetermined position X1. A plurality of second processing target regions 74B are extracted by changing the position and angle of the region 72B. However, the learning unit 12F adjusts the extraction position so that each of the plurality of extracted second processing target regions 74B becomes a region including the part P2.

The learning unit 12F then selects one of the first region of interest 72A and the plurality of first processing regions 74A extracted from the learning image 80, the second region of interest 72B extracted from the learning image 80, and one of the plurality of first processing regions 74A and 2. A plurality of input data with different pair combinations of one of the processing target areas 74B and 2 are prepared. The learning unit 12F then generates pairs of these plurality of input data and the type of vehicle V, which is correct data for each of the plurality of input data, as a plurality of teacher data.

In this manner, the learning unit 12F generates a plurality of pieces of teacher data using a plurality of different combinations of regions of interest 72 and regions of interest to be processed 74 as input data from one learning image 80. Similarly, when using a plurality of learning images 80, the learning unit 12F uses a plurality of regions of interest 72 and processed regions of interest 74 in different combinations as input data from each of the plurality of learning images 80. Generate multiple training data.

The learning unit 12F then selects a first region of interest 72A or a first processing region of interest 74A, a second region of interest 72B or a second processing region of interest 74B, which are input data included in each of the plurality of teacher data. A learning model 16A is trained in which the pair of , is input, and the type of vehicle V, which is output data corresponding to the input data included in the teacher data, is output.

Known machine learning may be used for learning the learning model 16A. For example, deep learning such as CNN (Convolutional Neural Network) may be used for machine learning.

In this way, the learning unit 12F processes the plurality of regions of interest 72 extracted from the learning image 80 and the plurality of processed regions of interest 74 in which at least one of the position and angle of the region of interest 72 in the learning image 80 is changed. The learning model 16A is learned using the learning model 16A. Therefore, the learning unit 12F can efficiently and highly accurately learn the learning model 16A from a small number of captured images 70.

The learning unit 12F also determines the position and angle of the region of interest 72 in the learning image 80 within the photographing range R relative to the vehicle V in real space, which is caused by the shaking of the vehicle V being transported. A plurality of regions of interest 74 to be processed are extracted, which are changed within the range of deviation. Then, these plural processing target regions 74 are used as teacher data. For this reason, the learning unit 12F can accurately identify the type of vehicle V even if the range reflected in the photographing range R of the vehicle V in real space shifts due to positional shift or shaking of the vehicle V being transported. The learning model 16A that can output the following can be learned.

Further, the learning unit 12F may learn the learning model 16A by transfer learning using a learned model. When performing transfer learning, the input data of the teacher data includes a plurality of regions of interest 72 extracted from the learning image 80 and a plurality of regions of interest 72 in which at least one of the position and angle of the region of interest 72 in the learning image 80 has been changed. The processing attention area 74 may be used.

By learning the learning model 16A by transfer learning, the learning unit 12F can efficiently learn the highly accurate learning model 16A in a short time with even less teacher data.

Note that the learning unit 12F may be configured to be provided in an information processing device outside the object determination device 10. In this case, the learning section 12F learns the learning model 16A using the above learning method with an information processing device external to the object determining device 10, and stores the learned learning model 16A in the storage section 16 of the object determining device 10. Bye.

Returning to FIG. 3, the explanation will be continued. The determination unit 12E inputs the first region of interest 72A and the second region of interest 72B extracted by the extraction unit 12D to the learning model 16A learned by the learning unit 12F. Then, the determination unit 12E determines the type of vehicle V by acquiring the type of vehicle V that is the output from the learning model 16A.

The output control section 12G outputs the determination result by the determination section 12E or the tag determination section 12B. The determination result is information representing the type of vehicle V.

The output control unit 12G outputs the type of vehicle V, which is the determination result, to the sequencer 20 via the communication unit 14.

Upon receiving the determination result of the type of vehicle V from the object determination device 10, the sequencer 20 transmits the determination result to the coating device 50. The painting device 50 executes a painting process according to the type of vehicle V represented by the determination result received from the sequencer 20. For example, the painting device 50 performs a painting process on a specific area of the vehicle V according to the type of vehicle V received from the sequencer 20. Note that the object determination system 1 only needs to be provided with a processing device that performs various processes according to the determination result of the type of vehicle V, and is not limited to a form in which a coating device 50 that performs painting is provided. Furthermore, the object determination system 1 may have a configuration in which a device such as the painting device 50 that performs processing according to the type of vehicle V is not provided.

Further, the output control unit 12G may display the type of vehicle V, which is the determination result, on the display unit 18. In this case, the output control unit 12G can visually provide the user with the type of vehicle V that has been transported and reached the predetermined position X1.

Next, an example of the flow of information processing executed by the object determination device 10 of this embodiment will be described.

FIG. 6 is a flowchart illustrating an example of the flow of information processing executed by the object determination device 10.

The reception unit 12A receives a determination start signal from the sequencer 20 (step S100).

When the tag determination unit 12B receives the determination start signal in step S100, it reads the cart ID from the tag 60 via the communication unit 14 (step S102). The tag determination unit 12B determines whether or not the determination of the type of vehicle V using the trolley ID read in step S102 has failed (step S104). The tag determination unit 12B determines the type of vehicle V by reading the type of vehicle V corresponding to the truck ID read in step S102 from the truck management information. When the tag determination unit 12B successfully determines the type of vehicle V using the trolley ID, it makes a negative determination in step S104 (step S104: No), and proceeds to step S112, which will be described later. If an affirmative determination is made in step S104 (step S104: Yes), the process advances to step S106.

In step S106, the acquisition unit 12C acquires the captured image 70 (step S106). The acquisition unit 12C acquires the photographed image 70 photographed by the photographing unit 30 when the determination start signal is received in step S100. Through the process of step S106, the acquisition unit 12C acquires the photographed image 70 of the photographing range R of the vehicle V that has been transported and reached the predetermined position X1.

The extraction unit 12D extracts a plurality of regions of interest 72 from the captured image 70 acquired in step S106 (step S108). For example, the extraction unit 12D extracts the first region of interest 72A and the second region of interest 72B from the photographed image 70.

The determination unit 12E uses the learning model 16A to determine the type of vehicle V from the plurality of regions of interest 72 (first region of interest 72A, second region of interest 72B) extracted in step S108 (step S110).

The output control unit 12G outputs the determination result in step S110 or the determination result received by making a negative determination in step S104 (step S104: No) (step S112). For example, the output control unit 12G outputs the type of vehicle V, which is the determination result, to the sequencer 20 via the communication unit 14. For example, the sequencer 20 transmits the determination result to the coating device 50. The painting device 50 executes a painting process according to the type of vehicle V represented by the determination result received from the sequencer 20.

Next, the processing unit 12 determines whether to end the object determination process (step S114). For example, the processing unit 12 makes the determination in step S114 by determining whether or not an instruction signal to end the object determination process has been received. For example, when terminating the process of determining the type of vehicle V, the user inputs an instruction signal for terminating the object determination process by operating the operation unit of the object determination device 10 or the like. When the processing unit 12 receives the termination instruction signal, it may determine that the object determination process is to be terminated. If a negative determination is made in step S114 (step S114: No), the process returns to step S100. If an affirmative determination is made in step S114 (step S114: Yes), this routine ends.

As explained above, the object determination device 10 of this embodiment includes the extraction section 12D and the determination section 12E. The extraction unit 12D extracts a plurality of regions of interest 72 from the photographed image 70 of the object to be transported. The determination unit 12E performs extraction using a learning model 16A that outputs the type of object from the plurality of regions of interest 72, which is learned using the plurality of regions of interest 72 included in the learning image 80 which is the photographed image 70. The type of object is determined from the plurality of regions of interest 72 thus obtained.

Here, conventionally, the type of the object has been determined by reading an identification code provided on the object to be transported or the transport mechanism of the object using a sensor. However, in the prior art, it may be difficult to determine the type of object due to dirt on the part where the identification code is installed or shaking of the object being transported. Further, due to deterioration of a support member that supports the transported object, the transported object may shift or shake, which may cause an error in reading the identification code. For this reason, in the prior art, it may be difficult to accurately determine the type of object to be transported.

On the other hand, the object determination device 10 of the present embodiment extracts a plurality of regions of interest 72 from a photographed image 70 of a vehicle V being transported, and inputs the extracted plurality of regions of interest 72 to the learning model 16A. The type of object such as the vehicle V is determined as the output from the model 16A.

Therefore, in the object determination device 10 of the present embodiment, it is possible to suppress a decrease in the accuracy of determining the type of object due to an abnormality in reading the identification code. Furthermore, since the object determination device 10 of the present embodiment uses a plurality of regions of interest 72 included in the photographed image 70 to determine the type of object, the object determination device 10 has a higher It is possible to accurately determine the type of object.

Therefore, the object determination device 10 of this embodiment can determine the type of the object to be transported with high accuracy.

Furthermore, in the conventional technology, when an error occurs in reading the identification code provided on the object to be transported or the transport mechanism for the object, the transport of the object by the transport mechanism 40 is stopped, and the user manually detects the abnormality. After carrying out work to resolve the issue, work was carried out to restart transport again. On the other hand, in the object determination device 10 of the present embodiment, even if an abnormality occurs in the reading of the cart ID read from the tag 60, for example, a plurality of regions of interest extracted from the photographed image 70 of the object to be transported 72 and the learning model 16A to determine the type of object. Therefore, in addition to the above-mentioned effects, the object determination device 10 of the present embodiment can suppress production line stoppage due to reading abnormalities and the like, and can also reduce the user's workload.

Note that a program for executing each of the above processes executed by the object determination device 10 of the above embodiment may be stored in the HDD. Moreover, the program for executing each of the above-mentioned processes executed by the object determination device 10 of the embodiment described above may be provided by being incorporated in the ROM 11B in advance.

Further, the program for executing the above processing executed by the object determination device 10 of the embodiment described above is a file in an installable format or an executable format and can be stored on a CD-ROM, CD-R, memory card, or DVD (Digital). The program may be stored in a computer-readable storage medium such as a flexible disk (FD) or a flexible disk (FD), and provided as a computer program product. Further, a program for executing the above processing executed by the object determination device 10 of the above embodiment is stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Good too. Further, a program for executing the above processing executed by the object determination device 10 of the above embodiment may be provided or distributed via a network such as the Internet.

Although the embodiments of the present invention have been described above, the embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. This embodiment and its modifications are within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.

1 Object determination system 10 Object determination device 12D Extraction section 12E Determination section 30 Photographing section

Claims (4)

an extraction unit that extracts a plurality of regions of interest from a photographed image of the object to be transported;
The plurality of points of interest are extracted using a learning model that outputs the type of the object from the plurality of regions of interest, which are learned using the plurality of regions of interest included in the learning image that is the photographed image. a determination unit that determines the type of the object from the region;
Equipped with
The target object is a vehicle,
The plurality of regions of interest are regions including mutually different parts of the vehicle,
Object determination device. an extraction unit that extracts a plurality of regions of interest from a photographed image of the object to be transported;
The plurality of points of interest are extracted using a learning model that outputs the type of the object from the plurality of regions of interest, which are learned using the plurality of regions of interest included in the learning image that is the photographed image. a determination unit that determines the type of the object from the region;
Equipped with
The target object is a vehicle,
The plurality of regions of interest are regions including mutually different parts of the vehicle,
The determination unit includes:
Using the learning model that is trained using the plurality of regions of interest included in the learning image and a plurality of processed regions of interest in which at least one of the position and angle of the region of interest in the learning image is changed. determining the type of the object from the plurality of extracted regions of interest;
Object determination device. a photography department that photographs the object being transported;
an extraction unit that extracts a plurality of regions of interest from the photographed image of the object;
The plurality of points of interest are extracted using a learning model that outputs the type of the object from the plurality of regions of interest, which are learned using the plurality of regions of interest included in the learning image that is the photographed image. a determination unit that determines the type of the object from the region;
Equipped with
The target object is a vehicle,
The plurality of regions of interest are regions including mutually different parts of the vehicle,
Object recognition system. a photography department that photographs the object being transported;
an extraction unit that extracts a plurality of regions of interest from the photographed image of the object;
The plurality of points of interest are extracted using a learning model that outputs the type of the object from the plurality of regions of interest, which are learned using the plurality of regions of interest included in the learning image that is the photographed image. a determination unit that determines the type of the object from the region;
Equipped with
The target object is a vehicle,
The plurality of regions of interest are regions including mutually different parts of the vehicle,
The determination unit includes:
Using the learning model that is trained using the plurality of regions of interest included in the learning image and a plurality of processed regions of interest in which at least one of the position and angle of the region of interest in the learning image is changed. determining the type of the object from the plurality of extracted regions of interest;
Object recognition system.

Images