JP6773829B2 - Object recognition device, object recognition method, and object recognition program - Google Patents

Description

The present invention relates to a technique for recognizing an object by detecting a part of a predetermined object from measurement data such as an image.

Research is being actively conducted to detect multiple parts of a person appearing in a photographed image based on machine learning.

For example, in the technique described in Non-Patent Document 1 below, a large number of learning images showing a person are used as input values, and an annotation describing the type and position of a person's part in the learning image is used as an output value target value. Let the model be deeply trained. Then, by inputting the photographed image into the trained model, the type and position of the part of the person appearing in the photographed image are output. This annotation is created for the part appearing in the learning image. By the way, the information of each part described in the annotation and the information of each part output by the trained model are called key points.

If the parts necessary for various recognitions about a person can be detected, it becomes possible to recognize the existence area, the proportion-based adult or the child (attribute), and the like, in addition to the posture recognition, for the person.

“Realtime Multi-Person 2D Pose Optimization using Part Affinity Fields.”, Z. Cao, T. Simon, S. Wei and Y. Sheikh (2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302-1310)

However, in the prior art, there is a problem that it is difficult to recognize the posture, existence area, attribute, etc. of the object if there is concealment because the accuracy of estimating the portion that does not appear in the captured image is low.

For example, when a person's waist is hidden by an object such as a table and the upper body is taken above the top plate of the table and the legs are taken below the top plate, the upper body is input to the trained model generated by the conventional technique. And either both the key points of the legs are not detected, only one is detected, or the key points of the upper body and the legs are detected separately (that is, the upper body and the legs are not detected as parts of the same person). turn into.

Therefore, when the existing area of a person is recognized based on the detection result, it is recognized that there is no existing area, the existing area for one person, and the existing area for two people, and it is difficult to recognize with high accuracy. In addition, it is difficult to recognize postures and attributes because only the position of a part of a person can be specified from one existing area.

As described above, in the prior art, since the relationship between the learning image and the portion appearing in the image is learned, it is difficult to detect the portion not appearing in the captured image. Therefore, in the prior art, it may be difficult to recognize the posture, the existing area, the attribute, etc. if there is concealment.

Further, the above problem occurs not only in the two-dimensional measurement data (image) but also in the three-dimensional measurement data, and also in the time series of the two-dimensional measurement data and the time series of the three-dimensional measurement data.

The present invention has been made in view of the above problems, and even if a part of a part is concealed and an object is measured, the object recognition device and the object can recognize the object by complementing the concealed part. An object of the present invention is to provide an object recognition method and an object recognition program.

(1) The object recognition device according to the present invention is a device that detects a plurality of detection-required parts constituting the object from the measurement data obtained by measuring the predetermined object, and is a device for detecting a plurality of detection-required parts of the object. About the sample Using the given data including the position of the detection-required site given for each sample, the deterioration data in which some of the said positions are omitted from the given data is input, and the original given data is the output target. The complement storage means that stores the complement generated in advance by learning as a value, and one or more of the positions of the detection-required sites in the object are detected from the measurement data, and the object A site detecting means for acquiring the detection data of the above, and a site complementing means for inputting the detection data into the complement and complementing the position of the detection-required site lacking in the detection data to generate the complemented detection data. And.

(2) Another object recognition device according to the present invention uses measurement data to detect an object area in which a predetermined object is measured, which is a plurality of parts constituting the object and detects the object area. It is a device that detects based on the detection-required part determined to be necessary, and provides learning-giving data including the position of the detection-required part assigned to each of the plurality of samples of the object in the learning data. It stores the complementer generated in advance by learning using the deterioration data in which some of the positions are omitted from the training grant data as input and the original training grant data as the output target value. The complement storage means, the site detection means that detects one or more of the positions of the detection-required sites in the object from the measurement data and acquires the detection data of the object, and the detection data. The site complementing means that inputs to the complementer and complements the position of the detection-required site that is lacking in the detection data to generate the complemented detection data, and the position of the detection-required site indicated by the complemented detection data. The object region detecting means for detecting a predetermined range as a reference as the object region in the measurement data is provided.

(3) Still another object recognition device according to the present invention is a plurality of parts constituting a person who is an object and estimates whether or not the person measured in the measurement data is a child. It is an apparatus that performs based on the detection-required site determined to be necessary, and uses the grant data including the position of the detection-required site assigned to each of the samples for a plurality of samples of the object, and one from the grant data. The complement storage means that stores the complement generated in advance by learning with the deterioration data lacking the position of the unit as the input and the original added data as the output target value, and the measurement data A site detection means that detects one or more of the positions of the detection-required sites on the object and acquires the detection data of the object, and inputs the detection data to the complement, and the detection data is insufficient. The measurement is performed by comparing the ratio of the distance between the detection-required site in the complemented detection data and the site-complementing means that complements the position of the detection-required site to generate the complemented detection data with a predetermined reference. It is provided with a child estimation means for estimating whether or not the person measured in the data is a child.

(4) In the object recognition device according to the above (1) to (3), the site complementing means obtains the detection data for two or more objects from the measurement data of one site detecting means. When acquired, the existence area for each object is estimated from the position of the detection-required part output by the complement, and the degree of overlap between the existence areas for the plurality of detection data is equal to or higher than a predetermined reference value. In this case, it is possible to generate the complemented detection data integrated with respect to the plurality of detection data as being due to the same object.

(5) In the object recognition device according to (4) above, when the degree of overlap between the existing regions with respect to the plurality of detected data is equal to or higher than a predetermined reference value, the plurality of site complementing means. The position of the same portion obtained by the complementer can be integrated into one for each of the detected data to generate the complemented detection data.

(6) In the object recognition device according to (4) above, when the degree of overlap between the existing regions with respect to the plurality of detected data is equal to or higher than a predetermined reference value, the plurality of site complementing means. By integrating the detection data and inputting it to the complementer, the complemented detection data can be generated.

(7) The object recognition method according to the present invention is a method of detecting a plurality of detection-requiring sites constituting the object from the measurement data obtained by measuring the predetermined object, and is a method of detecting a plurality of detection-required parts of the object. About the sample Using the given data including the position of the detection-required site given for each sample, the deterioration data in which some of the said positions are omitted from the given data is input, and the original given data is the output target. Acquire the detection data of the object by detecting one or more of the positions of the detection-required parts in the object from the measurement data and the step of preparing a complement generated in advance by learning to use the value. A site complementing step is provided in which the detection data is input to the complementer and the position of the detection-required site lacking in the detection data is complemented to generate the complemented detection data.

(8) The object recognition program according to the present invention is a program that causes a computer to perform a process of detecting a plurality of detection-required parts constituting the object from the measurement data obtained by measuring the predetermined object. The computer uses the given data including the positions of the detection-required sites given to each of the plurality of samples of the object, and inputs the deterioration data in which some of the said positions are omitted from the given data. , One of the complement storage means that stores the complement generated in advance by learning using the original added data as the output target value, and the position of the detection-required portion in the object from the measurement data. The site detecting means for detecting the above and acquiring the detection data of the object, and the detection data are input to the complement to complement the position of the detection-required site lacking in the detection data. It functions as a site complementing means for generating complemented detection data.

According to the object recognition device, the object recognition method, and the object recognition program of the present invention, even if a part of the part is concealed and the object is measured, the concealed part is complemented to recognize the object. It becomes possible to do.

It is a figure which shows the schematic structure of the object recognition apparatus which concerns on embodiment of this invention. It is a schematic functional block diagram concerning the learning stage of the object recognition apparatus which concerns on embodiment of this invention. It is a schematic diagram explaining the example of the given data. It is a schematic functional block diagram concerning the recognition stage of the object recognition apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the example of the photographed image and the detection data. It is a schematic diagram explaining the outline of the integration process of the detection data about the person on the right side of FIG. It is a flow chart about the learning stage of the object recognition apparatus which concerns on embodiment of this invention. It is a flow chart about the recognition stage of the object recognition apparatus which concerns on embodiment of this invention. It is a schematic flow chart of the duplication detection determination processing by a key point complementing means. It is a flow chart of the example which the integrated processing is different with respect to the recognition stage of the object recognition apparatus which concerns on embodiment of this invention.

Hereinafter, the object recognition device 1 according to the embodiment of the present invention (hereinafter referred to as the embodiment) will be described with reference to the drawings. The object recognition device according to the present invention detects the positions of a plurality of parts constituting the predetermined object from the measurement data, and based on the result, the area of the object (object area) in the measurement data. And attributes (object attributes) are obtained, and the object recognition device 1 shown as an example in the present embodiment detects a part and a region of a person appearing in the surveillance space from a photographed image of the surveillance space. Furthermore, it is estimated whether or not the person is a child. That is, in the present embodiment, the measurement data is a two-dimensional image, and the object is a person. The object recognition device 1 detects the positions of a plurality of parts constituting a person in a two-dimensional image, detects a region surrounding the parts, and estimates whether or not the child is a child based on the ratio of the distances between the parts.

A plurality of parts used for object recognition are referred to as detection-required parts, and representative points of the detection-required parts are referred to as key points. The key point information is represented by at least a combination of the type and position of the corresponding part, and the data including this combination is referred to as part data. Then, by detecting each key point, the position of the corresponding detection-required portion is detected. The type of the part to be detected is predetermined according to the object and the purpose of recognition.

In particular, the object recognition device 1 stores a complementer that complements the hidden portion learned by using the annotation (assigned data) of the portion appearing in the learning image. Here, the given data is the part data given to the object appearing in the measurement data for learning, the three-dimensional model of the object, and the like. The object recognition device 1 detects a detection-required portion from a captured image by, for example, a conventional detector to generate detection data, and complements the portion data of the detection-required portion that has failed to be detected by the complementer. Then, the object recognition device 1 detects the object area using the complemented detection data (complemented detection data), and determines the object attribute.

[Configuration of object recognition device 1]
FIG. 1 is a block diagram showing a schematic configuration of an object recognition device 1. The object recognition device 1 includes a photographing unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and an output unit 6.

The photographing unit 2 is a measuring unit that acquires measurement data, and is a surveillance camera in the present embodiment. The photographing unit 2 is connected to the image processing unit 5 via the communication unit 3, photographs the monitoring space at predetermined time intervals to generate a photographed image, and sequentially inputs the photographed images to the image processing unit 5. For example, the photographing unit 2 is installed on a pole installed in a corner of the event venue, which is a monitoring space, with a predetermined fixed field of view overlooking the monitoring space, and the monitoring space is photographed with a frame period of 1 second and colored. Generate an image. The photographing unit 2 may generate a monochrome image instead of the color image.

The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5 and the other end of which is connected to the photographing unit 2 and the output unit 6. The communication unit 3 acquires a photographed image from the photographing unit 2 and inputs it to the image processing unit 5, inputs the recognition result of the object from the image processing unit 5, and outputs it to the output unit 6.

The photographing unit 2, the communication unit 3, the storage unit 4, the image processing unit 5, and the output unit 6 are appropriately connected in a form according to the installation location of each unit. For example, when the photographing unit 2, the communication unit 3, and the image processing unit 5 are installed remotely, the photographing unit 2 and the communication unit 3 can be connected by an internet line. Further, the communication unit 3 and the image processing unit 5 can be connected by a bus. In addition, a LAN (Local Area Network), various cables, or the like can be used as the connection means.

The storage unit 4 is a memory device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and stores various programs and various data. For example, the storage unit 4 stores the learning image, the data given to the learning image, and the information of the detector and the complementer which are the learned models. The storage unit 4 is connected to the image processing unit 5 and inputs / outputs these information to and from the image processing unit 5. That is, information necessary for recognizing an object and information generated in the process of recognition processing are input / output between the storage unit 4 and the image processing unit 5.

The image processing unit 5 is a measurement data processing unit that processes measurement data, and is an arithmetic unit such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an MCU (Micro Control Unit), and a GPU (Graphics Processing Unit). It is composed. The image processing unit 5 operates as various processing means / control means by reading a program from the storage unit 4 and executing the program, reads various data from the storage unit 4 as necessary, and stores the generated data in the storage unit 4. Remember. For example, the image processing unit 5 generates a detector and a complementer by learning, and stores the generated detector and the complementer in the storage unit 4 via the communication unit 3. In addition, the image processing unit 5 uses a detector and a complement to perform processing for recognizing an object in the captured image.

The output unit 6 is a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like, and displays the recognition result input from the communication unit 3. The observer decides whether or not a countermeasure is necessary according to the displayed recognition result, and takes measures such as urgently rushing the responder as necessary.

Hereinafter, regarding the configuration of the object recognition device 1, first, the configuration related to the learning stage for learning the detector and the complementer will be described, and then the configuration for the recognition stage for recognizing the object using the detector and the complementer will be described. To do.

[Configuration of object recognition device 1 related to learning stage]
FIG. 2 is a schematic functional block diagram of the object recognition device 1 relating to the learning stage, in which the storage unit 4 functions as the learning data storage means 40, the detector storage means 41, and the complement storage means 42, and the image processing unit 5 Functions as the detector learning means 50, the degradation data generating means 51, and the complement learning means 52.

The learning data storage means 40 is a learning image storage means that stores a large number of learning images in advance, and also stores in advance the part data given to the person photographed in the learning image as the given data. It is a means of storing given data. The learning data storage means 40 is a learning image and each person photographed in the image (hereinafter, referred to as a sample. Another person is another sample, and even if they are the same person, if the images are different, they are different samples. ) Is linked and held. Specifically, sample IDs for identifying each other are assigned to each sample, image IDs are assigned to the learning images, and the correspondence between these IDs is stored in the learning data storage means 40. The learning image does not have to be an actual image actually taken by the camera, and may be, for example, an image created by computer graphics (CG) or the like. The grant data includes type and position information for each key point of each sample. In addition, a key point whose position is unknown can be used as information indicating that fact. That is, from the assigned data, it is possible to know whether or not the position of the key point has been assigned and the assigned position for each of the plurality of key points of each sample. The given data may be created manually, may be created by a person checking the extracted data by a machine and modifying it as necessary, or may be a mixture thereof.

FIG. 3 is a schematic diagram illustrating an example of the given data. FIG. 3A is an example in which the topology of the key point of the object is plotted with 17 parts requiring derivation. It is illustrated by 17 white circles representing the positions of key points and 16 line segments representing the connection relationship between key points. FIG. 3B shows an example in which the given data is defined as a table format database. Each row in the table represents a record of grant data for each sample. In each record, the index n (n = 1, 2, ..., N) representing the sample ID is stored at the beginning (left side), and then the set of three values representing the key point information is used as the key point type. 17 sets of indexes i (i = 1, 2, ..., 17) corresponding to the correspondence are stored in ascending order.

The set of the above three values is the x-coordinate x _{n, i} , y-coordinate y _{n, i of} each key point, and the flag (assignment flag) v _{n, i} indicating whether or not the key point is missing. The values set in the assignment flags v _{n and i} are "1" if the coordinates are assigned and "0" if the coordinates are not assigned. In each set, the three values are stored in the order of x _{n, i} , y _{n, i} , v _{n, i} .

Since the position of the key point is represented by a relative position on the image, the deterioration data generation means 51 normalizes the position of the key point in the assigned data and then generates the deterioration data. For example, normalization can be performed by translating a key point whose assigned flag of each sample is 1, to a coordinate system with the center of the key point corresponding to both shoulders of the sample as the origin. .. By the way, in this case, the sample in which the grant flag of either the right shoulder or the left shoulder is 0 is not normalized, but the sample not normalized in this way is not used for the learning of the complementer. Good.

The detector learning means 50 generates a trained model as a detector by learning using a learning image as an input and the given data corresponding to the learning image as an output target value.

The detector may use any means as long as it outputs one or more detection-required parts by inputting an image. In the present embodiment, the method proposed in Non-Patent Document 1 is used as a detector, and learning is performed in advance using training data. Specifically, in this embodiment, the detector is modeled by CNN (Convolutional Neural Networks).

The detector storage means 41 stores information including the filter coefficient and network structure of the filters constituting the CNN as a detector.

In addition to the detectors in the object recognition device 1, various known detectors such as a detector that identifies the position of a key point based on posture estimation using silhouette matching of an object are applied. Can be done. For example, when a detector based on silhouette matching is applied, the detector storage means 41 stores a background image, a posture pattern, and the like necessary for detection.

The deterioration data generation means 51 reads the given data from the learning data storage means 40, and creates the deteriorated data by omitting a part of the key points of each sample in the given data. Then, the given data and the deterioration data are set and output to the complement learning means 52. For example, the deterioration data generation means 51 can create deterioration data by randomly or regularly selecting the key points to be deleted and replacing the positions of the selected key points with unknown values. The deterioration data generation means 51 creates deterioration data by replacing the x-coordinate, y-coordinate, and assignment flag of the selected key point with 0, respectively. However, the deterioration data generation means 51 leaves a predetermined number or more of key points for which the assignment flag is 1.

That is, the deteriorated data generation means 51 reads out the given data to which the key point positions exceeding the required number are given from the learning data storage means 40, omits one or more positions from the given data, and causes the required number or more. Generate degradation data including the location of. On the other hand, among the given data, the key points with the given flag of 1 are not used for learning the complementer. In this embodiment, the required number is one.

Further, in the present embodiment, the sample is sampled at a random angle centered on the point where the xy coordinate is (0,0) so that the posture of the person shown in the fisheye lens image or the spherical image can be appropriately complemented. Rotate and use for learning complements. That is, the deterioration data generation means 51 generates deterioration data after performing rotation processing on the normalized assigned data to convert the xy coordinates of the key points.

The complement learning means 52 learns the complement using the pair of the given data and the degraded data input from the degraded data generating means 51. That is, the complement learning means 52 generates a complement by learning with the deterioration data as an input and the given data as an output target value. Learning here is to find the parameters of the complementer.

In this embodiment, the complement is modeled with a variational autoencoder (VAE). The VAE is composed of a linear conversion process, an activation function, and the like, and here, the ReLU function is used as the activation function. In the present embodiment, the complement learning means 52 learns to minimize the error function for the parameters of each element constituting the VAE. As the error function, the square error between the key point coordinates obtained by inputting the deterioration data into the complementer and the key point coordinates of the assigned data is used. At this time, the key point whose grant flag of the grant data is 0 is not included in the error function. A stochastic steepest descent method is used for minimization.

The complement storage means 42 stores the parameters of the complement obtained by the complement learning means 52. Further, the complement storage means 42 stores the structure of the VAE used as the complement.

[Configuration of object recognition device 1 related to recognition stage]
FIG. 4 is a schematic functional block diagram of the object recognition device 1 regarding the recognition stage, in which the storage unit 4 functions as the detector storage means 41 and the complement storage means 42, and the image processing unit 5 is the key point detection means 53. It functions as a key point complementing means 54, an object area detecting means 55, and an object attribute estimating means 56, and a communication unit 3 cooperates with an image processing unit 5 to function as a captured image acquisition means 30 and a recognition result output means 31. ..

The captured image acquisition means 30 sequentially acquires captured images from the photographing unit 2 and outputs them to the image processing unit 5.

As described above, the detector storage means 41 stores the detector generated in the learning stage.

The key point detecting means 53 (site detecting means) uses the detector stored in the detector storage means 41 to input the captured images sequentially acquired from the photographing unit 2 which is the photographing means and appears in the image. Detects the key points of a person for each person. When a detector based on silhouette matching is applied, processing such as generation of a background image based on past captured images and extraction of silhouettes by background subtraction processing is involved. In the learning stage, the assigned data of each sample is identified by the sample ID, but in the recognition stage, the object ID is assigned to the detected data of each detected person for identification.

As described above, the complement storage means 42 stores the complement generated in the learning stage.

The key point complement means 54 complements each person's detection data input from the key point detection means 53 by using the learned complement stored in the complement storage means 42. The complemented detection data is referred to as complemented detection data. That is, the key point complementing means 54 complements the detection data of each object by inputting the detection data for each object detected by the key point detecting means 53 from the captured image into the complementer. It is a site complementing means for generating complemented detection data. The key point complementing means 54 calculates the coordinates of all the key points for each person.

Specifically, when the key point complementing means 54 inputs the part and the position of the key point detected from the key point detecting means 53 as detection data, the coordinates (x _{n, i)} are used as the position information for each part. And y _{n, i} ) and a set of three values consisting of the state value related to the position are defined. That is, in the recognition stage, a state value is assigned to each part of the detection data instead of the assignment flag in the learning stage. When the data format of the detection data is compared with the above-mentioned record of the given data, an index representing the object ID is stored instead of the sample ID in the given data, and x _{n, i} , y _n corresponding to each part. _{, i,} v _n, v n of the three values stored in the order of _i, the state value to _i can be the stored form.

Here, as the state value, "1" is given to the part where the position is detected by the key point detecting means 53, "0" is given to the part where the position is not detected, and the x-coordinate x _{n, i} is given by the complementary process. , Y coordinates y _{n, i} are set, and "2" is given to the part. That is, the position of the key point that was missing at the time of input to the key point complement means 54 is complemented, and the complemented detection data storing the position information consisting of the calculated coordinates and the state value “2” for the key point is complemented. Is generated. For the key points that have already been detected at the time of input, the value at the time of input is used as the position information in the complemented detection data in the present embodiment, but the coordinates of the output of the complement may be used instead. .. For the undetected part, 0 is set in x _{n, i} and y _{n, i} as information indicating an unknown value as in the case of deterioration data.

The key point complementing means 54 provides the generated complemented detection data to the object area detecting means 55 and the object attribute estimating means 56 together with the object ID of the detected data before complementing.

Here, since the position of the key point included in the detection data before completion is represented by a relative position on the image, the key point complement means 54 normalizes the detection data in the same manner as the deterioration data generation means 51. After that, the completion process is performed. Then, the position obtained by the completion process by the key point complement means 54 is subjected to a process of restoring the portion that has been translated by normalization before completion to obtain the position of the key point of the complemented detection data. .. Although there may be an object that cannot be normalized as described above, the detection data before complementation is output as it is as the complemented detection data for the object.

Further, the key point complementing means 54 may perform integrated processing on the complemented detection data. The intention of the integrated process will be described with reference to FIG. FIG. 5 is a schematic diagram showing an example of a captured image and detection data. FIG. 5A is a diagram in which the captured image and the schematic detection data are superimposed and displayed. Two people 60 and 61 are shown in the image, and the detected key points are represented by white circles. FIG. 5B shows the detection data for the persons 60 and 61 in a table format as shown in FIG. 3B. The detection data of FIG. 5B is given the above-mentioned state value by the key point complementing means 54, and the three values of the coordinates and the state value are set to 0 for the undetected portion.

In the example of FIG. 5, all 17 key points "1" to "17" are detected for the person 60, and the rectangle 62 of FIG. 5 (a) shows the circumscribed rectangle of the key point group. The whole body of the person 60 is detected integrally, and one detection data (object ID = 001) is obtained correspondingly. On the other hand, the person 61 has the waist area hidden by the desk, and the two key points "12" and "13" located on the waist cannot be detected. As a result, the detector has detected the key points "1" to "11" of the upper body of the person 61 and the key points "14" to "17" of the legs as separate objects. The rectangle 63 in FIG. 5A shows the circumscribed rectangle of the key point group detected from the upper body, and the rectangle 64 shows the circumscribed rectangle of the key point group detected from the leg. Correspondingly, for the person 61, the detection data from the rectangle 63 (object ID = 002) and the detection data from the rectangle 64 (object ID = 003) are obtained.

The integrated process is a process of combining a plurality of detection data detected from one person like the person 61 in this example into one detection data. For example, the key point complementing means 54 uses the key point position output by the complementer when the key point detecting means 53 acquires detection data for two or more objects from one captured image in the integrated process. The existence area for each object is estimated, and when the multiplicity between the existence areas for multiple detection data is equal to or higher than a predetermined reference value, the plurality of detection data are integrated as if they are the same object. Generate the completed complemented detection data. When the existing regions of the parts overlap to the extent that they can be regarded as the same object by the determination of the degree of overlap, the integration process described later is performed. For example, in the example of FIG. 5, the detection data having the object ID 001 is the target. The two detection data having the object ID of 002,003 are integrated because the degree of duplication with the detection data of the object IDs 002 and 003 is less than the reference value.

FIG. 6 is a schematic diagram illustrating an outline of an integrated process of two detection data for the person 61 of FIG. 6 (a) to 6 (c) are views in which key points of the complemented detection data and their circumscribed rectangles 70 to 72 are superimposed on the captured image, and FIG. 6 (a) is taken from the rectangle 63 of FIG. The complement result for the detection data (object ID = 002), FIG. 6 (b) shows the complement result for the detection data (object ID = 003) from the rectangle 64 of FIG. 5, and FIG. 6 (c) shows those 2 The result of integrated processing of the two complementary results is shown. Here, the white circles represent the detected key points and the black circles represent the complemented key points, as in FIG. In FIG. 6A, key points are added by complementation, and an extrinsic rectangle 70 enlarged from the rectangle 63 is obtained. On the other hand, in FIG. 6B, the normalization based on the key point of the shoulder could not be performed, so that the original detection data is not complemented and the original detection data is the complemented detection data as it is, and the same circumscribed rectangle 71 as the rectangle 64 is obtained. Has been done. In FIG. 6 (c), the key points detected in both the rectangles 63 and 64 are integrated to obtain the extrinsic rectangle 72 suitable for the person 61. FIG. 6D shows the detection data for the persons 60 and 61 in a table format. The data having the object ID of 001 is the detection data of the person 60 as in FIG. 5 (b). On the other hand, in the detection data of the person 61, the object ID is divided into two, 002 and 003 in FIG. 5 (b), but in FIG. 6 (d), one object ID is assigned 002. It is integrated with the detection data, and the coordinates and state value "2" obtained by the complementary process are stored for the key points "12" and "13".

The key point complementing means 54 calculates the degree of duplication of the complemented detection data in the integration process, and integrates the complemented detection data having the degree of duplication equal to or higher than a predetermined reference value. By appropriately designing the degree of duplication, the same person is detected separately without performing integrated processing between the detection data of different persons (for example, the detection data of the object IDs 001 and 002 in FIG. 5B). Integrated processing can be performed between the detected data (for example, the detected data whose object IDs are 002 and 003 in FIG. 5B).

The target area detecting means 55 receives the complemented detection data from the key point complementing means 54, and detects and detects a predetermined range based on the position of the detection-required portion indicated by the complemented detection data as the target area in the measurement data. The information of the object area is output to the recognition result output means 31. For example, the object area detecting means 55 detects the circumscribed rectangle circumscribing the key point group included in the complemented detection data of each person as the person area which is the object area related to the person. In the example of FIG. 5, the rectangle 62 of the person 60 is the person area.

Further, the circumscribing rectangle may be enlarged by a predetermined ratio to form a person area. That is, when setting the person area, margins are provided on the top, bottom, left, and right in consideration of the fact that the key point is given or detected slightly inside the true area and the detection error. Further, the ratio to each direction of up, down, left and right may be different depending on the definition of each key point and the estimation of the detection error of each key point.

Alternatively, the complemented detection data may be converted into the person area of the person by inputting the complemented detection data of each person into the converter that has learned the conversion from the part data of the person to the person area. The converter is a trained model generated by learning in which the added data of the sample of the object captured in the training image is input and the object area of the sample in the training image is the output target value. The storage unit 4 functions as a converter storage means (not shown) and stores the converter in advance. Then, the object area detecting means 55 reads out the converter from the converter storage means and uses it.

As described above, the object area detecting means 55 detects the circumscribed rectangle circumscribing the detection data complemented by the key point complementing means 54 as the object area in the measurement data, or further multiplys the circumscribed rectangle by a predetermined number of times. Detects the object area expanded to.

Further, the object area detecting means 55 inputs the detection data complemented by the key point complementing means 54 with the added data of the sample of the object captured in the learning image, and the object area of the sample in the learning image is input. Is input to the converter generated by the learning with the output target value as the output target value, and the object area in the measurement data is detected.

Since the key points hidden in the measurement data are complemented in the complemented detection data, there is a problem that an extremely small object area is detected due to hiding, and an object area related to one object due to hiding. It is possible to significantly reduce the problem that the data is detected in multiple parts.

The object attribute estimation means 56 estimates the object attribute based on the complemented detection data input from the key point complement means 54, and outputs the estimated object attribute information to the recognition result output means 31. In the present embodiment, the object attribute estimation means 56 compares the ratio of the distances between the key points in the complemented detection data with a predetermined reference to estimate whether or not the person measured in the measurement data is a child. It is a child estimation method. For example, the head distance from the key point on the crown to the key point on the neck is calculated, and the head distance is from the key point on the shoulder to the key point on the waist and the key point on the heel. Calculate the whole body distance by adding the distance of, and calculate the ratio of the whole body distance to the head distance. However, the combination of shoulder, waist, knee and heel may be the combination of right shoulder, right waist, right knee and right heel, the combination of left shoulder, left waist, right knee and left heel, or the combination of right and left. Calculate and average both combinations. The calculated ratio is compared with a predetermined threshold value, and if the ratio is below the threshold value, it is estimated as a child, and if the ratio exceeds the threshold value, it is estimated as an adult. If any of the key points of the crown, shoulders, hips, knees, and heels is missing in the complemented detection data, it is output that the estimation result is unknown. The above ratio is the so-called head and body, and utilizes the fact that children tend to have a lower head and body than adults.

Since the complemented detection data is complemented with key points hidden in the measurement data, the object attribute estimation means 56 uses the complemented detection data to remarkably fail to calculate the ratio due to hiding. Can be reduced. In particular, in the above-mentioned child estimation, the key points of the child's neck are often hidden, so the estimation using the complemented detection data is effective.

The recognition result output means 31 outputs the object area detected by the object area detecting means 55 and the object attribute estimated by the object attribute estimating means 56 to the output unit 6. For example, the recognition result output means 31 draws a rectangle representing the object area on the captured image, and if the attribute is "child", further generates an image in which the characters "child" are drawn in the vicinity of the rectangle, and the output unit 6 Output to. If the object area is not detected, the captured image may be output as it is, assuming that the recognition result is that there is no object.

[Operation of object recognition device 1]
Next, the operation of the object recognition device 1 will be described separately for the learning stage and the recognition stage.

[Operation of object recognition device 1 in the learning stage]
FIG. 7 is a schematic flow chart relating to the operation of the object recognition device 1 in the learning stage.

The object recognition device 1 performs an operation of learning the detector and the complementer prior to the operation of recognizing the object appearing in the captured image.

When the learning operation is started, the image processing unit 5 functions as the detector learning means 50, and reads the learning image and the given data for each sample from the learning data storage means 40 as the data used for learning the detector. (Step S100). Then, the detector learning means 50 performs detector learning using the learning image as an input and the given data corresponding to the learning image as an output target value (step S105), and the learned data generated by the learning is performed. The model is stored in the detector storage means 41 as a detector (step S110).

Further, the image processing unit 5 functions as a deterioration data generation means 51 in order to learn the complementer, and reads out the given data from the learning data storage means 40 (step S115). The deterioration data generation means 51 creates deterioration data by removing a part of the key points after performing normalization processing and rotation processing on the given data (step S120).

The image processing unit 5 functions as a complement learning means 52, and the added data normalized and rotated by the deterioration data generating means 51 and the deteriorated data generated from the assigned data are input, and the complement is learned using the data. (Step S130), and the generated complementer is stored in the complement storage means 42 (step S135).

[Operation of object recognition device 1 in the recognition stage]
FIG. 8 is a schematic flow chart relating to the operation of the object recognition device 1 in the recognition stage.

When the object recognition device 1 starts the recognition operation, the photographing unit 2 sequentially outputs images taken in the monitoring space at predetermined time intervals. The image processing unit 5 cooperates with the communication unit 3 to repeat the operation shown in the flow chart of FIG. 8 every time a captured image is received from the photographing unit 2.

The communication unit 3 functions as the captured image acquisition means 30, and when the captured image is received, the captured image is output to the image processing unit 5 (step S200).

The image processing unit 5 functions as the key point detecting means 53, detects the key point for each person from the input captured image by using the detector stored in the detector storage means 41, and generates detection data ( Step S205).

When the key point is detected by the key point detecting means 53 (when “YES” in step S210), the image processing unit 5 functions as the key point complementing means 54. The key point complementing means 54 complements the detection data generated by the key point detecting means 53 after performing a normalization process (step S215) and a process of assigning a state value for each key point (step S220). Using the complementer stored in the instrument storage means 42, the key point is complemented and the complemented detection data is generated (step S225).

Further, the key point complementing means 54 performs a process (steps S230 to S240) of integrating a plurality of detected data. As described above, the integrated process is a process of collecting a plurality of detection data detected from one person, and the duplication detection determination process S230 is a duplication detection (same person) based on the degree of duplication between the plurality of detection data. It is determined that multiple detection data have been obtained. When there is duplicate detection (when “YES” in step S235), the key point complementing means 54 integrates the duplicate key points in the complemented detection data determined to be duplicate detection (step S240). On the other hand, if there is no duplication detection (in the case of "NO" in step S235), step S240 is omitted. In this way, the key point complementing means 54 completes the process of generating the complemented detection data, and supplies the data to the object area detecting means 55 and the object attribute estimating means 56.

The image processing unit 5 functions as the object area detecting means 55, and detects the person area as the object area based on the complemented detection data input from the key point complementing means 54 (step S245). Further, the image processing unit 5 functions as the object attribute estimation means 56, and based on the complemented detection data input from the key point complement means 54, determines the object attribute such as whether or not the detected person is a child. Perform the estimation process (step S250).

The image processing unit 5 and the communication unit 3 cooperate to function as the recognition result output means 31, and the recognition including the person area detected by the object area detecting means 55 and the attribute estimation result by the object attribute estimating means 56. The result is output to the output unit 6 (step S255). Further, when the key point is not detected from the captured image in step S205 (when “NO” in step S210), the recognition result output means 31 outputs the recognition result including that fact to the output unit 6.

When the object recognition device 1 outputs the recognition result for the captured image acquired in step S200, the object recognition device 1 returns to step S200 and repeats the above processing for the captured image acquired next.

FIG. 9 is a schematic flow chart of the duplication detection determination process S230 by the key point complement means 54. The key point complementing means 54 sorts the complemented detection data generated in step S225 of FIG. 8 in descending order for the number of key points having a state value of 1, and further, data having the same number of key points. After performing the sort process in which the object IDs are sorted in ascending order (step S300), the object IDs are temporarily released from all the complemented detection data (step S305). Then, the initial value of the newly assigned object ID is set to 1 (step S310), and the process of reassigning the object ID to each complemented detection data based on the determination result of duplicate detection is started.

If the complemented detection data has no object ID assigned (when “YES” in step S315), the unassigned data in the earliest order is selected (step S320). Hereinafter, the selected data is referred to as complemented detection data A. The key point complementing means 54 can calculate an circumscribed rectangle for a plurality of key points of the complemented detection data whose state value is not 0, and use the area as the area of the existing area of the person.

The current set value of the object ID is assigned to the complemented detection data A (step S325). After that, the complemented detection data for which the object ID has not been assigned is sequentially selected as the complemented detection data B (step S330). Then, when the complemented detection data B is selected (when “YES” in step S335), the degree of duplication between the complemented detection data A and B is calculated (step S340).

In this embodiment, a multiplicity alpha, using human region R _B for the human region R _A, and complemented detection data B for complemented detection data A,
(Number 1)
alpha = calculating the area of _{R A} and _R sum area of the area / _{R A} and _{R B} of the overlap region of the _B. Here, the region _R A, is _{R B} and the above-mentioned circumscribed rectangle. If both the complemented detection data A and B cannot be normalized and cannot be complemented (in this embodiment, the complemented detection data in which the state value of one of the key points on both shoulders is 0) , Duplicate judgment processing always returns the judgment result without integration.

If the complemented detection data of one of the complemented detection data A and B (referred to as B here) contains a key point whose state value is 0, the other method of calculating the multiplicity α is
(Number 2)
alpha = it may be the area of _{R A} and area / _{R B} of the overlap region of _{R B.} In addition, instead of the area, the smaller the average value of the distances between the positions of the key points of the same part, the higher the degree of overlap can be obtained. In this case, the key points of only one part are excluded from the distance calculation target. Further, preferably, the easiness of deviation for each part is converted into an index value in advance, and the distance of the part having a high degree of deviation is weighted lower to obtain the average value. Further, a multiplicity calculator that outputs the similarity between two or three or more part data as the multiplicity may be trained in advance and used.

When the degree of duplication calculated in step S340 is equal to or greater than a predetermined reference value (when “YES” in step S345), the current set value of the object ID is added to the complemented detection data B (step). S350). As a result, a common object ID is assigned to the complemented detection data A and B. When the key point complementing means 54 finishes step S350, it returns to step S330 and selects the next complemented detection data B. If the degree of duplication is less than the reference value (“NO” in step S345), the current complemented detection data B returns to step S330, leaving the object ID unassigned, and returns to the next complemented detection data B. The process moves to the processing of the detection data B.

When there is no complemented detection data that has not been assigned an object ID when the current complemented detection data A is selected and is not selected as the complemented detection data B (in the case of "NO" in step S335). ), The set value of the object ID is incremented (step S355), and the process returns to step S315. Then, if there is complemented detection data in which the object ID has not been assigned (when “YES” in step S315), new complemented detection data A is selected from among them (step S320), and the step If the processing of S325 to S355 is performed, while the complemented detection data for which the object ID has not been assigned does not remain (when “NO” in step S315), the duplication detection determination processing is terminated and FIG. The process proceeds to step S235.

If there are a plurality of complemented detection data having the same object ID in the complemented detection data after the duplicate detection determination process S230, the key point complement means 54 determines in step S235 that there is duplicate detection, and in that case, performs the integrated process S240. ..

The integrated process S240 will be described below. In this process, the position and state value of each key point of the integrated detection data (integrated detection data) is determined from a plurality of complemented detection data to be integrated according to a predetermined rule. Specifically, for the pair of complemented detection data having the same object ID, the position and state values as shown below are updated repeatedly until there are no more pairs. The following four combinations of state values of the same part in the two complemented detection data to be integrated can be considered.
(1) Key points obtained by complementation and detected key points (state values "2" and "1")
(2) Detected key points (both state values "1")
(3) Key points obtained by complementation (both state values "2")
(4) The position of only one key point is known (one is the state value "1" or "2" and the other is "0").

In the present embodiment, for (1), the position and state value of the detected key point are preferentially selected. For (2) and (3), priority is given to the complemented detection data having a large number of detected key points, and if the number of detected key points is the same, the positions are averaged. The state value is "1" in the case of (2) and "2" in the case of (3). Regarding (4), the position and state value of the key point whose position is known are used. According to this rule, a plurality of complemented detection data having the same object ID are integrated to generate one integrated detection data. By the way, 002 is assigned as a common object ID in the duplicate detection determination for the complemented detection data of the object IDs 002 and 003 in FIG. 5B, and the data are integrated by integrating them according to the above rule. As the completed detection data, the complemented detection data having the object ID of 002 in FIG. 6D is generated.

In the integration process of step S240, the part data after integration is determined by using the key points complemented in step S225 as they are, but the detection data to which the same object ID is given in the duplicate detection determination process S230 is complemented again. May be performed to determine the site data after integration. FIG. 10 is a schematic flow chart regarding the operation at the recognition stage in the object recognition device 1 having a configuration in which the objects are complemented and integrated again. In FIG. 10, the same reference numerals are given to the processes common to those in FIG. 8, and the processes in FIG. 10 replace the integrated process S240 in FIG. 8 with key point recomplementation for a set of detection data detected in duplicate. It is different in that it is an integrated process S400 accompanied by. Here, the set of detection data for which duplicate detection is detected is a combination of the original detection data corresponding to the complemented detection data to which the same object ID is assigned by the duplicate detection determination process S230. Yes, we will call it synthetic detection data. In the integrated process S400, a complementer is applied to the composite detection data as recomplementation, and the output is used as the integrated detection data. For example, from the part data obtained by applying the above rule to the complemented detection data, only the detected key point (state value “1”) is left and the rest is set to the state value “0”. Create detection data. By complementing again, it is possible to complement with the detected data having more detected key points, so that the position of the key points with higher accuracy can be obtained. Further, in the process described with reference to FIGS. 8 and 9 and the process of FIG. 10, it is possible to suppress multiple detection of the same person. Further, by using the complemented detection data, duplicate detection can be determined, and it is possible to reduce the error of recognizing the part data of one object as that of a plurality of objects.

[Modification example]
(1) In the above embodiment, an example in which the whole body of a person is an object is shown, but the object may be a part of a human body such as the upper body of a person or an object other than a person such as a vehicle or a chair. May be good.

(2) In the above embodiment, the measurement data in which the object is measured is a two-dimensional image, and the measurement unit that acquires the measurement data is the photographing unit 2 and is a camera that captures the two-dimensional image. The measurement data and the measurement unit are not limited to this example. For example, the measurement data may be a measurement of a three-dimensional space. Examples of the three-dimensional measurement data include a distance image obtained by using a distance image sensor in the measurement unit and three-dimensional data constructed from an image taken by a multi-view camera. Further, the measurement data can be a time series of two-dimensional images (time series of two-dimensional measurement data) or a time series of three-dimensional measurement data.

(3) In the above embodiment, an example is shown in which normalization is performed using one set of key points in which the midpoint of the key points on both shoulders is set as the origin after normalization. In another embodiment, a plurality of types of normalization are defined including normalization using another set of key points, and the key that can be used in the grant data for each grant data to be normalized. It can also be a method of selecting normalization according to a set of points. By doing so, it becomes possible to perform learning using samples without waste, and it is possible to improve the accuracy of the complementer. In addition, by doing so, the possibility of failing to estimate the attribute can be reduced.

Further, in the above embodiment, the origin is determined by using two key points, but the origin may be determined by using one key point or three or more key points.

(4) In the above embodiment, the part data is expressed in the form of coordinates of the position of the key point, but it can also be expressed in the form of an image. For example, a binary image in which the pixel value is 1 only at the coordinate position of the key point may be created for each key point, or an image obtained by applying a Gaussian filter to the binary image may be used. In that case, the coordinates of each key point correspond to the point that takes the maximum value in the image. Further, the input and output of the deterioration data generation means 51, the complement learning means 52, and the key point complement means 54 may have different representation forms of the key point position, and the position is represented by coordinates in the input and an image in the output. It can be represented, or conversely, the input can be an image and the output can be represented by coordinates.

(5) The complementer may output the reliability of the coordinate estimated value in addition to the estimated value of the coordinate of the key point. In this case, the key point complementing means 54 can set the state value in the complemented detection data to "1" only for the key points having a certain degree of reliability or higher. For example, when the number of key points detected by the key point detecting means 53 is small and it is difficult to estimate the coordinates, it is easy to estimate the wrong coordinates. In this regard, by not setting the state value to "1" for key points with low reliability of the obtained coordinate estimates, the accuracy of object recognition based on the complemented detection data can be improved or the result of object recognition. It becomes possible to transmit the accuracy index of. For example, by using this complemented detection data, the object attribute estimation means 56 is a combination of a combination of key points on the right side and a combination of key points on the left side, whichever has more key points whose state value is “1”. The attributes can be estimated using. Further, the object area detecting means 55 obtains and outputs an accuracy index according to the number of key points whose state value is "1" among the key points used for detecting the object area, and outputs the accuracy index, and is an object attribute estimating means. 56 obtains and outputs an accuracy index according to the number of key points whose state value is “1” among the key points used for estimating the object attribute, and the recognition result output means 31 recognizes these accuracy indexes. Include it in the result and output it.

In addition, the reliability may be retained as it is together with the complemented detection data without being converted into a state value expressed in binary. In that case, the selection index and accuracy index of the key point combination are the total value of the reliability. And so on.

(6) In the above embodiment, VAE is used as a complement, but other models capable of outputting continuous values such as a neural network and a Gaussian process may be used. In addition, by discretizing the coordinates of the key points in advance and formulating the estimation of the position of the part as a classification problem in which each key point belongs to one of them, identification of AdaBoost etc. as a complementer A model can also be used.

(7) In the above embodiment, the ReLU function is used as the activation function in the key point complementing means 54, but a tanh function, a sigmoid function, or the like may be used as the activation function. Further, the configuration may have a shortcut structure as used in ResNet (residual network).

(8) In the above embodiment, the circumscribing rectangle of the complemented detection data before integration is used for calculating the multiplicity α in the integration process in the key point complementing means 54, but instead, the circumscribing rectangle is used. An area enlarged by a predetermined time may be used, or an area obtained by inputting the complemented detection data before integration into the above-mentioned converter that has learned the conversion from the human part data to the human area may be used. Good.

(9) In the key point complementing means 54, when integrating the part data of the object, each part data obtained from the detector or the score or reliability of each key point may be used. Misintegration can be expected to be suppressed by preferentially integrating site data with high scores and reliability, or by not integrating site data with low scores and reliability during duplication detection determination processing. However, the detector and the key point detecting means 53 in that case output the score and the reliability of the key point in addition to the position of each key point.

Further, the sort order in the duplicate detection determination by the key point complement means 54 may be the descending order of the area.

(10) By executing step S240 in FIG. 8 immediately after step S350 in FIG. 9 or step S400 in FIG. 10 immediately after step S350 in FIG. 9, the degree of duplication may be updated sequentially. Good.

(11) In the above embodiment, an example in which common grant data is used for learning of the complementer and the detector is shown, but different grant data may be used. Further, the imparted data used only for the complement may be created independently of the learning image (for example, created by two-dimensionally projecting a three-dimensional model with part data of the object). In addition, there may be a time lag between the learning stage of the complement and the learning stage of the detector.

That is, a change that adds a new recognition process to an existing object recognition device using a detector can be realized only by adding a new recognition process and the necessary complementary process without re-learning the detector. Become.

(12) In the above embodiment, the storage unit 4 and the image processing unit 5 are provided on the image center side, but these may be provided on the event venue side.

1 Object recognition device, 2 Imaging unit, 3 Communication unit, 4 Storage unit, 5 Image processing unit, 6 Output unit, 30 Captured image acquisition means, 31 Recognition result output means, 40 Learning data storage means, 41 Detector storage Means, 42 Complementary storage means, 50 Detector learning means, 51 Degraded data generation means, 52 Complementer learning means, 53 Keypoint detecting means, 54 Keypoint complementing means, 55 Object area detecting means, 56 Object attribute estimation means.

Claims (8)

It is an object recognition device that detects a plurality of detection-required parts constituting the object from the measurement data obtained by measuring the predetermined object.
Using the given data including the positions of the detection-required sites given to each of the plurality of samples of the object, and inputting the deterioration data in which some of the said positions are omitted from the given data, the original Complementary device storage means that stores the complementer generated in advance by learning with the given data as the output target value, and
A site detection means that detects one or more of the positions of the detection-required sites on the object from the measurement data and acquires the detection data of the object.
A site complementing means for inputting the detection data into the complement and complementing the position of the detection-required site lacking in the detection data to generate complemented detection data.
An object recognition device characterized by being equipped with. From the measurement data, the target area in which a predetermined object is measured is detected based on the detection-required parts that are a plurality of parts constituting the object and are required to detect the target area. It ’s a recognition device,
Using the given data including the positions of the detection-required sites given to each of the plurality of samples of the object, and inputting the deterioration data in which some of the said positions are omitted from the given data, the original Complementary device storage means that stores the complementer generated in advance by learning with the given data as the output target value, and
A site detection means that detects one or more of the positions of the detection-required sites on the object from the measurement data and acquires the detection data of the object.
A site complementing means for inputting the detection data into the complement and complementing the position of the detection-required site lacking in the detection data to generate complemented detection data.
An object area detecting means for detecting a predetermined range based on the position of the detection-required part indicated by the complemented detection data as the object area in the measurement data.
An object recognition device characterized by being equipped with. An object recognition device that estimates whether or not a person measured in the measurement data is a child or not based on a plurality of parts constituting the person who is the object and a detection-required part determined to be necessary for the estimation. And
Using the given data including the positions of the detection-required sites given to each of the plurality of samples of the object, and inputting the deterioration data in which some of the said positions are omitted from the given data, the original Complementary device storage means that stores the complementer generated in advance by learning with the given data as the output target value, and
A site detection means that detects one or more of the positions of the detection-required sites on the object from the measurement data and acquires the detection data of the object.
A site complementing means for inputting the detection data into the complement and complementing the position of the detection-required site lacking in the detection data to generate complemented detection data.
A child estimation means for estimating whether or not the person measured in the measurement data is a child by comparing the ratio of the distances between the detection-required parts in the complemented detection data with a predetermined reference.
An object recognition device characterized by being equipped with. When the site complementing means acquires the detection data for two or more objects from one measurement data, the site complementing means measures the target from the position of the detection-required site output by the complement. The existence area of each object is estimated, and when the degree of overlap between the existence areas of the plurality of detection data is equal to or more than a predetermined reference value, the plurality of detection data are integrated as if they are due to the same object. The object recognition device according to any one of claims 1 to 3, wherein the complemented detection data is generated. When the degree of overlap between the existing regions of the plurality of detected data is equal to or higher than a predetermined reference value, the site complementing means has the same site obtained by the complementer for each of the plurality of detected data. The object recognition device according to claim 4, wherein the complemented detection data is generated by integrating the positions into one. When the degree of overlap between the existing regions of the plurality of detected data is equal to or higher than a predetermined reference value, the site complementing means integrates the plurality of detected data and inputs the plurality of detected data to the complement. The object recognition device according to claim 4, wherein the detected detection data is generated. It is an object recognition method that detects a plurality of detection-required parts constituting the object from the measurement data obtained by measuring the predetermined object.
Using the given data including the positions of the detection-required sites given to each of the plurality of samples of the object, the deterioration data in which some of the said positions are omitted from the given data is input, and the original said. The step of preparing a complementer generated in advance by learning with the given data as the output target value, and
A site detection step of detecting one or more of the positions of the detection-required sites on the object from the measurement data and acquiring the detection data of the object.
A site complement step in which the detection data is input to the complement, the position of the detection-required site lacking in the detection data is complemented, and complemented detection data is generated.
An object recognition method characterized by being equipped with. It is a program that causes a computer to perform a process of detecting a plurality of detection-required parts constituting the object from the measurement data obtained by measuring the predetermined object.
The computer
Using the given data including the positions of the detection-required sites given to each of the plurality of samples of the object, and inputting the deterioration data in which some of the said positions are omitted from the given data, the original Complementor storage means that stores complements generated in advance by learning with the given data as the output target value,
A site detection means that detects one or more of the positions of the detection-required sites in the object from the measurement data and acquires the detection data of the object, and
A site complementing means that inputs the detection data to the complement and complements the position of the detection-required site that is lacking in the detection data to generate complemented detection data.
An object recognition program characterized by functioning as.

Images