JP2018156449A - Abnormality object detection program, abnormality object detection method and abnormality object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an object out of specification from images of the object picked up in a various shapes due to movable parts included.SOLUTION: An abnormality object detection device 100 is configured to calculate the curvature of each of plural objects extracted from pick-up images. The abnormality object detection device 100 is configured to sort plural objects into plural groups by comparing characteristics of the plural objects the difference of the magnitude of curvatures is in a predetermined range based on the curvature of the plural objects. The abnormality object detection device 100 is configured to determine a group in which the number of the included objects is smaller than a predetermined number, and to output the objects included in the determined group.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an abnormal object detection program and the like.

  In various inspections such as inspection of abnormalities in the human body, investigation of posture during work, inspection of products, inspection of materials, etc., abnormal objects whose appearances are outside the range considered normal based on the appearance of other objects are detected. Things have been done.

  Generally, when detecting an abnormal object using image recognition technology, correct data (normal object image, abnormal object image) is collected in advance, and feature values are extracted from the correct data, A normal / abnormal object is discriminated using the extracted feature amount. However, in order to discriminate an abnormal object from correct answer data, a large amount of correct answer data is used. However, it may be difficult to collect a lot of correct answer data depending on applications.

  On the other hand, it is conceivable to detect abnormal objects by defining abnormal objects instead of correct answer data, but abnormal objects have various variations such as malformations and discoloration, and define abnormal objects. It's also difficult.

  As described above, when it is difficult to determine using correct answer data or to define an abnormal object, an operator visually determines an abnormal object. However, since it is a burden on the operator to visually determine an abnormal object, it is required to automatically detect the abnormal object.

  FIG. 12 is a diagram for explaining an example of the prior art. For example, as described above, it is difficult to discriminate using correct data, or to define an abnormal object and directly detect the abnormal object, so there are fewer abnormal objects than normal objects. Take advantage of something. For example, in the prior art, similar objects are collected, and an object having a small number of similar objects is detected as an abnormal object.

  In the example shown in FIG. 12, when comparing the objects 1a to 1e, the objects 1a to 1d are similar to each other, and the object 1e is not similar to the other objects 1a to 1d. In this case, in the prior art, the object 1e is detected as an abnormal object.

JP 2013-30782 A

  However, the above-described conventional technique has a problem that it is not possible to detect nonstandard objects from images of objects picked up in various shapes.

  In the prior art, if an object does not deform, it is possible to detect an abnormal object. However, if the object includes a movable part, the object itself may be deformed and the abnormal object may be erroneously detected. In the prior art, when determining the similarity of objects, it is determined that objects that are similar in shape or color are similar. It is determined that it is not. That is, bending itself is detected as anomaly.

  As an example, human and animal elbows, knees, spine, hips, robot arms, multi-segmental creatures, and the like are objects including movable parts. As for the bending condition, it is bent in one direction such as a joint, and basically, the bending condition within 180 degrees is assumed. In addition, it is assumed that the outer shell structure is bent at a portion where a plurality of structures are connected and is bent into a curved surface as a whole.

  In one aspect, an object of the present invention is to provide an abnormal object detection program, an abnormal object detection method, and an abnormal object detection apparatus capable of detecting an object out of specification from images of objects captured in various shapes. And

  In the first plan, the computer executes the following processing. The computer calculates curvatures of a plurality of objects extracted from the captured image. The computer classifies the plurality of objects into a plurality of groups by comparing the characteristics of the objects whose difference in curvature is within a predetermined range based on the curvatures of the plurality of objects. The computer determines a group in which the number of belonging objects is less than a predetermined number, and outputs the objects belonging to the determined group.

  Non-standard objects can be detected from images of objects captured in various shapes.

FIG. 1 is a schematic diagram illustrating an example of processing performed by the abnormal object detection device according to the first embodiment. FIG. 2 is a functional block diagram illustrating the configuration of the abnormal object detection device according to the first embodiment. FIG. 3 is a diagram illustrating an example of the data structure of the management table according to the first embodiment. FIG. 4 is a diagram for explaining the process of calculating the curvature from the skeleton. FIG. 5 is a diagram for explaining the processing of the classification unit. FIG. 6 is a flowchart illustrating the processing procedure of the abnormal object detection device according to the first embodiment. FIG. 7 is a flowchart showing a process for extracting the curvature and feature amount of an object. FIG. 8 is a functional block diagram illustrating the configuration of the abnormal object detection device according to the second embodiment. FIG. 9 is a diagram illustrating an example of the data structure of the image list. FIG. 10 is a flowchart illustrating the processing procedure of the abnormal object detection device according to the second embodiment. FIG. 11 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same function as the abnormal object detection apparatus. FIG. 12 is a diagram for explaining an example of the prior art.

  Embodiments of an abnormal object detection program, an abnormal object detection method, and an abnormal object detection device disclosed in the present application will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a schematic diagram illustrating an example of processing performed by the abnormal object detection device according to the first embodiment. For example, the abnormal object detection device performs the following steps S10 to S14.

  Step S10 will be described. The abnormal object detection device creates a silhouette based on the image of the object photographed by the camera. In the example shown in FIG. 1, object silhouettes 10 to 15 are detected. In the following description, object silhouettes 10 to 15 are referred to as objects 10 to 15 for convenience.

  Step S11 will be described. The abnormal object detection device calculates the curvature of each of the objects 10 to 15.

  Step S12 will be described. The abnormal object detection apparatus rearranges the objects 10 to 15 according to the curvature of the objects 10 to 15. In the example illustrated in FIG. 1, when the abnormal object detection device arranges the objects 10 to 15 in the order of decreasing curvature, the order is the order of the objects 10, 11, 12, 13, 14, and 15. Further, when arranging the objects 10 to 15, the abnormal object detection device discriminates both ends (tip, root, etc.) of the objects so that both ends and the convex direction of the objects are the same direction. In the example shown in FIG. 1, the upper side is the tip of the object, the lower side is the root of the object, and the convex direction is the right side. In the present embodiment, as an example, description will be given with one of both ends as the tip and the other as the root.

  Step S13 will be described. The abnormal object detection apparatus determines whether or not the object feature amounts are similar between objects having similar curvatures. For example, the abnormal object detection apparatus may determine the feature amount of the object using ART (Angular Radial Transform) conversion or wavelet conversion. For example, it is assumed that the feature quantities of the objects 10 and 11 are similar and the feature quantities of the objects 11 and 12 are similar. Further, the feature amounts of the objects 12 and 13 are not similar, but the feature amounts of the objects 12 and 14 are similar. Assume that the features of the objects 14 and 15 are similar.

  Step S14 will be described. As a result of the processing in step S13, the objects 10, 11, 12, 14, and 15 become objects having similar curvatures and similar features, and the object 13 is characterized by features having similar curvatures. Becomes a dissimilar object. For this reason, the abnormal object detection device outputs the object 13 as an abnormal object.

  As described above, the abnormal object detection apparatus executes the processing of steps S10 to S14 in FIG. 1 to thereby include a non-standard object (abnormality) from images of objects captured in various shapes to include a movable part. Object) can be detected.

  Next, an example of the configuration of the abnormal object detection device according to the first embodiment will be described. FIG. 2 is a functional block diagram illustrating the configuration of the abnormal object detection device according to the first embodiment. As illustrated in FIG. 2, the abnormal object detection device 100 includes a camera 110, a communication unit 120, an input unit 130, a display unit 140, a storage unit 150, and a control unit 160.

  The camera 110 is an apparatus that captures an image of an object to be inspected. The camera 110 outputs the captured image data to the control unit 160. In the following description, image data is simply referred to as image data. The camera 110 may be connected to the abnormal object detection apparatus 100 via a network.

  The communication unit 120 is a device that executes data communication with an external device via a network. The communication unit 120 corresponds to a communication device such as a network card.

  The input unit 130 is an input device for inputting various types of information to the abnormal object detection device 100. For example, the input unit 130 corresponds to a keyboard, a mouse, a touch panel, or the like.

  The display unit 140 is a display device that displays information output from the control unit 160. For example, the display unit 140 corresponds to a liquid crystal display or a touch panel.

  The storage unit 150 includes an image table 151 and a management table 152. The storage unit 150 corresponds to a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), and a flash memory, and a storage device such as a hard disk drive (HDD).

  The image table 151 is a table that stores image data captured by the camera 110.

  The management table 152 is a table that holds the curvature and feature amount of an object included in image data. FIG. 3 is a diagram illustrating an example of the data structure of the management table according to the first embodiment. As shown in FIG. 3, the management table 152 associates object identification information, curvature, and feature amounts. The object identification information is information that uniquely identifies an object. The curvature is the curvature of the object. The feature amount is a feature amount of the object.

  The control unit 160 includes a registration unit 161, a calculation unit 162, a classification unit 163, and a determination unit 164. The control unit 160 can be realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. The control unit 160 can also be realized by a hard-wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  The registration unit 161 is a processing unit that acquires image data from the camera 110 and registers the acquired image data in the image table 151. The registration unit 161 registers image data in the image table 151 every time image data is acquired from the camera 110.

  The calculation unit 162 is a processing unit that acquires image data from the image table 151 and calculates the curvature of an object included in the image data. For example, the calculation unit 162 calculates the curvature of the object after performing the process of extracting the silhouette of the object.

  A process in which the calculation unit 162 extracts the silhouette of the object will be described. Based on the image data, the calculation unit 162 generates a mask image in which the pixels of the object portion are filled with “black” and the pixels of the background portion are filled with “white”. Such a mask image corresponds to a silhouette. Note that a mask image in which the pixel of the object part is painted with “white” and the pixel of the background part with “black” may be generated.

  There are two methods for extracting silhouettes: a method of specifying and extracting a background image, and an object silhouette extraction by motion extraction. When a silhouette is extracted using a background image, the camera 110 generates a background image that is captured in the absence of an object in advance, and captures an object that exists on the same background. The calculation unit 162 obtains a difference from the pixel color determined in the background image for each pixel of the captured image data, and extracts a portion having a large difference as an object region. A monochromatic image may be used as a background image, and an object may be placed on a predetermined background color sheet for photographing.

  The effect is enhanced when the background color is set as a complementary color of the object color. For example, when shooting a red object, the use of a blue background or the like increases the accuracy of silhouette extraction. The background color may be any background color as long as it is different from the object color.

  Silhouette extraction by motion extraction extracts a background of an object by extracting a region where the object moves when the object moves. For example, the calculation unit 162 acquires the image data at the time t and the image data at the time t−1 from the image table 151, and calculates the difference between the pixel values of the image data for each pixel. Is generated. The calculation unit 162 refers to the difference of the difference image and determines an area where the difference value is equal to or greater than the threshold as an object area.

  In the first embodiment, as a bent object, a shape that basically deforms a single rod-like object is assumed. For example, human and animal elbows, knees, spines, hips, robot arms, and multi-segmental creatures can be handled as a single rod in most cases, even if they have some fine irregularities. Therefore, the calculation unit 162 may delete a fine part of the silhouette by further executing distance conversion described below.

  Distance conversion is a method of detecting the distance from each black object image to the nearest white background pixel when there is an image composed of a black object image and a white background image. Specifically, the calculation unit 162 executes the following steps 1 to 4. Procedures 1 to 3 are processing corresponding to distance conversion.

  Procedure 1: The calculation unit 162 prepares the same number of memory areas as each pixel in the storage unit 150 in order to obtain the distance of each pixel.

  Procedure 2: The calculation unit 162 scans each pixel of the silhouette from the upper left, and sets the distance value to 0 in the case of a white pixel (background). In the case of a black pixel (object), the calculation unit 162 sets a value obtained by adding 1 to the larger distance value of the left pixel and the distance value of the upper pixel as the distance value, and sets the current pixel. The calculation unit 162 performs such processing from the upper left to the lower right.

  Step 3: The calculation unit 162 scans each pixel of the silhouette from the lower right, and in the case of a black pixel, the calculation is performed by adding 1 to the smaller one of the lower distance value and the right distance value. The distance value is set to the current pixel. The calculation unit 162 performs such processing from the lower right to the upper left.

  Step 4: After executing Steps 1 to 3, the calculation unit 162 refers to the distance value of each pixel, excludes the pixel region whose distance value is less than the threshold from the object region, and identifies the object region To do. The calculation unit 162 assigns unique object identification information to each identified object.

  The calculation unit 162 associates the object region information extracted by executing steps 1 to 4 with the object identification information, and outputs the information to the classification unit 163.

  Subsequently, a process in which the calculation unit 162 calculates the curvature of the object will be described. The calculation unit 162 extracts a skeleton from the silhouette of the object whose curvature is to be calculated. A skeleton is a line segment corresponding to a skeleton that exists along a rod-like object. There are several skeleton extraction methods, but the method using distance transformation will be described below.

  The calculation part 162 performs distance conversion by performing said procedure 1-3 with respect to a silhouette. Since the portion where the distance value is the peak is the center line of the rod-like object, the calculation unit 162 extracts the skeleton by connecting the portions where the distance value is the peak with a line. When connecting the skeletons, the calculation unit 162 connects the skeletons in which the distance between the skeletons is less than the predetermined distance and the peak value of the distance value of each skeleton is equal to or greater than the predetermined value. Although the skeleton can have a graph structure, the calculation unit 162 extracts the skeleton as a single curve. The calculation unit 162 does not extract the peak of the distance value that does not constitute the curve as a skeleton.

  The calculation unit 162 may extract a skeleton using a thinning method or the like in addition to using distance conversion. The thinning technique is described in references (Takeshi Aoiin, Tomoharu Nagao (joint work), “Introduction to Image Processing in C Language”, Daiichi Shobo (2000)). The calculation unit 162 excludes short lines when branches and leaves are formed due to thinning.

  The calculation unit 162 calculates a curvature from the skeleton. When a single curve is extracted with the skeleton, the calculation unit 162 calculates the degree of bending (angle) of the skeleton as the curvature. FIG. 4 is a diagram for explaining the process of calculating the curvature from the skeleton. In the example shown in FIG. 4, a skeleton 21 is extracted from an object (silhouette) 20.

The calculation unit 162 calculates two vectors with reference to the first to fourth coordinates in the coordinates on the skeleton 21. Regarding the coordinates (x (d), y (d)), d is a parameter indicating the distance from the reference coordinates of the skeleton. As an example, the reference coordinate is the upper end of the skeleton. In each coordinate, n indicates the length of the distance from the reference coordinate, and w indicates the weight. Let the value of w be less than 0.5.
First coordinate: (x (0), y (0))
Second coordinate: (x (n × w), y (n × w))
Third coordinate: (x (n), y (n))
Fourth coordinate: (x (n × (1-w)), y ((n × (1-w))

The first vector 21a and the second vector 21b are as shown below. The calculation unit 162 calculates the angle formed by the first vector and the second vector from the relationship between the inner products of the vectors, and sets the calculated angle as the curvature of the object.
First vector 21a: (x (0) −x (n × w), y (0) −y (n × w))
Second vector 21b: (x (n) -x (n * (1-w)), y (n) -y (n * (1-w)))

  The calculation unit 162 calculates the curvature of each object by repeatedly executing the above processing for the silhouette of each object. The calculating unit 162 registers the object identification information of the object and the curvature in the management table 152 in association with each other.

  The classification unit 163 is a processing unit that classifies a plurality of objects into a plurality of groups by comparing feature quantities of the objects whose difference in curvature is within a predetermined range based on the curvatures of the plurality of objects. It is. For example, the classification unit 163 performs a process for determining a difference between both ends of an object, a process for extracting a feature amount of an image, and a process for classifying an object.

  A process in which the classification unit 163 determines the difference between both ends of the object will be described. For example, the classification unit 163 determines whether the both ends obtained by the skeleton are the tip or the root of the object. The classification unit 163 prepares typical template images in advance, compares the images with the images at both ends, and determines which is the tip and which is the root based on the combination having the shortest distance.

  The classification unit 163 may determine, using a general image feature amount at the tip, an end having a higher similarity to the feature amount as the tip. Any feature amount may be used as the feature amount of the image. For example, the image itself is used as the feature amount. In addition, as feature values of shapes, feature values using ART (Angular Radial Transform) conversion specified by MPEG (Moving Picture Experts Group) -7, shape feature values using Wavelet transform, etc. are used. Also good.

  FIG. 5 is a diagram for explaining the processing of the classification unit. In FIG. 5, an image 30a is an object image, and an image 30a 'is an inverted object image. A is a region estimated to be the tip of the object. Region B is a region that is estimated to be the root. Region B- is a region estimated to be the tip of the object. The region A- is a region estimated to be the root.

  The image 40a is a template of an object whose tip and root are known, and the image 40a 'is an inverted template. Region C is a region at the tip of the object. A region D is a fundamental region of the object. A region C- is a fundamental region of the object. A region D- is a region at the tip of the object.

The classification unit 163 calculates the first distance and the second distance based on the expressions (1) and (2). If the first distance is smaller than the second distance, it means that the initial estimation is correct. If the second distance is smaller than the first distance, it indicates that the initial estimation is wrong, so the relationship between the two ends of the object is reversed.
First distance = Distance (A, C) + Distance (B, D) (1)
Second distance = Distance (A, C −) + Distance (B, D−) (2)

  The classification unit 163 performs the above process on each object and determines both ends of the object.

  Subsequently, a process in which the classification unit 163 extracts the feature amount of the image will be described. The classification unit 163 rearranges each object so that the both ends and the convex direction of each object are aligned, and the object is changed from the one having the smallest curvature to the one having the larger curvature. For example, the classification unit 163 rearranges the objects 10 to 15 as described in step S12 of FIG. The classification unit 163 acquires information on the curvature of each object from the management table 152.

  The classification unit 163 extracts the feature amount of each object after performing the above rearrangement. For example, the classification unit 163 extracts a feature amount using ART (Angular Radial Transform) conversion or a feature amount using Wavelet conversion as the shape feature amount. The classification unit 163 registers the feature amount of each object in the management table 152 in association with the object identification information.

  Subsequently, a process in which the classification unit 163 classifies an object will be described. The classification unit 163 refers to the management table 152, and performs a process of classifying objects whose difference in curvature between objects is within a predetermined range and whose feature amounts are similar to each other into the same group. For example, the classification unit 153 determines that each object is similar when the difference value of the feature amount of each object is less than a threshold value.

  The classification unit 153 outputs the classification result to the determination unit 164. In the example described with reference to FIG. 1, the classification unit 153 classifies the objects 10, 11, 12, 14, and 15 into the same group, and classifies the object 13 into the same group.

  The determination unit 164 is a processing unit that determines an abnormal object based on the classification result of the classification unit 153. For example, the determination unit 164 counts the number of objects belonging to the same group, and determines that the object belonging to the group is an abnormal object when the counted number is less than a specified number. For example, in the example described with reference to FIG. 1, the object 13 is determined as an abnormal object. The determination unit 164 may output the determination result to the display unit 140, or may notify the external device connected to the network via the communication unit 120.

  Next, a processing procedure of the abnormal object detection device 100 according to the first embodiment will be described. FIG. 6 is a flowchart illustrating the processing procedure of the abnormal object detection device according to the first embodiment. As illustrated in FIG. 6, the calculation unit 162 and the classification unit 163 of the abnormal object detection device 100 acquire unanalyzed image data (step S101). The calculation unit 162 and the classification unit 163 extract the curvature and feature amount of the object from the object (step S102). The calculation unit 162 and the classification unit 163 register the curvature and the feature amount in the management table 152 (step S103).

  If the abnormal object detection apparatus 100 has not analyzed all the image data (step S104, No), the process proceeds to step S101. When the abnormal object detection apparatus 100 has analyzed all the image data (step S104, Yes), the abnormal object detection apparatus 100 proceeds to step S105.

  The classification unit 163 acquires the analyzed and undetermined image data (step S105). The classification unit 163 classifies the objects into groups based on the similarity of the feature quantities of the objects whose curvature difference is within a certain value (step S106).

  The determination unit 164 of the abnormal object detection device 100 determines whether or not the number of objects belonging to a group of similar objects is less than a specified number (step S107). If the number of objects belonging to the group of similar objects is less than the specified number (step S107, Yes), the determination unit 164 determines that the object belonging to the group that is less than the specified number is an abnormal object (step S107). S108), the process proceeds to step S109.

  On the other hand, when the number of objects belonging to the group of similar objects is not less than the specified number (No at Step S107), the determination unit 164 proceeds to Step S109. It is determined whether or not all image data have been determined (step S109).

  If the abnormal object detection device 100 has not determined all the image data (No at Step S109), the abnormal object detection device 100 proceeds to Step S105. If the abnormal object detection device 100 determines all the image data (step S109, Yes), the process ends.

  Next, the process of extracting the curvature and feature amount of the object in step S102 in FIG. 6 will be described. FIG. 7 is a flowchart showing a process for extracting the curvature and feature amount of an object. As illustrated in FIG. 7, the calculation unit 162 of the abnormal object detection device 100 extracts an object region from the image data (step S201). The calculation unit 162 extracts a skeleton of the object region (step S202).

  The calculating unit 162 calculates a curvature from the skeleton (step S203). The classification unit 163 of the abnormal object detection device 100 acquires images of the regions at both ends of the skeleton (step S204). The calculation unit 162 determines the end by comparing the image of the end of the skeleton with the template (step S205).

  The classification unit 163 of the abnormal object detection device 100 aligns both ends and the convex direction of the object, and sorts them according to the curvature (step S207). The classification unit 163 extracts the feature amount of each object from the image data (step S208).

  Next, effects of the abnormal object detection device 100 according to the first embodiment will be described. The abnormal object detection apparatus 100 calculates the curvature of each of a plurality of objects, classifies objects having similar feature quantities between objects having similar curvatures, and has a small number of objects belonging to the same group. Is detected as an abnormal object. Thereby, even an object that bends at a specific location can be detected as an abnormal object. For example, even if it is a normal object, if the curvature is different, the feature quantity differs from other normal objects, but this problem can be solved by comparing the feature quantities of objects with similar curvature. it can.

  The abnormal object detection apparatus 100 determines whether or not the feature amounts between the objects are similar after aligning both ends and the convex direction of each object to be compared. Thereby, it can be determined appropriately whether the feature-value of each object is similar. For example, even if it is a normal object, if both ends and the convex direction are different, the feature amount differs from that of another normal object. However, this problem can be solved by aligning both ends and the convex direction.

  FIG. 8 is a functional block diagram illustrating the configuration of the abnormal object detection device according to the second embodiment. As illustrated in FIG. 8, the abnormal object detection apparatus 200 includes a camera 210, a communication unit 220, an input unit 230, a display unit 240, a storage unit 250, and a control unit 260. Among these, the description about the camera 210, the communication unit 220, the input unit 230, and the display unit 240 is the same as the description about the camera 110, the communication unit 120, the input unit 130, and the display unit 140 described in FIG.

  The storage unit 250 includes an image table 251, a management table 252, and an image list 253. The storage unit 250 corresponds to a semiconductor memory element such as a RAM, a ROM, and a flash memory, and a storage device such as an HDD. The description regarding the image table 251 and the management table 252 is the same as the description regarding the image table 151 and the management table 152 described with reference to FIG.

  The image list 253 is a list that stores the classification results of the classification unit 263. As described in the first embodiment, the objects belonging to the same group are objects in which the difference in curvature between the objects is within a predetermined range and the feature amounts of the objects are similar.

  FIG. 9 is a diagram illustrating an example of the data structure of the image list. As shown in FIG. 9, this image list 253 associates group identification information with object information. The group identification information is information that uniquely identifies a group. The object information includes curvatures, feature amounts, and images of objects belonging to the same group. The object information may further include information on both ends of each object.

  The control unit 260 includes a registration unit 261, a calculation unit 262, a classification unit 263, and a determination unit 264. The control unit 260 can be realized by a CPU, MPU, or the like. The control unit 260 can also be realized by a hard wired logic such as ASIC or FPGA.

  The registration unit 261 is a processing unit that acquires image data from the camera 210 and registers the acquired image data in the image table 251. The registration unit 261 registers the image data in the image table 251 every time image data is acquired from the camera 210.

  The calculation unit 262 is a processing unit that acquires image data from the image table 251 and calculates the curvature of an object included in the image data. The calculation unit 262 calculates the curvature of the object every time new image data is registered in the image table 251. The other processing of the calculation unit 262 is the same as the processing of the calculation unit 162.

  The classification unit 263 is a processing unit that classifies a plurality of objects into a plurality of groups by comparing feature quantities of the objects whose difference in curvature is within a predetermined range based on the curvatures of the plurality of objects. It is. The classification unit 263 registers the classification result in the image list 253. For example, the classification result includes group identification information and object information. Other processing of the classification unit 263 is the same as the processing of the classification unit 163.

  The classification unit 263 may newly execute the following process after a plurality of classification results are accumulated in the image list 253. For example, when new image data is captured from the camera 210, the classification unit 263 includes the curvature and feature amount of the object included in the new image data, and the curvature and feature of each group of objects included in the image list 253. The quantity may be compared to determine which group it belongs to. In this case, the classification unit 263 outputs information indicating to which group the object included in the new image data belongs to the determination unit 264.

  The determination unit 264 is a processing unit that determines an abnormal object based on the classification result of the classification unit 263. For example, the determination unit 264 counts the number of objects belonging to the same group, and determines that the object belonging to the group is an abnormal object when the counted number is less than a specified number.

  In addition, when the determination unit 264 acquires the classification result of the group to which the object included in the new image data belongs from the classification unit 263, the determination unit 264 counts the number of objects included in the group to which the object belongs. When the counted number is less than the specified number, the determination unit 264 determines an object included in the new image data as an abnormal object.

  Next, a processing procedure of the abnormal object detection device 200 according to the second embodiment will be described. FIG. 10 is a flowchart illustrating the processing procedure of the abnormal object detection device according to the second embodiment. As shown in FIG. 10, the abnormal object detection device 200 generates an image list 253 (step S301). For example, step S301 is generated by executing the processing shown in steps S101 to S109 described in FIG.

  The calculation unit 262 and the classification unit 263 of the abnormal object detection device 200 acquire one piece of new image data (step S302). The calculation unit 262 and the classification unit 263 extract curvature and feature amounts from the object (step S303). The calculation unit 262 and the classification unit 263 register the curvature and feature amount of the object in the management table 252 (step S304).

  The classification unit 263 classifies the objects included in the new image data based on the image list 253 (step S305). The determination unit 264 of the abnormal object detection apparatus 200 determines whether or not the number of objects is less than the specified number for a group to which objects of new image data belong (step S306).

  If the number of objects is less than the specified number (step S306, Yes), the determination unit 264 determines that the object included in the new image data is an abnormal object (step S307). On the other hand, when the number of objects is not less than the specified number (No in step S306), the determination unit 264 registers object information of the objects in the image list 253 (step S308).

  Next, effects of the abnormal object detection device 200 according to the second embodiment will be described. The abnormal object detection apparatus 200 analyzes a plurality of image data in advance and generates an image list 253. When the abnormal object detection apparatus 200 newly acquires image data, the abnormal object detection apparatus 200 compares the curvature and feature amount of the object included in the image data with the image list 253, thereby including the new image data. It is determined whether or not the object is an abnormal object. Thereby, once the image list 253 is generated, it is possible to determine whether or not an object included in the single image data is abnormal when the single image data is acquired.

  Next, an example of a hardware configuration of a computer that realizes the same function as the abnormal object detection devices 100 and 200 shown in the above embodiment will be described. FIG. 11 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same function as the abnormal object detection apparatus.

  As illustrated in FIG. 11, the computer 300 includes a CPU 301 that executes various arithmetic processes, an input device 302 that receives input of data from a user, and a display 303. The computer 300 also includes a reading device 304 that reads a program and the like from a storage medium, and an interface device 305 that exchanges data with other computers via a network. The computer 300 has a camera 306. The computer 300 also includes a RAM 307 that temporarily stores various types of information and a hard disk device 308. The devices 301 to 308 are connected to the bus 309.

  The hard disk device 308 includes a calculation program 308a, a classification program 308b, and a determination program 308c. The CPU 301 reads the calculation program 308 a, the classification program 308 b, and the determination program 308 c and expands them in the RAM 307.

  The calculation program 308a functions as a calculation process 307a. The classification program 308b functions as a classification process 307b. The determination program 308c functions as a determination process 307c.

  The processing of the calculation process 307a corresponds to the processing of the calculation units 162 and 262. The processing of the classification process 307b corresponds to the processing of the classification units 163 and 263. The process of the determination process 307c corresponds to the processes of the determination units 164 and 264.

  Note that the programs 308a to 308c are not necessarily stored in the hard disk device 308 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 300. Then, the computer 300 may read and execute each of the programs 308a to 308c.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1)
Calculate the curvature of each of the multiple objects extracted from the captured image,
Based on the calculated curvatures of the plurality of objects, by comparing the characteristics of the objects whose difference in curvature is within a predetermined range, the plurality of objects are classified into a plurality of groups,
An abnormal object detection program for executing a process of determining a group in which the number of belonging objects is less than a predetermined number and outputting an object belonging to the determined group.

(Supplementary Note 2) In the classification process, the first object is selected from the plurality of objects, and the magnitude of the curvature within a predetermined range is different from the magnitude of the curvature of the first object. The abnormal object detection program according to appendix 1, wherein second objects similar to the characteristics of the first object are classified into the same group as the first object.

(Supplementary note 3) The supplementary note 2 is characterized in that the classification process determines whether or not the features between the objects are similar after aligning the direction of both ends and the convex direction of each object to be compared. Abnormal object detection program.

(Additional remark 4) The said classification | category process memorize | stores the curvature and characteristic of each object which belong to a group in a memory | storage part, and the said determination process is a said memory | storage part when a new object is detected from a picked-up image. And further executing a process of determining whether or not to output the newly detected object based on the stored curvature and feature of each object and the curvature and feature of the newly detected object. The abnormal object detection program according to attachment 1.

(Appendix 5) An abnormal object detection method executed by a computer,
Calculate the curvature of each of the multiple objects extracted from the captured image,
Based on the calculated curvatures of the plurality of objects, by comparing the characteristics of the objects whose difference in curvature is within a predetermined range, the plurality of objects are classified into a plurality of groups,
An abnormal object detection method comprising: executing a process of determining a group in which the number of belonging objects is less than a predetermined number and outputting an object belonging to the determined group.

(Supplementary Note 6) In the classification process, the plurality of objects are selected as the first object from the plurality of objects, and the difference between the first object and the curvature of the first object is within a predetermined range. The abnormal object detection method according to appendix 5, wherein second objects similar to the characteristics of the first object are classified into the same group as the first object.

(Supplementary note 7) The supplementary note 6 is characterized in that the classification process determines whether or not the features between the objects are similar after aligning the direction of both ends and the convex direction of each object to be compared. Abnormal object detection method.

(Additional remark 8) The said classification process memorize | stores the curvature and the characteristic of each object which belong to a group in a memory | storage part, and the said determination process stores in the said memory | storage part when a new object is detected from a picked-up image. And further executing a process of determining whether or not to output the newly detected object based on the stored curvature and feature of each object and the curvature and feature of the newly detected object. The abnormal object detection method according to appendix 5.

(Supplementary Note 9) A calculation unit that calculates the curvature of each of a plurality of objects extracted from a captured image;
Based on the calculated curvatures of the plurality of objects, a classification unit that classifies the plurality of objects into a plurality of groups by comparing the characteristics of the objects whose difference in curvature is within a predetermined range;
An abnormal object detection apparatus comprising: a determination unit that determines a group in which the number of belonging objects is less than a predetermined number, and outputs an object belonging to the determined group.

(Additional remark 10) The said classification | category part selects the said several object, the 1st object from the said several object, and the magnitude | size of the curvature in which the difference with the magnitude | size of the curvature of the said 1st object is within a predetermined range. And the second object similar to the feature of the first object is classified into the same group as that of the first object.

(Additional remark 11) The said classification | category part determines whether the characteristic between objects is similar, after aligning the direction of both ends of each object used as a comparison object, and a convex direction, or whether it is similar. Abnormal object detection device.

(Additional remark 12) The said classification | category part memorize | stores the curvature and characteristic of each object which belong to a group in a memory | storage part, and the determination part memorize | stored in the said memory | storage part, when a new object is detected from a picked-up image. Appendix 9 characterized by further executing a process of determining whether or not to output a newly detected object based on the curvature and characteristic of each object and the curvature and characteristic of the newly detected object The abnormal object detection device described.

100, 200 Abnormal object detection device 110, 210 Camera 120, 220 Communication unit 130, 230 Input unit 140, 240 Display unit 150, 250 Storage unit 160, 260 Control unit

Claims (6)

On the computer,
Calculate the curvature of each of the multiple objects extracted from the captured image,
Based on the calculated curvatures of the plurality of objects, by comparing the characteristics of the objects whose difference in curvature is within a predetermined range, the plurality of objects are classified into a plurality of groups,
An abnormal object detection program for executing a process of determining a group in which the number of belonging objects is less than a predetermined number and outputting an object belonging to the determined group.   The classifying process selects the plurality of objects, a first object from the plurality of objects, and a difference in curvature from the first object is a curvature within a predetermined range; and The abnormal object detection program according to claim 1, wherein second objects similar to the characteristics of the first object are classified into the same group as the first object.   3. The abnormal object according to claim 2, wherein the classifying process determines whether the features between the objects are similar after aligning the directions of both ends and the convex direction of each object to be compared. Detection program.   The classification process stores the curvature and characteristics of each object belonging to a group in a storage unit, and the determination process is performed when each object stored in the storage unit is detected when a new object is detected from a captured image. The process for determining whether or not to output a newly detected object based on the curvature and feature of the object and the curvature and feature of the newly detected object is further executed. The abnormal object detection program described. An abnormal object detection method executed by a computer,
Calculate the curvature of each of the multiple objects extracted from the captured image,
Based on the calculated curvatures of the plurality of objects, by comparing the characteristics of the objects whose difference in curvature is within a predetermined range, the plurality of objects are classified into a plurality of groups,
An abnormal object detection method comprising: executing a process of determining a group in which the number of belonging objects is less than a predetermined number and outputting an object belonging to the determined group. A calculation unit that calculates the curvature of each of a plurality of objects extracted from the captured image;
Based on the calculated curvatures of the plurality of objects, a classification unit that classifies the plurality of objects into a plurality of groups by comparing the characteristics of the objects whose difference in curvature is within a predetermined range;
An abnormal object detection apparatus comprising: a determination unit that determines a group in which the number of belonging objects is less than a predetermined number, and outputs an object belonging to the determined group.

Images