JP2004295781A - Image recognition method and image recognition apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition method and an image recognition apparatus capable of positively discriminating an object to be monitored from the profile shape of an imaged moving object and realizable with a comparatively low-cost device. <P>SOLUTION: The image recognition apparatus 1 is provided with a monitoring camera 1 for imaging the moving object and outputting its moving image data; a frame separating means 3 for inputting the moving image data and separating it in frame images of prescribed time intervals; a converting means 5 for determining the presence of an image change on two sequentially separated frame images for every pixel and converting the frame images into numerical string data based on the determined results; a code generating means 6 for generating a shape code corresponding to the profile shape of the moving object based on the numerical string data; and a discriminating means 7 for discriminating whether the moving object is the object to be monitored, based on the shape code. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image recognition method and an image recognition apparatus, and more particularly to an image recognition method for recognizing a monitoring target such as a human, an animal, and a car by image recognition, and an apparatus using the method.
[0002]
[Prior art]
Conventionally, as a method of recognizing a monitoring target such as a human or a car based on image data, a difference between previously registered background image data and image data of a moving object is acquired, and the difference image data is acquired. There is known a method of recognizing a monitoring target based on the following.
[0003]
The above prior art is naturally performed by those skilled in the art, and the applicant does not know the document describing this prior art.
[0004]
[Problems to be solved by the invention]
However, since this method involves high-level image processing, an expensive device such as a computer having a high processing capability is required, and a relatively long time is required to recognize the monitoring target. Further, when the shape of the monitored object is not constant, that is, when the contour of the monitored object changes as needed due to its operation, it is determined whether or not the captured moving object is the monitored object (human). Had become difficult. In addition, when the position of the moving object with respect to the camera or the distance between the camera and the moving object is different, the size of the moving object to be imaged is different, which makes the determination difficult.
[0005]
In view of the above circumstances, the present invention provides an image recognition method and an image recognition device that can reliably determine a monitoring target from a contour shape of a captured moving object and can be realized by a relatively inexpensive device. It is.
[0006]
[Means for Solving the Problems]
The image recognition method according to the present invention includes: a frame separation step of separating moving image data of a moving object into frame images at predetermined time intervals; and, for two frame images separated in sequence, the presence or absence of an image change for each pixel. Or a determination step for each divided small area, and a conversion step of converting the frame image into numerical sequence data based on the determination result; and a shape code corresponding to the contour shape of the moving object based on the numerical sequence data. And a discriminating step of discriminating whether or not the moving object is a monitoring target based on the shape code.
[0007]
According to this image recognition method, moving image data is subjected to frame separation in a frame separation step, and subsequent recognition processing is performed based on the frame image. The time interval of the frame separation at this time is appropriately set according to the speed of the operation to be recognized, the accuracy of the recognition, and the processing capability.
[0008]
In the subsequent conversion step, the change of the image in the two frame images separated at a predetermined time interval is determined for each pixel or for each small area divided in advance, and a numerical value based on the determination result is given. For example, when it is determined that there is a change, “1” is output, and when it is determined that there is no change, “0” is output. In this way, the frame image is changed to the numerical sequence data. In the next code generation step, a shape code corresponding to the contour shape of the moving object is generated based on the numerical sequence data. In the determining step, it is determined whether the moving object is a monitoring target by comparing the generated shape code of the moving object with a preset shape code of the monitoring target.
[0009]
The image recognition device according to the present invention is configured such that an imaging unit that images a moving object and outputs moving image data thereof, a frame separation unit that inputs the moving image data and separates the moving image data into frame images at predetermined time intervals, Conversion means for judging the presence or absence of an image change for each of the two frame images for each pixel or for each small area previously divided, and converting the frame image into numerical sequence data based on the judgment result; and Code generating means for generating a shape code corresponding to the contour shape of the moving object based on the above, and determining means for determining whether or not the moving object is a monitoring target based on the shape code It is.
[0010]
According to this image recognition apparatus, similarly to the above-described image recognition method, the moving image data captured by the imaging unit is separated into frames by the frame division unit, and the subsequent recognition processing is performed based on the frame image. Further, the changing unit determines whether or not there is a change in the frame image for each pixel or for each small area divided in advance, and gives a numerical value based on the determination result. After that, the code generation means generates a shape code corresponding to the contour shape of the moving object based on the numerical sequence data. Then, the determining means determines whether the moving object is a monitoring target by comparing the generated shape code of the moving object with a preset shape code of the monitoring target.
[0011]
In the image recognition apparatus, the code generation unit may include a coordinate detection unit configured to detect a coordinate of a feature point in the contour shape of the moving object, and a length and a slope of a line segment connecting the feature points may be calculated. And generating the shape code based on a combination of the length and the inclination of each of the line segments.
[0012]
Here, the feature points are a plurality of points located on the contour of the moving object. As a method of setting the feature points, points on the contour may be sampled at regular intervals, but the vertices or inflection points of the curve constituting the contour (the curve indicating the contour shape changes from convex to concave or from concave to concave) (A point that changes to a convex shape) may be extracted.
[0013]
According to this image recognition device, when generating the shape code, the coordinates of the characteristic points in the contour shape of the moving object are detected by the coordinate detection means. By sequentially connecting the plurality of feature points, a polygon having a simplified outline shape, that is, a figure consisting of only straight lines is formed. Therefore, the line segment calculating means calculates the length of the line segment connecting each feature point and its inclination, and generates a shape code based on a combination of the length and the inclination of each line segment, that is, information on a polygonal figure. . It should be noted that a graphic consisting only of straight lines has a much smaller amount of information than a collection of curves forming the contour shape of the moving object, so that the processing can be simplified.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an image recognition apparatus according to an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a functional configuration of the image recognition device, FIGS. 2 and 3 are explanatory diagrams for explaining an image recognition method, and FIG. 4 is a flowchart showing a flow of processing in the image recognition device. is there.
[0015]
The image recognition device 1 according to the present embodiment is configured to determine a movable monitoring target such as a human, an animal, and a car based on the outline shape thereof. The present invention can be applied to a home security system to be operated, a vehicle recognition system for discriminating the type of a running automobile, and a separation system for recognizing and separating the type of industrial products flowing on a conveyor.
[0016]
In the present embodiment, a case where the image recognition device 1 is applied to a home security system will be described as an example. As a home security system, a system equipped with a surveillance camera has been conventionally known.However, in the conventional system, it is not possible to distinguish between a human being an intruder and a pet such as a dog or a cat, so that a pet is indoors. When the resident goes out of the house, it is necessary to take measures such as taking out the pet with the resident or putting the pet in a circle or a krait so as not to move around the room. However, if the image recognition device 1 of the present invention is incorporated in a home security system, it is possible to distinguish between an intruder and a pet (for example, a dog), and to activate the notification means only when the intruder is determined. Becomes possible.
[0017]
As shown in FIG. 1, the image recognition device 1 of the present embodiment includes a monitoring camera 2 (corresponding to an imaging unit of the present invention) such as a CDD camera and the like. It has a functional configuration for determining whether a moving object is a monitoring target (that is, an intruder) or a pet. Note that this functional configuration is realized by an execution program stored in a storage unit of a general-purpose computer, and includes a frame separation unit 3, a conversion unit 5, a code generation unit 6, and a determination unit 7. I have. The frame separating unit 3 receives moving image data from the monitoring camera 2 and generates frame images (still images) separated at predetermined time intervals (for example, every 0.1 second). The separated frame images are stored in the image storage unit 9 of the storage device 8. Here, the reason why the frame images are stored in the image storage unit 9 is to compare with the previous frame image when the latest frame image is separated.
[0018]
The conversion unit 5 includes a change determination unit 10 that reads the latest frame image and the previous frame image separated at a predetermined time interval from the image storage unit 9 and determines whether or not there is a change in the image for each pixel. The frame image is converted into numerical value sequence data based on the determination result. More specifically, the change determination unit 10 compares the amount of change detected for each pixel with a predetermined threshold value, and determines a pixel whose amount of change is larger than the threshold value as a moving pixel (moving pixel). Pixel). As a result, only a moving part can be extracted from the subject. That is, it is possible to separate the background with no movement and the moving object with movement from the subject. Thus, the conversion means 5 outputs, for example, “1” to a moving pixel and “0” to a still pixel, and then arranges these numerical values in a predetermined order to obtain a numerical value sequence relating to a moving object. Create data.
[0019]
The code generation means 6 generates a shape code corresponding to the contour shape of the moving object based on the numerical sequence data generated by the conversion means 5, and includes a coordinate detection means 11 and a line segment calculation means 12. are doing. The coordinate detecting means 11 obtains a plurality of feature points for a curve constituting the contour shape of the moving object and detects the coordinates. For example, as shown in FIG. 2A, when the shape of the extracted moving object is a human upper body, vertices T1, T2, T3,... Tn of a plurality of curves constituting the contour shape are set as feature points. Set. By connecting these feature points, a polygon having a simplified outline shape, that is, a figure consisting of only straight lines as shown in FIG. 2B is formed. The line segment calculating means 12 calculates the lengths B, C, D,... Of the line segments connecting the feature points, and the interior angles b, c, d,. , And a shape code (for example, BbCcDd...) Composed of a combination of an inner angle and an inner angle is generated and output.
[0020]
The determining means 7 shown in FIG. 1 determines whether the moving object is a human (intruder) or a pet such as a dog based on the shape code generated by the code generating means 6, Here, an aspect ratio determining unit 13, a region specifying unit 14, and a region determining unit 15 are provided. First, the aspect ratio determining means 13 determines whether the outline shape of the entire moving object is vertically long or horizontally long. If the portrait shape is long, the moving object is a human, and if it is horizontally long, the moving object is a non-human moving object. Is determined. This is because, as shown in FIG. 3, the human H often moves in a standing posture, and the animal A such as a dog or a cat often moves in a prone posture. Is determined.
[0021]
On the other hand, the part specifying unit 14 specifies a part of the body included in the subject based on the shape code generated by the code generating unit 6. In the code storage unit 16 of the storage device 8, shape codes representing the outline shape of each part of the body such as the head, the torso, and the foot are stored in advance, and the generated shape code and the code storage unit 16 are stored. By comparing the generated shape code with a shape code stored in the form code, the generated shape code is identified as a shape code representing a portion. For example, when a shape code that is elongated and extends in the vertical direction is generated as the shape code, the shape code indicating the foot is specified.
[0022]
The part determining means 15 determines whether or not the detected moving object is a human according to the part specified by the part specifying means 14. Specific examples of the discrimination include a joint discrimination unit 17 for the foot, a head discrimination unit 18 for the head, a shoulder width discrimination unit 19 for the shoulder, and a tail discrimination unit 20 for the buttocks.
[0023]
When the part specified by the part specifying unit 14 corresponds to the vicinity of the “foot”, the joint determination unit 17 determines based on “the formation of the joint of the foot”. This is because, as shown in FIG. 3, the leg of the human H has a “U” shape due to the bending operation of the femur 25 and the tibia 26 extending up and down around the knee, whereas a dog or the like. Of the animal A, the tibia 27 and the metatarsal 28 are formed in the shape of an inverted "ku" extending in the vertical direction from the front with respect to the fly segment. It is determined by using the information. In other words, if the generated shape code is in the shape of a “ku” in the traveling direction (moving direction), it is determined that the person is a human H (intruder). If so, it is determined that the animal is an animal A such as a dog. This determination is effective when the moving object crosses the monitoring camera 2 in the left-right direction, that is, when the side surface of the moving object is imaged.
[0024]
When the part specified by the part specifying unit 14 corresponds to the vicinity of the “head”, the head determining unit 18 determines based on “the position of the head with respect to the trunk”. As shown in FIG. 3, when the moving object is viewed from the side, the head of the human H is located above the torso, whereas the head of the animal A such as a dog is located obliquely above the torso. Therefore, the determination is made using the difference between these relative positions. That is, when the size of the moving object hardly changes, that is, when the moving object crosses the surveillance camera 2 in the left-right direction and the head is positioned above the torso, the human H (invading On the other hand, if the head is positioned diagonally above or in front of the torso, it is determined to be an animal A such as a dog.
[0025]
When the part specified by the part specifying means 14 corresponds to the vicinity of the “shoulder and head”, the shoulder width determining unit 19 determines based on “the width of the shoulder width relative to the head”. This is because when looking at the front or back of the moving object, the human H has a wide shoulder width (width of the torso), whereas the animal A such as a dog has a narrow width of the torso. It is determined by using the information. That is, when the size of the moving object changes, that is, when the moving object moves in the perspective direction with respect to the monitoring camera 2 and the shoulder width with respect to the head is larger than the predetermined ratio, the moving object is a human H (intruder). On the other hand, when the ratio is smaller than the predetermined ratio, it is determined that the animal is an animal A such as a dog.
[0026]
When the part specified by the part specifying means 14 corresponds to the vicinity of the “tail”, the tail determination unit 20 determines based on “the presence of the tail”. Since the human H does not have a tail like a dog, it is determined using this difference. That is, when an elongated cord-like shape is recognized near the buttocks, the animal is determined to be an animal A such as a dog.
[0027]
Note that the discrimination between the human H and the animal A is not limited to the above method. For example, any shape or posture that can distinguish the human H and the animal A, such as the position of the arm or the overall size, is used. It is possible to determine using the difference.
[0028]
By the way, in the present embodiment, since the moving object is extracted based on the presence or absence of a change in the frame image, that is, the difference between the frame images separated at predetermined time intervals, a swinging curtain or laundry may be recognized as the moving object. There is. However, according to the present invention, since it is determined whether or not a person is a person according to a characteristic part of the person, the notification means is not activated even if the curtain or the laundry is blown by the wind.
[0029]
When the image recognition device 1 or the monitoring camera 2 built in the image recognition device 1 swings due to an earthquake, the entire frame image is determined to be a moving pixel, and the background itself may be recognized as a moving object. However, in this case, the contour shape of the moving object becomes the shape of the entire screen and is clearly different from the contour of a human, so that it is not determined to be a human.
[0030]
Next, the flow of image processing in the security system 1 of the present embodiment will be described with reference to the flowchart shown in FIG. First, moving image data is separated into frame images in a frame separation step (step S1), and subsequent recognition processing is performed based on the frame images. The time interval of the frame separation at this time is appropriately set according to the speed of the operation to be recognized, the accuracy of the recognition, and the processing capability. The separated frame image is stored in the image storage unit 9 (Step S2).
[0031]
Subsequently, the presence / absence of a change is determined for each pixel of the sequentially separated frame images (step S3), and the presence / absence of the change is converted into numerical sequence data (step S4). Then, feature points in the contour shape of the moving object are extracted from the numerical value sequence data and their coordinates are detected (step S5), and the length and interior angle of the line connecting each feature point are detected (step S6). A shape code is generated based on the shape code (step S7). Here, the processing of steps S3 and S4 corresponds to the conversion step of the present invention, and the processing of steps S5 to S7 corresponds to the code generation step of the present invention. Thereafter, it is determined whether or not the moving object is a human (intruder) based on the shape code indicating the external shape of the moving object (step S8). In the home security system, when it is determined that the person is an intruder, that is, when it is determined that at least one of the determination units is a human, the notification unit is activated.
[0032]
As described above, in the image recognition device 1 described above, when applied to a home security system, it is possible to accurately determine whether a captured moving object is an intruder or a pet. Only the notification means can be operated. For this reason, it is not necessary to take any measures such as taking the pet out of the room or putting the pet in a circle or clay so as not to move around the room, and the usability can be greatly improved. In addition, the image recognition device 1 described above has a plurality of discriminating units corresponding to the characteristic parts of the monitoring target, so that the accuracy of the discrimination can be greatly improved.
[0033]
Further, in the image recognition device 1 described above, the amount of data to be subjected to recognition processing can be reduced by efficiently processing moving image data having enormous information. Thus, real-time processing can be performed by a general-purpose computer, and a relatively inexpensive image recognition device can be provided.
[0034]
As described above, the present invention has been described with reference to the preferred embodiments. However, the present invention is not limited to these embodiments, and various improvements and modifications can be made without departing from the scope of the present invention as described below. Design changes are possible.
[0035]
That is, in the image recognition device 1 described above, the presence or absence of a change in the frame image is determined for each pixel. However, the image recognition device 1 includes an area dividing unit that divides the frame image into a plurality of small areas. The presence or absence of the change may be determined for each of the divided small areas. Since the number of divisions of the frame image is set to be much lower than the total number of pixels of the frame image, it is possible to simplify the processing by this region division.
[0036]
In the above-described image recognition apparatus 1, when generating the shape code, the one that calculates the interior angle between adjacent line segments as the line segment inclination has been described, but the inclination angle and the vector of the line segment may be calculated. Good.
[0037]
In the above embodiment, the image recognition device is applied to the home security system to discriminate between a human (intruder) and a pet. However, the monitoring target is not particularly limited and has a contour shape. A moving object having a characteristic can be determined by the image recognition device of the present invention. For example, when the present invention is applied to a vehicle recognition system, it is possible to determine the type of a running car from its contour shape, and it is possible to use it for searching for a arranged vehicle, investigating traffic conditions, and the like. In addition, when the present invention is applied to a sorting system, for example, products flowing on a conveyor can be determined according to a contour shape and automatically sorted. In particular, since the image recognition device of the present invention can be realized by a general-purpose inexpensive computer, its application range is wide, and it is applied to a wide range of fields such as security equipment, production control devices, safety devices, amusement machines, and toys. It is possible to do.
[0038]
【The invention's effect】
According to the present invention, a shape code corresponding to the contour shape of a moving object is generated, and the shape code is analyzed to determine whether or not the monitored object is a monitored object. Can be accurately recognized. In addition, by efficiently processing moving image data having enormous information, the amount of data to be subjected to recognition processing can be reduced. As a result, real-time processing can be performed by a general-purpose computer, and can be realized by a relatively inexpensive device.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a functional configuration of an image recognition device according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram for explaining an image recognition method.
FIG. 3 is an explanatory diagram for explaining a difference in contour between a human and a dog.
FIG. 4 is an explanatory diagram showing a flow of processing in the image recognition device.
[Explanation of symbols]
1 image recognition device 2 surveillance camera (imaging means)
3 frame separating means 5 converting means 6 code generating means 7 discriminating means 11 coordinate detecting means 12 line segment calculating means

Claims (3)

A frame separation step of separating moving image data of a moving object into frame images at predetermined time intervals,
For the two frame images separated in order, the presence or absence of a change in the image is determined for each pixel or for each small region divided in advance, a conversion step of converting the frame image into numerical sequence data based on the determination result,
A code generation step of generating a shape code corresponding to the contour shape of the moving object based on the numerical sequence data;
A determination step of determining whether or not the moving object is a monitoring target based on the shape code. Image capturing means for capturing a moving object and outputting moving image data thereof; frame separating means for inputting the moving image data and separating the moving object data into frame images at predetermined time intervals;
For two frame images separated in order, a conversion unit that determines the presence or absence of a change in the image for each pixel or for each small region divided in advance, and converts the frame image into numerical sequence data based on the determination result.
Code generation means for generating a shape code corresponding to the contour shape of the moving object based on the numerical sequence data,
An image recognition apparatus comprising: a determination unit configured to determine whether the moving object is a monitoring target based on the shape code. The code generation means includes:
Coordinate detection means for detecting the coordinates of a feature point in the contour shape of the moving object,
Line segment calculation means for calculating the length of the line segment connecting each of the feature points and the inclination thereof,
The image recognition apparatus according to claim 2, wherein the shape code is generated based on a combination of the length and the inclination of each line segment.

Images