JP4218963B2 - Information extraction method, information extraction apparatus, and recording medium - Google Patents

Description

  The present invention relates to an information extraction apparatus and method for extracting target information such as a movement and position of a recognition target from a distance image.

  Conventionally, in playground equipment such as arcade game machines, many games using human movement such as a boxing game and a shooting game have been developed. In the boxing game, the pressure when the user actually hits the sandbag is physically measured. At this time, only the pressure at the moment when the user's punch hits the sandbag is measured. Information on how and how fast the punch is fed from the user is not used. For this reason, it is a monotonous game that competes only for the strength of a simple punch.

  In the shooting game, a light emitter is attached to the tip of a toy gun, and the light on the screen is detected where the light is. When a gun trigger is fired, it is determined that a bullet has hit an object that is exposed to light. Because of this simplicity, rather than competing for shooting skills, it has become a game that only competes for reflexes to quickly find and shoot the presented target.

  On the other hand, a method of recognizing the position and movement of a human body by analyzing an image captured using a CCD camera or the like is used. In this method, the traveling direction is first photographed using an imaging device such as a CCD camera. In the case of extracting a human from the image, for example, since the color of the face or hand is a skin color, a preprocessing is performed such that an extra portion such as the background is removed and only an object to be recognized such as an obstacle is cut out. . Then, the shape, movement, etc. of the object are recognized from the processed image.

  First, the pre-processing part of extracting the recognition target will be described. In the conventional technique, as an operation means for cutting out only a part of an object to be acquired from an image photographed by a camera, the object is cut out by using some difference between the object and other parts as a clue.

  As a clue, a method using a change in hue, a method using a difference image, and the like are used. When cutting out using colors, a portion with a large hue difference is extracted, and processing such as thinning is performed to extract edges. When targeting humans, attention is paid to the skin color of the face and hand parts, and only the hue part is extracted. However, the method of using the hue difference changes depending on the color and angle of the lighting, even if it is called skin color, and if the background hue is close to skin color, it is difficult to distinguish it from human skin color. It is difficult to cut out regularly. In addition, in the state where there is no illumination, the entire captured image becomes dark, and it is difficult for a human to discriminate an object from a photograph taken in the dark.

  When extracting information such as movement and shape of the recognition target from the captured image of the recognition target, when there are a plurality of the recognition targets, for example, a person kneads clay using both the right hand and the left hand. When using both hands, such as engraving or performing operations such as deforming the corners of a photograph, it is necessary to recognize each hand separately (movement and shape extraction). For example, the right hand image and the left hand image must be acquired and recognized separately, which is expensive.

  There is also a method of calculating motion vectors between frames of video images and analyzing moving objects, but in this case there is no problem as long as there are few moving objects, but if there are many moving objects, The number of vectors suddenly increases, the motion vector calculation load increases between frames, and the calculation cannot keep up.

  In addition, when extracting the movement of the object, particularly when the object is a human being, the movement includes a lot of useless movements in addition to the target movement, and is unstable. It may be difficult to extract only the desired movement.

  For example, when playing a shooting game using a hand, a human hand is more unstable in terms of position than the user is aware of. For example, suppose your hand is in the shape of a handgun and you decide the position and shoot. Let's say that a bullet is fired at the moment when the shooting position and direction are determined with the index finger hitting the muzzle and the thumb hitting the hammer is bent. Here, the problem is that when the thumb is bent, the pointing direction of the index finger changes slightly when the shooting position and direction are determined. If skilled, it is possible to bend the thumb so that the direction of the index finger does not change. However, if such skill is required for all users, it becomes difficult for everyone to enjoy games and the like easily.

  Even if the operation method is changed so that the trigger is pushed and the bullet is fired by positioning the hand instead of bending the thumb and moving the hand that hits the handgun closer to the target. When moving to move closer, the direction and position of the index finger changes. If you try to move it so that it does not change, the muscles of the lower limbs will be unnaturally strained and your hands will be easier to lift.

  A similar problem is known as camera shake that occurs when the shutter is pressed when shooting with a camera or the like. In the case of a camera, a camera shake prevention function is provided on the assumption that an object to be imaged is stopped, that is, a still image. However, since gestures using the hands and body are premised on moving, it is impossible to take a method of stabilizing the operation of the hand, such as a camera shake prevention function mounted on the camera.

  An object of the present invention is to provide an information extraction apparatus and method that can extract a motion of a recognition target with high accuracy from an acquired distance image without being affected by extra motion other than the motion to be recognized.

  The present invention irradiates the object with light from the light emitting means and receives the reflected light from the object with the light receiving means, whereby each elementary value indicates the intensity of the reflected light received by the light receiving means, A plurality of time-series captured images of the object to be recognized, acquired by distance image acquisition means for acquiring a distance image representing the distance to the object and the three-dimensional shape of the object based on the intensity of the reflected light indicated by the value Based on each prime value of at least one distance image of the distance images, a base image that is an image other than the target object is generated, and for each distance image acquired by the distance image acquisition means, the base image Motion to be recognized by obtaining the difference image and the change direction of the position of the object between different distance images from the three-dimensional coordinates indicating the position of the object of each difference image. To extra movement other than Without being sound can be extracted motion of the recognition target with high precision.

  The movement of the recognition target can be extracted from the acquired distance image with high accuracy without being affected by extra movement other than the movement to be recognized.

Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 shows a configuration example of an information extraction apparatus according to the first embodiment of the present invention, in which information on the shape, movement, position, etc. of an object is extracted from a plurality of locations in one imaging region ( The extracted information is input to another system such as a mole hitting game.

  As shown in FIG. 1, the information extraction apparatus according to this embodiment receives reflected light and acquires a distance image, for example, a distance image acquisition unit 1 described in Japanese Patent Application No. 9-299648, and an acquisition The area division processing unit 2 that divides the distance image into separately designated areas, and the distance image is analyzed for each area divided by the area division processing unit 2 to extract the contour and center of gravity of the object and the object An image processing unit 3 that extracts target information about the recognition target such as calculation of distance to the target, movement speed of the target object, calculation of movement vector, and the like, and a presentation unit composed of, for example, a large liquid crystal panel or a speaker 4 and an information management unit 5 that controls the above-described units.

  The information extracted by the image processing unit 3 is input to another system via the information management unit 5, for example. That is, as another system, for example, a mole hitting game or the like is connected via the information management unit 5. The other system may be a program that can be executed by a computer, for example.

  The information extraction apparatus having the configuration as shown in FIG. 1 causes, for example, a mole to appear from an arbitrary display area on the display screen of the presentation unit 4, and the user pulls toward the display area where the mole appears (user displays If the player is connected to a mole hitting game that competes for the number of moles hit, the one distance image acquisition unit 1 can select which one on the display screen of the presentation unit 4 It is installed so that it can recognize whether it is trying to pull it toward the mole (so that the position of the hand as an object relative to the entire display screen can be recognized). Then, the area division processing unit 2 performs area division of the distance image acquired by the distance image acquisition unit 1 corresponding to the mole appearance area on the display screen of the presentation unit 4.

  Here, the distance image acquisition unit 1 and the distance image acquired by the distance image acquisition unit 1 will be briefly described.

  The appearance of the distance image acquisition unit 1 is shown in FIG. A light receiving unit 103 composed of a circular lens and an area sensor (not shown) at the rear thereof is arranged in the center, and the circular lens is surrounded by an LED that emits light such as infrared rays along its outline. A plurality of (for example, eight) light emitting units 101 are arranged at equal intervals.

  The light emitted from the light emitting unit 101 is reflected by the object, collected by the lens of the light receiving unit 103, and received by the area sensor at the rear of the lens. The area sensor is, for example, a sensor arranged in a 256 × 256 matrix, and the intensity of reflected light received by each sensor in the matrix becomes a pixel value. The image acquired in this way is a distance image as the intensity distribution of reflected light as shown in FIG.

  FIG. 3 shows a configuration example of the distance image acquisition unit 1, which mainly includes a light emitting unit 102, a light receiving unit 103, a reflected light extraction unit 102, and a timing signal generation unit 104.

  The light emitting unit 101 emits light whose intensity varies with time in accordance with the timing signal generated by the timing signal generating unit 104. This light is applied to the target object in front of the light emitting unit.

  The light receiving unit 103 detects the amount of light reflected by the target object of the light emitted from the light emitting unit 101.

  The reflected light extraction unit 102 extracts a spatial intensity distribution of the reflected light received by the light receiving unit 103. Since the spatial intensity distribution of the reflected light can be captured as an image, this is hereinafter referred to as a distance image.

  In general, the light receiving unit 103 simultaneously receives not only light reflected from an object of light emitted from the light emitting unit 101 but also external light such as illumination light and sunlight. Therefore, the reflected light extraction unit 102 takes the difference between the amount of light received when the light emitting unit 101 emits light and the amount of light received when the light emitting unit 101 does not emit light, thereby obtaining the light emitting unit 101. Only the reflected light component of the light from the target object is extracted.

  The reflected light extraction unit 102 extracts the intensity distribution, that is, a distance image as shown in FIG. 4 from the reflected light received by the light receiving unit 103.

  For the sake of simplicity, FIG. 4 shows the case of an 8 × 8 pixel distance image that is a part of a 256 × 256 pixel distance image.

  The reflected light from the object decreases significantly as the distance of the object increases. When the surface of the object scatters light uniformly, the amount of light received per pixel in the distance image decreases in inverse proportion to the square of the distance to the object.

  In FIG. 4, the cell value (pixel value) in the matrix indicates the intensity of the acquired reflected light in 256 gradations (8 bits). For example, a cell having a value of “255” is closest to the distance image acquisition unit 1, a cell having a value of “0” is far from the distance image acquisition unit 1, and reflected light is a distance image acquisition unit 1 is not reached.

  Each pixel value of the distance image represents the amount of reflected light received by the unit light receiving unit corresponding to the pixel. Reflected light is affected by the nature of the object (specularly reflects, scatters, absorbs, etc.), the direction of the object, the distance of the object, etc., but the entire object scatters light uniformly. In this case, the amount of reflected light is closely related to the distance to the object. Since hands and the like have such properties, the distance image when the hand is put out in front of the distance image acquisition unit 1 reflects the distance to the hand, the inclination of the hand (the distance is partially different), and the like. A three-dimensional image as shown in FIG. 5 can be obtained.

  Next, the processing operation of the information extraction apparatus configured as shown in FIG. 1 will be described with reference to the flowchart shown in FIG.

  First, when the power is turned on or activated by an operation start instruction, a predetermined initialization process is performed. Thereafter, the distance image acquisition unit 1 uses the light emitting unit 101 and the light receiving unit 103 as shown in FIG. 2, for example, to acquire a distance image at a speed of about one or two per second (step S1). . An example of the distance image acquired at this time is shown in FIG. In FIG. 8A, there are two objects, the right hand and the left hand, and it is necessary to recognize the movement, shape, position, etc. for each of them.

  Therefore, next, the area division processing unit 2 receives information (area information) regarding the area to be divided from the information management unit 5. For example, in the case of the mole hitting game as described above, the area on the distance image corresponding to each of the display areas of the moles that appeared on the display screen of the presentation unit 4 during the distance image acquisition interval (for example, 1 second). As information, the end point coordinates, vertical length, and horizontal length of the area are read from the information management unit 5 (step S2). Based on this information, for example, the distance image as shown in FIG. 8A is divided as shown in FIG.

  For example, assume that two moles appear on the display screen of the presentation unit 4. First, the area information of the distance image corresponding to the display area of the first mole from the information management unit 5 (the upper left end point T1 is (x1, y1), the upper right end point T2 is (x2, y2), and the vertical length is L1. When the horizontal length R1) is given, the image area AR1 as shown in FIG. 8B is divided from the distance image shown in FIG. Also, area information of the distance image corresponding to the display area of the second mole (the upper left end point T2 is (x2, y2), the upper right end point T3 is (x3, y3), the vertical length is L2, and the horizontal length is When R2) is given, the image area AR2 as shown in FIG. 8B is divided from the distance image shown in FIG.

  In FIG. 8B, the divided image areas are not overlapped, but may be overlapped. Although the number of image areas to be divided is two in FIG. 8B, it can be divided into a larger number. Or the area of the image area | region to divide | segment does not necessarily need to be equal. Further, the shape of the image area to be divided is not limited to a rectangle, and can be divided into various shapes such as a triangle, a polygon, and a circle.

  After the area division, the area division processing unit 2 assigns an image processing process to each divided image area (step S4). Here, the image processing processes assigned to the respective image regions may all be the same, or may be different depending on circumstances. In any case, the image processing unit 3 needs to prepare an image processing process in advance according to the maximum number of divided image areas to be processed.

  The area division processing unit 2 repeats the above processing operations from step S2 to step S4 until there is no area information to be read from the information management unit 5 (step S5).

  In the image processing unit 3, only the one corresponding to the image region divided by the region division processing unit 2 among the distance image data sent from the distance image acquisition unit 1 as shown in FIG. Various image processing for extracting information such as the shape, movement, and position of the imaging body, such as edge extraction (contour extraction of the imaging object), centroid extraction, area calculation, distance calculation to the imaging object, and motion vector calculation (Step S6). For example, here, it is assumed that two identical image processing processes assigned to each of two divided image regions (divided images) are executed in parallel.

  In each of the image processing processes assigned to each divided image of the image processing unit 3, first, except for cells whose pixel values are equal to or smaller than a predetermined value, for example, as shown in FIG. The contour information of the imaged object is extracted.

  In order to extract the contour information as shown in FIG. 6, the pixel values of adjacent pixels are compared, a constant value is inserted only where the pixel value is equal to or greater than a certain value α, and consecutive images assigned with the same constant value. What is necessary is just to extract the pixel of an area | region.

That is, if the pixel value at the coordinate position (i, j) on the matrix is P (i, j) and the pixel value of the contour information is R (i, j),
{P (i, j) -P (i-1, j)}> α, and {P (i, j) -P (i, j-1)}> α, and {P (i, j) −P (i + 1, j)}> α and {P (i, j) −P (i, j + 1)}> α
Then R (i, j) = 255
・ R (i, j) = 0 in other cases
By doing so, the contour information of the object as shown in FIG. 6 can be obtained.

  Each image processing process of the image processing unit 3 calculates the center of gravity of the object based on the contour information as shown in FIG. 6 extracted from each divided image.

  It can be determined whether or not the object is moving by comparing the position of the center of gravity with the position of the center of gravity extracted from the immediately preceding frame. When it is determined that the object is moving, a motion vector representing the change amount and the change direction of the center of gravity is calculated. Further, not only the position of the center of gravity but also the contour information extracted from each of the continuous distance images may be compared with each other to obtain a motion vector of an arbitrary point on the contour.

In order to calculate the distance d to the object from which the contour is extracted, first, the representative pixel value of the image near the center of gravity of the object is obtained. There are several representative pixel values such as an average value and a nearest neighbor value. Here, the nearest neighbor value is used. The intensity of reflected light from the object decreases in inverse proportion to the square of the distance to the object. That is, if the representative pixel value of the image of the object is Q (i, j),
Q (i, j) = K / d ² (1)
It can be expressed as.

  Here, for example, K is a coefficient adjusted so that the value of the pixel value R (i, j) becomes “255” when d = 0.5 m. The distance value d can be obtained by solving the equation (1) for d.

  In this way, information such as the shape, movement, and position of the target object can be extracted from each of the divided images.

  For example, in the case of a mole hitting game, when there are two moles appearing on the display screen of the presentation unit 4 during a distance image acquisition interval (for example, 1 second), the image processing unit 3 The image processing as described above is performed on the divided images corresponding to the display areas of the two moles among the distance images acquired in 1. In this case, if the centroid position (for example, three-dimensional coordinates including the distance value) of the contour information is extracted from each of the two divided images as described above (the user's hand as shown in FIG. 8). Good. The barycentric position extracted from each of the divided images is sent to the information management unit 5. The information management unit 5 may determine that the mole has been hit when the position of the center of gravity extracted from each divided image is within a predetermined range. When it is determined that the mole has been hit, the hit mole displayed on the presentation unit 4 is screamed, or a score corresponding to the number of hit moles displayed on the presentation unit 4 is added. Etc. are performed (step S7).

  As shown in FIG. 8, the distance image acquired by the distance image acquisition unit 1 is captured by the distance image acquisition unit 1 when both the right hand and the left hand are imaged and the movements of both are to be recognized. In order to perform image processing as it is (without dividing an area), an image processing program corresponding to the image processing has to be newly developed. However, in the information extraction apparatus of the present invention, the area division processing unit 2 divides the distance image as shown in FIG. 8A into two left and right areas as shown in FIG. 8B. Then, by performing the same image processing separately for each of the divided images, for example, even when the number of recognition targets increases, if the number is within the number of image processing processes prepared in advance, for example, Therefore, it is possible to deal flexibly (without newly developing an image processing program) by newly dividing an area where the recognition target exists and assigning an image processing process.

  That is, if the number of recognition targets increases, the information management unit 5 may hold the area information corresponding to the increase. Alternatively, a user interface may be provided for presenting the distance image acquired by the distance image acquisition unit 1 to the presentation unit 4 and allowing the user to instruct and input a desired region based on the distance image. The user may input a position where a desired recognition target exists using a pointing device such as a mouse or a pen with respect to the distance image presented on the presentation unit 4. The information management unit 5 holds the input area information.

  The area division processing unit 2 may read the area information from the information management unit 5 and divide the distance image acquired by the distance image acquisition unit 1.

  For example, when the information extraction device of FIG. 1 is connected to a mole hitting game, the distance image acquired by the distance image acquisition unit 1 according to the mole arranged to be displayed on the display screen of the presentation unit 4 Divide and recognize the center of gravity position in each divided image. For example, if there are 16 moles and there are only 4 moles to be hit now, the center of gravity position is recognized only for the divided images where the 4 moles exist, and which mole is hit. Since it only needs to be determined, high-speed processing can be performed.

  The information extraction apparatus of the present invention is not applied only to a mole hitting game, but can be applied to various systems. For example, when applied to a system in which an instrument is played using both hands, it is possible to play a plurality of instruments, such as a cymbal with the right hand and a timpani with the left hand. In that case, for example, the image processing process assigned to each image area extracts the center of gravity position of the hand in the image area, and the movement speed in the depth (z-axis) direction and the movement in the left-right (x-axis) direction. The speed is calculated. The system that has received the image processing result assigns the speed of movement in the depth (z-axis) direction to the volume of the sound to be played, and assigns the speed of movement in the left and right (x-axis) direction to the frequency of the sound to be played. It is possible to play easily and intuitively.

  It should be noted that when the range image is divided by the area division processing unit 2, if the size of one divided image is too small, even a subtle movement unrelated to the movement of the object to be recognized is recognized, which is not preferable. For example, when the information extraction apparatus in FIG. 1 is used in another portable system, the human hand unconsciously makes a fine movement that is meaningless (for example, camera shake), and is divided into individual pieces. If the image is too small, the movement of the recognition target cannot be accurately recognized due to the influence of the camera shake. For this reason, it is desirable that a lower limit of the size of the divided image is provided in advance so that the size of the divided image does not become so small that camera shake affects the extraction of the movement of the recognition target.

  As described above, according to the first embodiment, even when there are a plurality of recognition objects in the acquired distance image, the distance image is divided according to the number of recognition objects, and an image is obtained for each divided image. By performing each processing, it is possible to easily extract information such as the movement, shape, and position of each of the plurality of recognition objects from the distance image.

(Second Embodiment)
FIG. 9 shows a configuration example of an information extraction apparatus according to the second embodiment of the present invention. In addition, the same code | symbol is attached | subjected to FIG. 1 and an identical part, and only a different part is demonstrated. That is, the area division processing unit 2 in FIG. 1 is replaced with the base extraction unit 6 in FIG.

  The base extraction unit 6 extracts a portion that becomes a base of relative movement, such as the upper arm or the body, when extracting the relative movement of the lower arm with respect to the upper arm or the relative movement of the arm with respect to the body. Is.

  For example, when a human is enjoying a sport or a game using a tool, the whole body is moving, but the thrown ball is a racket in the action of throwing the thrown ball against the racket and throwing it back. It is sufficient to analyze only the movement of the part of the arm that holds the person's racket, etc. There is no need to analyze how the body is moving. Accordingly, in the present invention, focusing on this point, the movement of the object to be recognized (for example, the movement of the arm) is recognized as a relative movement with respect to the movement of the base (for example, the movement of the trunk). More specifically, after deleting an object other than the object to be recognized (base image) from the acquired distance image, image processing for extracting target information regarding the object to be recognized is performed. ing.

  Next, the processing operation of the information extraction apparatus in FIG. 9 will be described with reference to the flowchart shown in FIG.

  First, when the power is turned on or activated by an operation start instruction, a predetermined initialization process is performed. Thereafter, the distance image acquisition unit 1 acquires a distance image at a speed of, for example, about 100 sheets per second using the light emitting unit 101 and the light receiving unit 103 as illustrated in FIG. 2 (step S11).

  For example, when the distance image acquisition unit 1 acquires 100 distance images per second, the base extraction unit 6 extracts one or two of the acquired distance images every second (base extraction timing). (Step S12). Assume that the base extraction unit 6 captures a distance image as shown in FIG. 11 at a base extraction timing.

  The base extraction unit 6 extracts, for example, an image having a distance value larger than a certain value (that is, far away) from the distance image as shown in FIG. 11 as a base image. Alternatively, the base image is extracted based on a criterion such as a distance value close to a certain value or a portion having the least movement (step S13).

  A case where a base image is extracted based on the distance value will be described. The distance image as shown in FIG. 11 is actually the intensity distribution of the reflected light as shown in FIG. 4. As described above, the pixel value is the distance to the distance image acquisition unit 1 of the imaging body. It may be considered that the value is expressed (for example, the larger the pixel value, the farther the image pickup body is, and the smaller the pixel value, the closer the image pickup body is). Therefore, if the distance image acquisition unit 1 is fixed, a correspondence relationship between a required pixel value and a distance value can be obtained in advance from the distance image acquired on a trial basis and the equation (1). It is preferable that the base extraction unit 6 includes a table in which the correspondence relationship is recorded in advance. The distance value used as a reference is determined according to the feature of the base to be extracted (or the feature of the object to be recognized), and a pixel value far from the reference distance value is determined by referring to the correspondence relationship. The base image is extracted by extracting or by extracting a pixel value that is in the vicinity of the reference distance value.

  A case where a base image is extracted based on movement will be described. As described above, the contour information of the object is extracted from the distance image sent to the base extraction unit 6 at each base extraction timing, and the center of gravity of the object is obtained. What is necessary is just to compare the gravity center position of the corresponding objects calculated | required from each distance image, and to extract as a base image of the image of the object whose change amount is below a predetermined value.

  For example, FIG. 14 shows an example of a base image obtained by extracting pixels from which the distance value is equal to or greater than a predetermined value (that is, the pixel value is equal to or less than the predetermined value) from the distance image as shown in FIG.

  After extracting the base part at each base extraction timing, the base extraction unit 6 then obtains a difference (difference image) from the base image from the distance image acquired by the distance image acquisition unit 1 (step S14). That is, for example, as shown in FIG. 14, a torso image is extracted as a base image from a distance image obtained by imaging the upper body of a human, and thereafter, from the distance image acquisition unit 1, shown in FIGS. 11, 12, and 13. Suppose that a distance image that captures the upper body of a person is captured. When the difference between the distance image shown in FIG. 11 and the base image as shown in FIG. 14 is obtained, a distance image of both human arms is obtained as the difference image, as shown in FIG. Similarly, FIG. 16 shows an example of a difference image between the distance image shown in FIG. 12 and the base image as shown in FIG. An example of the difference image between the distance image shown in FIG. 13 and the base image as shown in FIG. 14 is shown in FIG.

  For the difference image, the image processing unit 3 performs edge extraction (imaging object contour extraction), centroid extraction, area calculation, distance to the imaging object, motion vector calculation, and the like in the same manner as the first embodiment. Then, various image processing is performed to extract information such as the shape, movement, and position of the image pickup body (step S15). That is, in the case of a mole hitting game, for example, the outline of the arm and its center-of-gravity position (three-dimensional coordinates) may be extracted from the difference image in which only both arms are extracted as shown in FIGS.

  The barycentric position extracted from each distance image is sent to the information management unit 5. The information management unit 5 compares the position of the center of gravity extracted from each image area with the appearance position of the mole displayed on the display screen of the presentation unit 4, and if the mole is within a predetermined range, It can be determined that When it is determined that the mole has been hit, the hit mole displayed on the presentation unit 4 is screamed, or a score corresponding to the number of hit moles displayed on the presentation unit 4 is added. Etc. are performed (step S16).

  When the information extraction apparatus according to the second embodiment is used for an arcade version of a mole hitting game, for example, the user taps the mole with a natural gesture of hitting the mole with the elbow portion as a fulcrum. Can be easily reproduced.

  As described above, according to the second embodiment, since only the relative movement of the arm with respect to the torso is extracted from the distance image, for example, the acquisition is performed without being affected by the extra movement other than the movement to be recognized. The motion can be recognized from the obtained distance image with high accuracy.

(Third embodiment)
FIG. 18 shows a configuration example of an information extraction apparatus according to the third embodiment of the present invention. In addition, the same code | symbol is attached | subjected to FIG. 1 and an identical part, and only a different part is demonstrated. That is, in FIG. 18, the area division processing unit 2 and the base extraction unit 6 are provided.

  The processing operation of the information extraction apparatus in FIG. 18 will be briefly described. That is, as described with reference to FIG. 7, the distance image captured at predetermined intervals (for example, two images per second) by the distance image acquisition unit 1 is received from the information management unit 5 by the region division processing unit 2. Is divided based on the area information. The base extraction unit 6 extracts a base image from each divided image to be processed at predetermined intervals (for example, one per second) and obtains a difference image between the split image and the base image. Start the image processing process for each of the difference images obtained from each divided image, edge extraction (imaging object contour extraction), centroid extraction, area calculation, distance calculation to the imaging object, motion as described above Various image processing for extracting information such as the shape, movement, and position of the image pickup body such as vector calculation is performed.

  1, 9, and 18, the processing operations of the components other than the distance image acquisition unit 1 are a program that can be executed by a computer, such as a floppy (registered trademark) disk, a hard disk, It can also be stored and distributed in a recording medium such as a CD-ROM, DVD, or semiconductor memory.

1 is a diagram schematically illustrating a configuration example of an information extraction device according to a first embodiment of the present invention. The figure which showed an example of the external appearance of a distance image acquisition part. The figure which showed the structural example of the distance image acquisition part. The figure which showed an example of the distance image which uses the intensity | strength of reflected light as a pixel value. FIG. 4 is a three-dimensional representation of a matrix-type distance image as shown in FIG. 3. The figure which showed an example of the outline image of the object extracted from the distance image. The flowchart for demonstrating the processing operation of the information extraction apparatus of FIG. FIG. 3 is a specific example of a distance image acquired by a distance image acquisition unit, and is a diagram showing a distance image captured with both hands and a divided image based on the distance image. The figure which showed the structural example of the information extraction apparatus which concerns on the 2nd Embodiment of this invention. The flowchart for demonstrating the processing operation of the information extraction apparatus of FIG. The figure which showed the specific example of the distance image acquired by the distance image acquisition part. The figure which showed the specific example of the distance image acquired by the distance image acquisition part. The figure which showed the specific example of the distance image acquired by the distance image acquisition part. The figure which showed an example of the base image extracted from the distance image of FIG. The difference image of the distance image of FIG. 11 and the base image of FIG. The difference image of the distance image of FIG. 12 and the base image of FIG. The difference image of the distance image of FIG. 13 and the base image of FIG. The figure which showed the structural example of the information extraction apparatus which concerns on the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Distance image acquisition part 2 ... Area division process part 3 ... Image processing part 4 ... Presentation part 5 ... Information management part 6 ... Base extraction part 101 ... Light emission part 102 ... Reflection light extraction part 103 ... Light reception part 104 ... Timing signal generation Part

Claims (5)

By irradiating the object with light from the light emitting means and receiving the reflected light from the object with the light receiving means, each pixel value indicates the intensity of the reflected light received by the light receiving means, and the reflection indicated by each pixel value A first step in which distance image acquisition means for acquiring a distance image representing a distance to an object and a three-dimensional shape of the object by light intensity acquires a plurality of time-series distance images captured of the object. When,
From the acquired distance image, (a) by extracting an image area in which the amount of change in the center of gravity position is a predetermined value or less, or (b) an image area of a recognition target that is a part of the object in the distance image The reference value determined based on the distance value to the recognition target calculated from the pixel value of each pixel is compared with each elementary value of the distance image, and a value larger or smaller than the reference value is calculated from the distance image. A second step of extracting, as a base image, an image region corresponding to a portion of the object other than the recognition target by extracting a pixel value ;
For each distance image acquired in the first step, a third step is performed for obtaining a difference between the base image and obtaining a difference image including the recognition target image area of the object ;
A fourth step of determining a change amount and a change direction of the position of the recognition target between different distance images from three-dimensional coordinates indicating the position of the recognition target in each difference image ;
Information extraction method, including. 2. The information extraction method according to claim 1, wherein the second step extracts the base image based on the distance image acquired by the distance image acquisition means at every predetermined time. By irradiating the object with light from the light emitting means and receiving the reflected light from the object with the light receiving means, each pixel value indicates the intensity of the reflected light received by the light receiving means, and the reflection indicated by each pixel value Distance image acquisition means for acquiring a distance image representing the distance to the object and the three-dimensional shape of the object by the intensity of light;
From the distance image acquired by the distance image acquisition means , (a) by extracting an image region where the amount of change in the center of gravity position is a predetermined value or less, or (b) a part of the object in the distance image The reference value determined based on the distance value to the recognition target calculated from the pixel value of the image area of the recognition target is compared with each elementary value of the distance image, and the reference value is calculated from the distance image. Extracting means for extracting, as a base image, an image region corresponding to a portion other than the recognition target of the target object by extracting a pixel value having a larger or smaller value than
For each distance image acquired by the distance image acquisition means, a difference between the distance image acquisition means and the base image is obtained, and a means for obtaining a difference image including the recognition target image area of the object ;
Means for determining a change amount and a change direction of the position of the recognition object between different distance images from three-dimensional coordinates indicating the position of the recognition object in each difference image;
An information extraction apparatus comprising: 4. The information extracting apparatus according to claim 3 , wherein the extracting unit extracts the base image based on the distance image acquired by the distance image acquiring unit every predetermined time. By irradiating the object with light from the light emitting means and receiving the reflected light from the object with the light receiving means, each pixel value indicates the intensity of the reflected light received by the light receiving means, and the reflection indicated by each pixel value A computer comprising distance image acquisition means for acquiring a distance image representing the distance to the object and the three-dimensional shape of the object by the intensity of light,
From the distance image acquired by the distance image acquisition means , (a) by extracting an image region where the amount of change in the center of gravity position is a predetermined value or less, or (b) a part of the object in the distance image The reference value determined based on the distance value to the recognition target calculated from the pixel value of the image area of the recognition target is compared with each elementary value of the distance image, and the reference value is calculated from the distance image. Extracting means for extracting, as a base image, an image region corresponding to a portion other than the recognition target of the target object by extracting a pixel value having a larger or smaller value than
For each distance image acquired by the distance image acquisition means, a difference between the distance image acquisition means and the base image is obtained, and a means for obtaining a difference image including the recognition target image area of the object ;
Means for obtaining a change amount and a change direction of a position of a part of the recognition object between different distance images from three-dimensional coordinates indicating the position of the recognition object in each difference image;
A computer-readable recording medium in which a program for functioning as a computer is recorded.

Images