JP2013020353A - Discriminator generation method, device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase discrimination accuracy of an open/close state of eyes when detecting eyes from an input image and tracking the same while reducing tracking failure.SOLUTION: An open/close state of eyes is classified into three or more stages. Plural sample images which are known to be images representing eyes at each of the stages respectively and plural sample images which are known to be images not representing eyes are prepared. A discriminator which determines whether a determination object image is an image representing eyes at the stages is generated by causing it to learn the plural sample images which are known to be the images representing eyes at each of the stages as correct samples, so that a sample image which is known to be an image representing eyes in a state one stage above or one stage below the stages is not determined as incorrect, and one of the plural sample images which is known to be an image representing eyes in a state two stages above or two stages below the stages and the plural sample images which are known to be the images not representing eyes as incorrect samples.

Description

  The present invention relates to a discriminator generating method for generating a plurality of discriminators for discriminating each of the predetermined states for whether or not the determination target image is an image including a predetermined object whose state continuously changes. The present invention relates to a device and a program.

  2. Description of the Related Art Conventionally, a dozing detection device that detects and alerts a driver's dozing state by monitoring the open / closed state of the driver's eyes using an image obtained by an in-vehicle camera or the like has been used.

  As a method for monitoring the open / closed state of eyes, for example, a plurality of sample images that are known to be images representing eyes in an open state are used as correct samples, and a plurality of samples that are known not to represent images of eyes A discriminator that discriminates whether or not a discrimination target image is an image representing an eye in an open state by learning an image as an incorrect answer sample, and is an image representing an eye in a closed state An image representing an eye in which the image to be discriminated is closed by learning a plurality of sample images that are known not to represent an eye as a correct sample and a plurality of sample images that are known not to represent an eye as an incorrect sample. Are generated, and the eyes are sequentially detected from a plurality of input images obtained by continuous shooting using these discriminators. Technique that has been known.

  By the way, in the above method, when generating the discriminator for discriminating each state, the eye feature in the state is learned as a correct answer, while the eye feature in another state is correct or incorrect. Since the image representing the eye in the closed state is discriminated as the image representing the eye in the open state, the image representing the eye in the open state is closed. It may be discriminated as an image representing the eye, which causes a problem.

  On the other hand, in Patent Document 1, when generating a discriminator for discriminating whether or not the discrimination target image is an image representing an eye in an open state in the above method, the image is not an image representing an eye. In addition to the known sample image, a sample image that is known to be an image representing an eye in a closed state is also learned as an incorrect sample, and is an image that represents an eye in which a discrimination target image is in a closed state In addition to the sample image that is known not to represent the eye, the sample image that is known to represent the eye in the open state is also generated when generating the discriminator that determines whether or not A technique for solving the above problem by learning as a correct sample has been proposed.

JP 2007-323104 A

  However, in the method of the above-mentioned Patent Document 1, since the eye features in different states are learned as incorrect solutions when generating the discriminator for discriminating each state, a half open that is an intermediate between the two states is performed. It may be determined that the image representing the eye in the state is not an image representing the eye, and as a result, the eye may be lost and tracking of the eye may fail.

  In view of the above circumstances, the present invention is a discriminator generation method capable of improving the state discrimination accuracy while reducing tracking failure when detecting and tracking a predetermined object whose state continuously changes. An object of the present invention is to provide an apparatus and a program.

  The discriminator generation method of the present invention includes a plurality of discriminators for discriminating whether or not each discrimination target image is an image including a predetermined object whose state changes repetitively and continuously for each predetermined state. A plurality of sample images that are known to be images representing a predetermined object in each state, and the predetermined target A plurality of sample images that are known not to represent an object are acquired, and a plurality of sample images that are known to be images representing a predetermined object in the state are set as correct samples for each state. A sample image that is known to be an image representing a predetermined object in a state at a stage next to the state is not an incorrect sample, and is a predetermined object that is two or more steps away from the state. Paintings At least one of the plurality of sample images that are known to be and the plurality of sample images that are known not to represent the predetermined object are learned as incorrect samples, and the discrimination target image is in this state A discriminator that discriminates whether or not the image represents a certain predetermined object is generated.

  In the classifier generation method, the predetermined object may be an eye, and the state may be an eye open / closed state.

  Further, the three or more stages may be three stages of a closed eye state, a half open state, and a fully open state.

  Further, the predetermined object is an eye, and the three or more stages are in three stages: a state where the pupil is located at the center of the eye, a state where the pupil is located at the left end of the eye, and a state where the pupil is located at the right end of the eye. There may be.

  Further, the predetermined object is a face, and the three or more stages may be three stages of a face facing front, a face facing diagonally, and a face facing sideways. .

  The discriminator generation device of the present invention is characterized by comprising sample image acquisition means for implementing the discriminator generation method and learning means.

  The discriminator generation program of the present invention is a program that is executed by at least one computer. This program is recorded on a recording medium such as a CD-ROM or DVD, or recorded in a state where it can be downloaded to a storage attached to a server computer or a network storage, and provided to the user.

  According to the discriminator generation method, device, and program of the present invention, whether or not the discrimination target image is an image including a predetermined object whose state changes repetitively and continuously is determined for each predetermined state. When generating a plurality of discriminators to discriminate, the state is divided into three or more stages, a plurality of sample images that are known to be images representing a predetermined object in each state, A plurality of sample images that are known not to represent an object are acquired, and a plurality of sample images that are known to be images representing a predetermined object in the state are obtained as correct samples for each state. A sample image that is known to be an image representing a predetermined object in a state at a stage adjacent to the state is not an incorrect sample, and a predetermined image in which the state and the stage are separated by two or more Target The discriminating target image is learned as an incorrect sample by learning at least one of a plurality of sample images known to be images representing the image and a plurality of sample images known to be images not representing the predetermined object. Since the discriminator for determining whether or not the image represents the predetermined object in the state is generated, any of the images representing the predetermined object in the state between successive stages is selected. It is possible to discriminate the object as a predetermined object in the state, and to reduce tracking failure due to non-detection. Further, it is possible to prevent a determination error between images representing a predetermined object in a state where two or more stages are separated from each other, and the state determination accuracy can be improved.

  In the discriminator generation method, apparatus, and program, when the predetermined object is an eye and the state is an open / closed state of the eye, an eye tracking failure occurs when the eye is detected and tracked from the input image. The accuracy of discriminating the open / closed state of the eye can be improved.

  Further, when the three or more stages are the three stages of the eyes closed state, the half-open state, and the fully-open state, the image representing the eye in which the discrimination target image is closed or the fully-open state is displayed. It is possible to reduce eye tracking failure while improving the accuracy of determining whether the image represents an eye in the above state.

  Further, the predetermined object is an eye, and the three or more stages are in three stages: a state where the pupil is located at the center of the eye, a state where the pupil is located at the left end of the eye, and a state where the pupil is located at the right end of the eye. In some cases, it is possible to reduce eye tracking failure while improving the discrimination accuracy of whether the discrimination target image is an image representing an eye looking at the left side or an eye looking at the right side. it can.

  In the case where the predetermined object is a face, and the three or more stages are three stages of a state where the face is facing the front, a state where the face is facing obliquely, and a state where the face is facing sideways. Further, it is possible to reduce face tracking failure while improving the discrimination accuracy of whether the discrimination target image is an image representing a face facing front or an image representing a face facing sideways.

Block diagram showing the configuration of the eye detection device Block diagram showing the configuration of the classifier generator The figure which shows typically the sample image used for the production | generation of the discriminator which discriminate | determines each state A flowchart showing a method of generating a discriminator

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. An eye detection apparatus 10 shown in FIG. 1 detects and tracks eyes from a moving image of a plurality of frames using a classifier group generated by a classifier generation apparatus 20 described later. An eye detection unit 12, an eye tracking unit 13, a storage unit 14, and the like are provided. The function of each unit in the eye detection apparatus 10 can be realized by the computer system operating according to a predetermined program.

  The frame memory 11 stores a frame image for at least several frames among the moving images composed of a plurality of frames input to the eye detection device 10.

  The eye detection unit 12 sequentially acquires frame images from the frame memory 11 and detects an eye to be detected from each frame image using a classifier group. Specifically, with the entire image as a search range, search frames having a predetermined size within the search range are sequentially set while shifting their positions, and partial images surrounded by the set search frames are cut out. Then, the cut-out partial image is input to the classifier group, and eyes are detected by determining whether or not the partial image is an image representing an eye in a predetermined state.

  Here, the classifier group is composed of a plurality of classifiers that determine whether or not the partial image (image to be determined) is an image representing an eye in a predetermined state. A first discriminator that is an image representing an eye in which the power image is closed, a second discriminator that is an image representing an eye in which the image to be discriminated is half open, and the image to be discriminated is fully open A third discriminator that is an image representing an eye face in a state is connected in parallel.

  Each discriminator calculates a feature amount related to a difference in pixel values between a plurality of predetermined points as at least one feature amount related to the distribution of pixel values of the partial image, and the partial image is predetermined using the feature amount. It is determined whether or not the image represents an eye in the state, and the determination result obtained from all or part of the plurality of classifiers is evaluated, and the determination target image is in any of the states A determination result is obtained as to whether or not the image represents an eye in the area. Note that details regarding the configuration, generation method, and the like of each classifier constituting the classifier group will be described later.

  When an eye is detected in the current frame, the eye detection unit 12 displays information such as the position of the eye in the detected image and information indicating which of the three states the detected eye is in. Store in the storage unit 14. Further, the eye detection unit 12 notifies the eye tracking unit 13 that the eye has been detected, and interrupts the eye detection process until there is a notification from the eye tracking unit 13 that the eye has been lost. When the eye tracking unit 13 notifies that the eye has been lost, the eye detection process is started again.

  When an eye is detected by the eye detection unit 12, the eye tracking unit 13 sequentially acquires frame images from the frame memory 11, and uses the discriminator group to detect eyes from the frame images, thereby detecting the eye. 12, specifically, an area having a predetermined size based on the position of the eye detected in the immediately preceding frame is set as a search range, and a predetermined range is set within the search range. A search frame having a size is sequentially set while shifting its position, a partial image surrounded by each set search frame is cut out, the cut out partial image is input to a classifier group, and the partial image is in a predetermined state. The process of detecting the eye by determining whether or not the image represents the eye in the eye is repeated until the eye is lost.

  When the eye tracking unit 13 detects an eye in the current frame, the eye tracking unit 13 displays information such as the position of the eye in the detected image, the information indicating which of the three states the detected eye is in. When the eye is not detected, the eye detection unit 12 is notified that the eye has been lost, and the eye detection unit 12 notifies the eye that the eye has been detected. Interrupt the tracking process. When the eye detection unit 12 notifies that the eye has been detected again, the eye tracking process is resumed.

  Next, the discriminator generation device 20 will be described. As shown in FIG. 2, the discriminator generation device 20 includes a sample image acquisition unit 21 and a learning unit 22. The function of each unit in the discriminator generation device 20 can be realized by the computer system operating according to a predetermined program.

  The sample image acquisition unit 21 is for inputting a sample image used for learning of the discriminator to the discriminator generation device 20. The discriminator generation device 20 determines whether the image to be discriminated is an image representing an eye in a closed state, an image representing an eye in a half open state, or an eye face in a fully open state. The sample image acquiring unit 21 generates a plurality of samples that are known to be images representing eyes in each state. An image and a plurality of sample images that are known not to represent an eye are acquired and provided to the learning unit 22 as learning data.

  The learning unit 22 performs learning using the learning data provided from the sample image acquisition unit 21 by a predetermined learning algorithm, and sets the first, second, and third discriminators constituting the classifier group, respectively. Generate.

  Each discriminator is composed of a plurality of weak discriminators, in which weak discriminators effective for discrimination selected from a large number of weak discriminators by learning described later are connected in series in the effective order. Each weak discriminator calculates an amount of feature from a discrimination target image according to a predetermined algorithm specific to each weak discriminator, and is an image representing an eye in which the discrimination target image is in a predetermined state based on the feature amount. The score which shows the probability of a certain thing is calculated | required. Each discriminator evaluates the score obtained from all or part of the plurality of weak discriminators to obtain a discrimination result as to whether or not the discrimination target image is an image representing an eye in a predetermined state.

  Since these discriminators have different eye states that can be discriminated, different sample images for each state are used as learning data. FIG. 3 is a diagram schematically showing a sample image used for generating a discriminator for discriminating each state. As shown in FIG. 3, when a discriminator that is an image representing an eye in a closed state is generated by learning, the image to be discriminated is known to be an image representing an eye in a closed state. Multiple sample images are used as correct samples, and multiple sample images that are known to be images representing eyes that are fully open and multiple sample images that are known not to represent eyes are incorrect samples Used as. However, an image representing an eye that is half open is not used as either a correct sample or an incorrect sample.

  In addition, when generating a discriminator that is an image representing an eye in a state where the image to be discriminated is half open, a plurality of sample images that are known to be images representing the eye in a half open state are generated. A plurality of sample images that are used as correct samples and are known not to represent images are used as incorrect samples. However, an image representing an eye in a closed state or an image representing an eye in a fully opened state is not used as either a correct sample or an incorrect sample.

  In addition, when the discriminator, which is an image representing an eye in a fully open state, is generated by learning, a plurality of sample images that are known to be images representing an eye in a fully open state are generated. A plurality of sample images that are used as correct samples and are known to be images representing eyes in a closed state and a plurality of sample images that are known not to represent images are used as incorrect samples. However, an image representing an eye that is half open is not used as either a correct sample or an incorrect sample.

  Next, the generation (learning) process of each discriminator performed by the learning unit 22 will be described. Note that the processing for generating each discriminator is the same except that the different sample images are used as learning data for each state to be discriminated as described above. The processing flow will be described on the premise that a sample image suitable for the generation is prepared.

  FIG. 4 is a flowchart showing a method for generating a discriminator. First, initial values of weights of all sample images (correct samples and incorrect samples) appropriately prepared corresponding to the classifier to be generated are set equal to 1 (step S11).

  Next, a weak semi-separator is created for each of the plural types of pair groups when a plurality of types of pair groups consisting of a plurality of pairs are set with two predetermined points set on the sample image as one pair. (Step S12). Here, each weak discriminator is a predetermined discrimination target image using a combination of pixel value difference values between two points in each pair constituting a specific pair group assigned to the weak discriminator. A value indicating the probability of being an image representing an eye in the state is output.

  Subsequently, the weak classifier that is most effective for determining whether or not the determination target image is an image representing an eye in a predetermined state is selected from the plurality of weak half-separators created in step S12. . The most effective weak classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rates of the weak classifiers are compared, and the weak classifier showing the highest weighted correct answer rate is selected (step S13). That is, in the first step S13, since the weight of each sample image is equal to 1, the weak discriminator simply determines correctly whether or not the discrimination target image is an image representing an eye in a predetermined state. The sample with the largest number of sample images is selected as the most effective weak classifier.

  Next, the correct answer rate of the combination of weak classifiers selected so far, that is, whether or not each sample image is an image representing an eye in a predetermined state using a combination of the weak classifiers selected so far It is ascertained whether or not the rate at which the result of the determination matches the answer indicating whether or not the image actually represents an eye in a predetermined state has exceeded a predetermined threshold (step S14). When the predetermined threshold is exceeded, it is possible to determine with a sufficiently high probability whether or not the determination target image is an image representing an eye in a predetermined state by using the weak classifier selected so far. finish. If it is equal to or less than the predetermined threshold value, the process proceeds to step S16 in order to select an additional weak classifier to be used in combination with the weak classifier selected so far. In step S16, the weak discriminator selected in the most recent step S13 is excluded so as not to be selected again.

  Next, the weak discriminator selected in the most recent step S13 cannot increase whether the discrimination target image is an image representing an eye in a predetermined state or not, and the weight of the sample image is increased. The weight of the sample image that can correctly determine whether or not is an image representing an eye in a predetermined state is reduced (step S15). Subsequently, the process returns to step S13, and the next effective weak classifier is selected based on the weighted correct answer rate as described above.

  As a weak discriminator suitable for discriminating whether or not the image represents an eye in a predetermined state by repeating the above steps S13 to S16, two predetermined points of each pair constituting a specific pair group If a weak discriminator corresponding to a combination of difference values of pixel values between them is selected, if the correct answer rate confirmed in step S14 exceeds a threshold, whether or not the image represents an eye in a predetermined state. The type of weak discriminator used for discrimination and the discrimination condition are determined (step S17), thereby completing the learning. The selected weak classifiers are linearly combined in descending order of the weighted correct answer rate to generate a classifier.

  With the above configuration, according to the present embodiment, when generating a discriminator that discriminates whether or not the discrimination target image is an image representing an eye in a fully open state, the eye in a closed state is represented. When a classifier that learns a sample image that is known to be an image as an incorrect sample and generates a discriminator that determines whether or not the discrimination target image is an image that represents a closed eye is generated Since a sample image that is known to be an image representing an eye in the image is learned as an incorrect sample, the image to be identified represents an eye that is in a closed state or an eye that is in a fully open state The accuracy of discriminating whether the image is an image can be improved.

  Further, when generating a discriminator for discriminating whether or not the discrimination target image is an image representing an eye in a fully open state, and a discriminator for discriminating whether or not the discriminating image is an image representing an eye in a closed state When generating an image, a sample image that is known to be an image representing a half-opened eye is not used as an incorrect sample, and it is determined whether the image represents a half-opened eye. When generating the discriminator, neither the image representing the eyes in the fully open state nor the image representing the eyes in the closed state is used as an incorrect sample, so the discrimination target image is in the fully open state. If the image represents an eye that is in the half-open state, the image is identified as either an image that represents the eye that is fully open or an image that represents the eye that is half-open. , The discrimination target image is closed and half If the image represents an eye in an open state, the image is identified as either an image representing an eye in a closed state or an image representing an eye in a half-open state. Can be maintained and eye tracking failure can be reduced.

  In addition, since the human eye repeatedly blinks (opening and closing movement of the eyelid), paying attention to this point, while tracking the eye as described above, further preparing a change in the opening / closing state of the eye being tracked in advance. By checking with the eye open / closed state change model, that is, by monitoring whether or not blinking opening and closing movements are made, it is determined whether or not the object being tracked is a modest eye, By stopping tracking when it is determined that it is not a modest eye, it is possible to reduce eye mistracking and to further improve eye detection and tracking accuracy.

  In the above embodiment, the case has been described where the discriminator generation method, apparatus and program of the present invention are applied when the predetermined object is the eye and the state to be discriminated is the open / closed state of the eye. In the present invention, for example, when the predetermined object is a face and the state to be determined is the face orientation, or when the predetermined object is an eye and the state to be determined is the line-of-sight direction, etc. The present invention can be widely applied to the case where it is determined in a plurality of stages whether the target image is an image representing a predetermined target or not.

  In the above embodiment, the case where the state of the predetermined object is divided into three stages is illustrated. However, the state of the predetermined object is divided into four or more stages, and a discriminator is provided for each state. You may make it produce | generate.

  In the above embodiment, the case where the eye tracking unit 13 performs the eye tracking process using the classifier group has been described. However, the eye tracking unit 13 performs the eye tracking process using template matching. Also good. For example, when an eye is detected by the eye detection unit 12, the partial image in which the eye is detected is stored as a template image in the storage unit 14, and each partial image sequentially cut out in the subsequent frame images is stored. By comparing with the template image stored in the unit 14, it is determined whether or not the similarity is equal to or higher than a predetermined threshold, that is, whether the image represents the eye detected by the eye detection unit 12. The eyes may be tracked.

DESCRIPTION OF SYMBOLS 10 Eye detection apparatus 11 Frame memory 12 Eye detection part 13 Eye tracking part 14 Storage part 20 Discriminator production | generation apparatus 21 Sample image acquisition part 22 Learning part

Claims (11)

This is a discriminator generation method for generating a plurality of discriminators for discriminating, for each predetermined state, whether or not the discrimination target image is an image including a predetermined object whose state changes repetitively and continuously. And
The state is divided into three or more stages, and a plurality of sample images that are known to be images representing the predetermined object in the respective states, and an image that is not an image representing the predetermined object Get multiple sample images,
For each of the states, a plurality of sample images that are known to be images representing the predetermined object in the state are used as correct samples to represent the predetermined object in a state adjacent to the state. A sample image that is known to be an image is not an incorrect sample, and is a plurality of sample images that are known to be images representing the predetermined object that is two or more steps away from the state. An image representing the predetermined object in which the determination target image is in the state by learning at least one of the image and a plurality of sample images that are known not to represent the predetermined object as incorrect samples. A discriminator generation method for generating a discriminator for determining whether or not there is a discriminator.   The classifier generating method according to claim 1, wherein the predetermined object is an eye, and the state is an eye open / closed state.   3. The discriminator generation method according to claim 2, wherein the three or more stages are three stages of a closed eye state, a half-open state, and a fully-open state. The predetermined object is an eye;
2. The determination according to claim 1, wherein the three or more stages are three stages of a state in which the pupil is located at the center of the eye, a state at the left end of the eye, and a state at the right end of the eye. Generator generation method. The predetermined object is a face;
2. The discriminator generation method according to claim 1, wherein the three or more stages are three stages: a state where the face is facing front, a state where the face is facing obliquely, and a state where the face is facing sideways. A discriminator generation device that generates a plurality of discriminators for discriminating each of predetermined states according to whether or not the determination target image is an image including a predetermined object whose state changes repetitively and continuously. And
The state is divided into three or more stages, and a plurality of sample images that are known to be images representing the predetermined object in the respective states, and an image that is not an image representing the predetermined object Sample image acquisition means for acquiring a plurality of sample images;
For each of the states, a plurality of sample images that are known to be images representing the predetermined object in the state are used as correct samples to represent the predetermined object in a state adjacent to the state. A sample image that is known to be an image is not an incorrect sample, and is a plurality of sample images that are known to be images representing the predetermined object that is two or more steps away from the state. An image representing the predetermined object in which the determination target image is in the state by learning at least one of the image and a plurality of sample images that are known not to represent the predetermined object as incorrect samples. A discriminator generating apparatus comprising learning means for generating a discriminator for determining whether or not there is a discriminator.   The discriminator generation device according to claim 6, wherein the predetermined object is an eye, and the state is an eye open / closed state.   8. The discriminator generation device according to claim 7, wherein the three or more stages are three stages of a closed eye state, a half-open state, and a fully open state. The predetermined object is an eye;
7. The determination according to claim 6, wherein the three or more stages are three stages of a state in which the pupil is positioned at the center of the eye, a state at the left end of the eye, and a state at the right end of the eye. Generator. The predetermined object is a face;
7. The discriminator generating device according to claim 6, wherein the three or more stages are three stages of a state where the face is facing front, a state where the face is facing obliquely, and a state where the face is facing sideways. A discriminator generator that generates a plurality of discriminators for discriminating each computer according to a predetermined state, whether or not the discrimination target image is an image including a predetermined object whose state changes repetitively and continuously. A program for functioning as a device,
The computer is divided into three or more stages, and a plurality of sample images that are known to be images representing the predetermined object in each state, and are not images representing the predetermined object Sample image acquisition means for acquiring a plurality of sample images that are known;
For each of the states, a plurality of sample images that are known to be images representing the predetermined object in the state are used as correct samples to represent the predetermined object in a state adjacent to the state. A sample image that is known to be an image is not an incorrect sample, and is a plurality of sample images that are known to be images representing the predetermined object that is two or more steps away from the state. An image representing the predetermined object in which the determination target image is in the state by learning at least one of the image and a plurality of sample images that are known not to represent the predetermined object as incorrect samples. A discriminator generation program that functions as a learning means for generating a discriminator for determining whether or not there is a discriminator.

Images