JP2009069975A - Behavior estimating device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a behavior estimating device for detecting that a driver is taking a specific action. <P>SOLUTION: Behavior estimating processing to be performed by an action estimating device includes extracting featured points from a photographic image (S120), and comparing the extracted featured point with an action estimation model, and performing model collation processing for temporarily deciding that there is a possibility that the driver is taking specific action (for example, shoulder-turning action), when the featured point is positioned within a permitted range (130). Furthermore, when the featured point is positioned continuously within an allowable range by a specific threshold or higher as the result of temporary decision decided by the model collation processing, action certification processing for deciding that it seems certain that the driver is taking the specific action is performed (S140); and when it is confirmed that it seems certain that the driver is taking the specific action (S150: YES), fatigue reduction processing is executed (S160). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a behavior estimation device that estimates the behavior of a person reflected in an image.

  Conventionally, on the basis of a photographing device mounted on a vehicle and photographing an image including the driver, and an image photographed by the photographing device (hereinafter referred to as a photographed image), a dangerous action ( Hereinafter, an action estimation device including an image processing device that determines whether or not a behavior using a mobile phone (hereinafter referred to as “dangerous behavior”) is taken (hereinafter referred to as “phone usage behavior”) is known (for example, , See Patent Document 1).

In the behavior estimation device described in Patent Document 1, driving is performed when a hand in a captured image continues to be positioned within a predetermined region defined in advance around a face in the captured image for a preset time. It is determined that the person is taking a phone call.
JP 2005-205943 A

By the way, as the dangerous behavior, in addition to the telephone use behavior, there are a shoulder turning behavior in which the hand is released from the steering wheel and a shoulder is turned, a arm turning behavior in which the arm is turned (hereinafter referred to as a specific behavior).
However, the behavior estimation device described in Patent Document 1 cannot determine whether or not the driver is performing a specific behavior. For example, an alarm is issued when the driver takes a specific behavior. The process for driving safely could not be executed.

  That is, in the behavior estimation device described in Patent Document 1, feature points (such as wrists and elbows) that are defined in advance on a person's body, such as shoulder-turning behavior and arm-turning behavior, are different from the feature points. There is a problem that it is impossible to detect that the driver is taking a specific action that repeatedly moves around a feature point (shoulder or the like).

  Then, an object of this invention is to provide the action estimation apparatus which can detect taking specific action.

  In the behavior estimation apparatus of the present invention made to achieve the above object, the captured image acquisition unit acquires a captured image obtained by capturing a person every time the captured image is captured, and the feature point detection unit includes: Each time a photographed image is acquired by the photographed image acquisition means using a preset point on the person's body, a detected feature point that is a feature point of the person reflected in the photographed image is detected.

  Then, a behavior that repeatedly moves around a feature point that is defined separately from the feature point that is defined in advance is defined as a specific behavior, and a plurality of feature points that continuously change a person who has taken the specific behavior are located. The model storage means is an action consisting of a return point and a permissible area, where the target point is the area where the feature point of a person who has taken a specific action is likely to be located. The estimation model is stored, and further, the behavior estimation unit is configured to preliminarily define a detection feature point detected by the feature point detection unit within an allowable area of the behavior estimation model stored in the model storage unit. If the authorized state is located more than a predetermined number of times in time, it is estimated that the person reflected in the captured image is taking a specific action.

According to such an action estimation apparatus of the present invention, it is possible to detect that a person reflected in a captured image is taking a specific action.
Here, the specific action refers to a shoulder turning action for turning a shoulder, an arm turning action for turning an arm, a waist turning action for twisting the waist to the left and right, and the like.

  Further, the behavior estimation means in the behavior estimation device of the present invention is configured to be in the authorized state when the detected feature point is located in the allowable region continuously for a prescribed number of times as described in claim 2. It is desirable that

According to the behavior estimation apparatus of the present invention configured as described above, it is possible to improve the detection accuracy that a person reflected in a captured image is taking a specific behavior.
Furthermore, the behavior estimation device according to the present invention, as claimed in claim 3, when it is estimated that the behavior estimation means is taking a specific behavior, the fatigue reduction means It is desirable to be configured to execute fatigue reduction control for reducing fatigue.

  According to the behavior estimation apparatus configured as described above, the fatigue level of the person taking the specific action is assumed to be higher than normal, and the fatigue level of the person is reduced because the fatigue reduction control is executed. Can be made.

In particular, when the behavior estimation device of the present invention configured as described above is mounted on a vehicle and the driver is included in a captured image, the driver can be driven more safely.
Fatigue reduction control executed by the fatigue reduction means in the behavior estimation device of the present invention detects the current position as described in claim 4, obtains information on facilities located around the detected current position, It is also possible to notify information on a resting facility where a person reflected in a captured image can take a break from a navigation device capable of notifying the acquired information (that is, well-known). It is also possible to blow cold air on a person reflected in a photographed image from an air conditioner having at least a cooling function.

According to the behavior estimation apparatus that performs fatigue reduction control as in the former, it is possible to cause the person reflected in the photographed image to take a rest, thereby allowing the person to take a safer action.
In addition, the rest facility here refers to a facility where a person reflected in a photographed image such as a shopping center or a park can take a rest in addition to facilities such as a service area, a parking area, and a road station. It shall be shown.

  In addition, according to the behavior estimation device that performs the fatigue reduction control like the latter, it is possible to reduce the degree of fatigue of the person by blowing cold air on the person reflected in the photographed image, and to perform safer behavior. Can be taken.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of an action estimation apparatus to which the present invention is applied and mounted on a vehicle. Hereinafter, a vehicle on which the behavior estimation device 1 is mounted is referred to as a mounted vehicle.
<overall structure>
The behavior estimation device 1 includes an imaging device 15 that captures an occupant of a mounted vehicle, a known navigation device, a navigation device 21 that issues a warning, and an air conditioner (a so-called air conditioner) mounted on the vehicle. The air conditioning control device 22 for controlling the vehicle) and an image captured by the imaging device 15 (hereinafter referred to as a captured image) are image-processed to estimate the behavior of the occupant of the mounted vehicle, and according to the estimation result, the navigation device 21 and an image processing processor 10 that controls the air conditioning controller 22.

  The photographing device 15 is a well-known digital camera disposed near the base of a rearview mirror that protrudes from the top plate (or windshield) of the mounted vehicle with the lens facing into the vehicle interior. Are sequentially output to the image processor 10. As shown in FIG. 2, the image taken by the photographing device 15 is the upper body of the driver seated in the driver's seat (that is, the driver's head, both arms, both hands, the torso, etc.). It is intended to be at least contained.

  The navigation device 21 includes a current position detector that detects the current position of the mounted vehicle, a storage unit that stores map data, a speaker that outputs sound, a monitor that displays an image, a current position detector, a storage unit, And at least a control device for controlling the speaker and the monitor. The control device is configured around a known microcomputer including a ROM, a RAM, and a CPU, and is configured to execute known current position display processing, route setting processing, route guidance processing, and the like.

The air conditioning control device 22 is configured to execute cold air supply control for blowing cold air from the air conditioning device to the driver in accordance with a control command from the image processing processor 10.
<Image processor>
Next, the image processing processor will be described.

  The image processor 10 is a known microcomputer having at least a ROM 10a for storing processing programs, a RAM 10b for temporarily storing data, and a CPU 10c for executing various processes in accordance with the processing programs stored in the ROM 10a or RAM 10b. It is structured around.

  Then, the ROM 10a sequentially processes the captured images taken by the photographing device 15 to determine whether or not the driver has taken a specific action that has been prescribed in advance. When an action is taken, a processing program for executing an action estimation process for outputting a control command to the air conditioning control device 22 is stored.

Further, the ROM 10a stores a feature point list to be referred to when executing the behavior estimation process and a behavior estimation model group.
Among them, the feature point list is for detecting feature points set in advance on the person's arm from the driver reflected in the photographed image. The feature point list is shown in FIG. The right shoulder (d1 in the figure), left shoulder (d4), right elbow (d2), left elbow (d5), right wrist (d3), and left wrist (d6) of the human body are set as feature points.
<About the behavior estimation model>
In addition, the behavior estimation model group determines whether or not the person reflected in the photographed image has taken a specific action prescribed in advance in accordance with the movement of the feature point detected from the photographed image (hereinafter referred to as a detected feature point). In order to determine whether or not, a plurality of behavior estimation models set in advance for each specific behavior to be detected are included.

  In the present embodiment, specifically, as the specific action to be detected, the action of turning the right shoulder and the action of turning the left shoulder are defined (hereinafter, the action of turning the shoulder is referred to as the shoulder turning action). Called).

  Each behavior estimation model is provided with a return point, which is a plurality of points where continuously changing feature points of the person who has taken the shoulder-turning action are located, and for each return point around the return point. An allowable area, which is an area in which a feature point of a person who took an action may be located, is set. However, the coordinates of the feature points, the return points, and the permissible areas in the present embodiment are represented by a relative coordinate system with the right shoulder or left shoulder of the human body as the origin.

And in this embodiment, in order to set the return point of each action estimation model and an allowable area, the following procedure is taken.
Here, FIG. 7 is an explanatory diagram showing a procedure for creating a behavior estimation model, and FIG. 8 is an explanatory diagram for explaining a procedure for setting a return point and an allowable region.

  First, a plurality of subjects (for example, a statistically reliable number is preferable) (A to M in FIG. 7) are caused to take a plurality of (1 to 3 in FIG. 7) shoulder-shake behaviors. The state of taking a shoulder turning action is included in the photographed image. From the captured images, after each subject raises his arm (hereinafter referred to as a starting posture), until the arm returns to the same point (that is, the same posture as the starting posture, hereinafter referred to as a terminal posture). ) (That is, a state in which the shoulder is turned once) is extracted for each subject and for each trial that is a single shoulder turning action.

  Furthermore, the feature points of the subjects reflected in the captured images are extracted from the captured images of the shoulder rotation extracted for each subject (that is, for each trial), and the feature point coordinates at all times are extracted for all subjects. Find all trials. Hereinafter, the feature point coordinates at one time are also referred to as samples.

Then, as shown in FIG. 7, since the speed of turning the shoulder is different for each subject, and the speed of turning the shoulder is different for each trial even if the subject is the same, the start posture to the end posture are accommodated. The number of photographed images is different, and the number of samples to be obtained (N _Xi (where X represents the subject, i represents the number of trials, and in FIG. 7, N _A1 , N _A2 , N _A3 ... N _M1 , N _M2, N _M3 corresponds) will be different. for example, subject a, as compared to the subject M, the number of samples is reduced (which was illustrated in FIG. _{7, N Ai> N Mi, i} = 1~ 3) There is a possibility that the number of samples of subject M's trial I and trial J (where I and J are natural numbers (1 to 3 in the example shown in FIG. 7)) is different (illustrated in FIG. 7). In some cases, N _M1 ≠ N _M2 ≠ N _M3 ).

However, even if the number of samples is small, it is clear that the feature points pass between the obtained feature point coordinates, so the calculated feature points are the same as the trials with the largest number of samples. The feature point coordinates that would have passed between the coordinates are obtained according to a known interpolation process, and the feature point coordinates of the trial with a small number of samples are increased (in the example shown in FIG. 7, the number of samples L after executing the interpolation process is N _A3 (L = N _{A3 in the} figure)).

  In the feature point coordinate set (ie, a collection of feature point coordinates resampled by the interpolation process based on the feature point coordinates detected from the captured image) in which the number of samples is the same by the interpolation process, at approximately the same time. Feature point coordinates that are considered to be located (that is, the same number of samples have advanced from the starting posture) are extracted and grouped over all trials of all subjects (hereinafter, features formed by grouping) A set of point coordinates is a coordinate group). Here, the positions where the feature point coordinates forming each coordinate group are extracted are set as re-sampling time positions.

  Further, an average coordinate obtained by averaging all feature point coordinates extracted at each resample time position (that is, all feature point coordinates constituting the coordinate group at each resample time position) is obtained, and the average coordinate is obtained. Is set as the return point. Then, the distance between the calculated return point and all the feature point coordinates extracted at each resampling time position (that is, all the feature point coordinates constituting the coordinate group at each resampling time position) ( Hereinafter, the detection distance is calculated, and the standard deviation is obtained from all of the calculated detection distances. Thereafter, an area (a circle for a two-dimensional plane, a sphere for a three-dimensional space) having a radius (hereinafter referred to as an allowable distance) three times the calculated standard deviation is set as an allowable area.

That is, as shown in FIG. 8, the allowable area is set to a size that includes a plurality of feature point coordinates around the return point, and is represented by the return point and the allowable distance.
As described above, one behavior estimation model includes a return point for sample coordinates and an allowable area centered on the return points (hereinafter, each return point and allowable area set for each sample coordinate). A curve that is closed by the trajectory of the return point (or allowable region) is represented.

That is, the image processing processor 10 is configured to be able to execute a behavior estimation process by comparing feature points detected from a captured image with each behavior estimation model.
<About behavior estimation processing>
Next, a behavior estimation process executed by the image processor 10 will be described.

Here, FIG. 4 is a flowchart showing the processing procedure of the behavior estimation processing.
This behavior estimation process is executed when the image processor 10 is activated (in this embodiment, the image processor 10 is activated when the ignition key is turned ON). When the behavior estimation process is executed, first, in S110, a photographed image photographed by the photographing device 15 is acquired. In S120, the feature point list is collated with the captured image acquired in S110 to extract all the feature points of the driver reflected in the captured image.

Note that the process of extracting the feature points is a well-known technique as described in, for example, Japanese Patent Application Laid-Open No. 2003-109015, and thus detailed description thereof is omitted here.
Furthermore, in S130, the feature matching extracted in S120 and the behavior estimation model stored in the ROM 10b are compared, and a model matching process is performed to determine whether or not the driver is taking a shouldering action.

  In S140, an action recognition process for confirming whether or not the determination result in the model matching process in S130 is certain is executed. That is, in this action recognition process, it is determined in the model matching process, and it is determined whether or not it is certain that the driver is taking a shoulder action.

  Then, in S150, if it is confirmed that it is certain that the shoulder turning action is taken as a result of the determination in the action recognition process in S140, the process proceeds to S160. Specifically, in S150 of the present embodiment, it is assumed that it is certain that the shoulder-turning action is taken when the later-described confirmed action flag is at a high level.

  In S160, assuming that the driver feels fatigue, a fatigue reduction process for reducing the driver's fatigue is performed, and then the process returns to S110. Specifically, in the fatigue reduction process of the present embodiment, the control command is output to the air conditioning control device 22, and the air conditioning control device 22 that has received the control command controls the air conditioner to blow cool air to the driver. To do.

On the other hand, when it is confirmed in the previous S150 that it is not certain that the shoulder turning action is taken, the process returns to S110.
<About model matching processing>
Next, the model matching process executed in S130 of the action estimation process will be described.

Here, FIG. 5 is a flowchart showing the processing procedure of the model matching processing.
As shown in FIG. 5, when the model matching process is started in S130 of the action estimation process, first, in S210, one action estimation model (ie, right shoulder model) is selected from the action estimation model group stored in the ROM 10a. Or the action that turns the left shoulder or the action that turns the left shoulder is prescribed).

  In S220, the feature points extracted in S120 are converted into coordinates on the relative coordinate system with the right shoulder (or left shoulder) of the human body as the origin (hereinafter, the converted feature points are referred to as relative feature points). To do). However, the origin coordinate (that is, right shoulder or left shoulder) is an action that turns the right shoulder or the left shoulder if the action turns the right shoulder according to the action specified in the action estimation model read in S210. If left shoulder is decided.

Specifically, in the present embodiment, the relative feature points are obtained by subtracting the shoulder coordinates from the wrist or elbow coordinates.
Further, in S230, distances between the relative feature points obtained in S220 and all return points included in the behavior estimation model read out in S210 (hereinafter referred to as determination distances) are calculated.

  In subsequent S240, the shortest determination distance with the shortest distance is extracted from all the determination distances calculated in S230, and the allowable distance of the allowable region set around the return point used for calculating the shortest determination distance. And proceeds to S250.

In S250, the allowable distance read in S240 is compared with the shortest determination distance. If the shortest determination distance is equal to or smaller than the allowable distance as a result of the comparison, the process proceeds to S280.
In S280, the specific action flag indicating that there is a possibility that the driver is taking a shoulder turn action is set to the high level. However, the specific behavior flag is prepared in advance for each behavior estimation model, that is, for each of the behavior of turning the right shoulder and the behavior of turning the left shoulder.

That is, in S280, it is possible to identify the possibility that the driver is taking a shoulder swinging action in the processes after S280 (that is, the action recognition process).
Thereafter, the model matching process in the process (hereinafter referred to as the current cycle) for the captured image acquired this time is terminated, and the process proceeds to the action estimation process S140 (that is, the action recognition process).

On the other hand, as a result of the comparison in S250, if the shortest determination distance is longer than the allowable distance, the process proceeds to S260.
In S260, all the action estimation models stored in the ROM 10a are applied, and whether or not the processes from S210 to S230 are executed, that is, the shoulder turning action of turning the right shoulder and the shoulder turning action of turning the left shoulder. It is determined whether or not a behavior estimation model for detecting either one is not applied. If it is determined as a result of the determination that all the behavior estimation models have not been applied and the process has not been executed, the process returns to S210 to read a behavior estimation model that has not yet been applied, and then the processes from S220 to S250. Execute.

  As a result of the determination in S260, when it is determined that all the behavior estimation models have been applied and the processing has been completed, all the specific behavior flags are displayed so as to indicate that the driver is not taking a shoulder-turning behavior. After that, the model matching process in the current cycle is terminated, and the process proceeds to the action estimation process S140 (that is, the action recognition process).

That is, in the model matching process, if the relative feature point is located within the allowable range, it is temporarily determined that the driver may be turning and the specific action flag is set to the high level, If the relative feature point is located outside the allowable range, it is temporarily determined that the driver does not take a shoulder-turning action, and the specific action flag is set to a low level.
<About action certification processing>
Next, the action recognition process executed in S140 of the action estimation process will be described.

Here, FIG. 6 is a flowchart showing the processing procedure of the action recognition processing.
When the action recognition process is executed in S140 of the action estimation process, first, in S410, it is determined whether or not the specific action flag is at a high level. If the specific action flag is high as a result of the determination, the process proceeds to S415. That is, in S260 in the model matching process in the current cycle, if any one of all the specific action flags is set to the high level, the process proceeds to S415.

  In S415, whether or not the specific action flag set to the high level in the current cycle is also set to the high level in the processing (hereinafter referred to as the previous cycle) for the captured image acquired last time, It is determined whether or not the action flag is at a low level. As a result, if it is determined that either one is satisfied, the process proceeds to S420.

In S420, the specific action counter is incremented by 1, and the process proceeds to S430.
In subsequent S430, it is determined whether or not the specific action counter is equal to or greater than a predetermined threshold value. Note that the specified threshold value in the present embodiment is, for example, the number of captured images captured during 0.5 seconds. That is, in S430, it is determined whether or not the shortest determination distance is equal to or less than the allowable distance continuously for several predetermined cycles.

As a result of the determination in S430, if the specific action counter is equal to or greater than the specified threshold, that is, if the shortest determination distance is equal to or less than the allowable distance for several cycles, the process proceeds to S440.
In S440, the fixed action flag is set to the high level on the assumption that the driver is taking a shoulder-turning action for a specified period of time, that is, the driver is surely taking the shoulder-turning action. The confirmed action flag is prepared in advance for each action estimation model, that is, each of the left and right shoulder turning actions, and if it is high level, it indicates that the driver is taking the specific action, A low level indicates that the driver is not taking that specific action.

Thereafter, the action recognition process in the current cycle is terminated, and the process proceeds to S150 of the action estimation process.
On the other hand, if the specific action flag is at the low level as a result of the determination in S410, or is not satisfied as a result of the determination in S415 (that is, different from the specific action flag set to the high level in the current cycle). If only the specific action flag is at the high level), the process proceeds to S450, and in S450, all the specific action counters are set to initial values (that is, 0). Then, the process proceeds to S460.

Further, if the result of determination in S430 is that the specific action counter is less than the specified threshold value, that is, if the driver does not take a turning action for a specified time, the process proceeds to S460.
In S460, the confirmed action flag is set to the low level, assuming that the driver does not take the shoulder turning action.

Thereafter, the action recognition process in the current cycle is terminated, and the process proceeds to S150 of the action estimation process.
In other words, in the action recognition process, if the same specific action flag is at a high level for several consecutive cycles in advance, it is determined that the driver is surely taking a shouldering action and confirmed. The action flag is set to high level.
<Operation example>
Next, an operation example of the behavior estimation apparatus 1 will be described.

  Here, FIG. 9 shows how the driver takes the shoulder turning action while turning the left shoulder, the transition from the start position to the end position, and the feature points when the driver takes the shoulder action. It is explanatory drawing which showed a mode that it changes.

  In the following, as shown in FIG. 9, the driver takes a starting posture ((A) in the figure) to take a shoulder-turning action, and then intermediate postures ((B), (C) in the figure). , (D)) and the behavior estimation process is executed in a state in which captured images continuously containing the state until the end posture ((A) in the figure) can be sequentially acquired are described.

  In the behavior estimation process, when a captured image containing a starting posture as shown in FIG. 9A is acquired, relative feature points obtained from the captured image are within an allowable region in one template among a plurality of templates. (That is, the relative feature point is located inside the allowable region as shown in FIG. 9). For this reason, the behavior estimation apparatus 1 tentatively determines that the driver is taking a shoulder swinging action, and acquires the next captured image.

  Then, when the behavior estimation apparatus 1 acquires a captured image in which intermediate postures are continuously stored as shown in FIGS. 9B, 9C, and 9D, the start posture is stored in the behavior estimation processing. As in the case where the captured image is acquired, it is recognized that the relative feature point is located within the allowable area in one template among a plurality of templates. That is, since the relative feature point is located inside each allowable region as shown in FIGS. 9B, 9 </ b> C, and 9 </ b> D, it is temporarily determined that the driver is taking a shoulder turn. .

  Further, in the behavior estimation process, if it is determined that the shortest determination distance is equal to or less than the allowable distance continuously (that is, the driver is taking a shoulder-turning action) continuously over a specified threshold value, the driver performs a shoulder-turning action. It is recognized that it is certain that it is taken.

As a result, the behavior estimation apparatus 1 determines that the driver is taking a shoulder swinging action, and executes a fatigue reduction process to reduce the driver's fatigue.
[Effect of the embodiment]
As described above, according to the behavior estimation apparatus 1 of the present embodiment, the person (in the captured image) is compared by comparing the feature points extracted from the captured image with a preset behavior estimation model. That is, it is possible to detect that the driver is taking a shoulder turn.

  Then, in the behavior estimation device 1, when it is detected that the driver is taking a shouldering action, the fatigue reduction control is executed on the assumption that the degree of fatigue is higher than normal, and the driver Since cold air is sprayed on the driver, the driver's fatigue can be reduced. As a result, the driver can be driven more safely.

Further, according to the behavior estimation device 1 of the above embodiment, when it is determined that the shortest determination distance is equal to or less than the permissible distance in a predetermined number of cycles in the behavior recognition process, the shoulder turning behavior is performed. When it is detected that the shoulder is accidentally swung for one cycle (more precisely, less than the specified threshold), the fatigue reduction process is performed. It can be prevented from being executed. In other words, according to the behavior estimation device 1, it is possible to prevent the drivers from feeling uncomfortable by performing the fatigue reduction process by certifying that the shoulder-taking behavior is taken by mistake.
[Other effects]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  For example, in the behavior estimation device 1 of the above-described embodiment, the control command is output to the air conditioning control device 22 in the fatigue reduction process executed in S160 of the behavior estimation process, and the driver is blown with cold air. However, the control executed in the fatigue reduction process is not limited to this, and a control command is output to the navigation device 21 and information on the resting facilities that are located around the vehicle on which the driver can rest. (For example, the location of the break facility and the contents of the facilities that the break facility has) may be notified.

Even in such a case, by driving the driver to take a break, it is possible to cause the driver to perform safer driving as with the behavior estimation device 1 of the above-described embodiment.
Further, in the behavior estimation apparatus 1 of the above embodiment, the fatigue reduction process for reducing the driver's fatigue is performed in S160 of the behavior estimation process, but the process performed in S160 is limited to this. For example, processing such as issuing an alarm may be used.

  In the above-described embodiment, the driver of the on-board vehicle is included in the captured image as a target person who determines whether or not the shoulder-turning action is taken. The person is not limited to a person, and may be, for example, a person sitting in the passenger seat of the mounted vehicle or another person.

  In the above embodiment, the shoulder turning action is the specific action to be detected. However, the specific action to be detected is not limited to this. For example, after the driver raises the forearm (starting posture), the forearm Rotate the arm until it returns to the same point (end posture) (ie, rotate the forearm using the elbow as a fulcrum), or twist the body from the neutral posture (starting posture) to the left (or right). Then, turn your body to the right (or left), turn it back to the neutral position (end posture) It may be a specific action.

  However, in the case of setting the former arm turning action as the specific action to be detected, the return point when the wrist moves due to the arm turning action and the allowable area are actions represented by a relative coordinate system with the elbow as the origin. The estimation model needs to be stored in the ROM 10a. In addition, when the latter is a specific action to be detected, the return point when the left and right shoulders move due to the hip action and the allowable area are relative to the midpoint of the left and right shoulders as the origin. The behavior estimation model expressed by the coordinate system needs to be stored in the ROM 10a.

  Furthermore, in the said embodiment, although the action estimation apparatus 1 was mounted and used for the vehicle, the action estimation apparatus does not need to be mounted in the vehicle, for example, may be mounted in the aircraft or the ship. It may be used without being mounted on these moving bodies.

It is the block diagram which showed schematic structure of the action estimation apparatus. It is an example of a picked-up image. It is explanatory drawing which illustrated the feature point set to the arm part. It is a flowchart which shows the process sequence of an action estimation process. It is a flowchart which shows the process sequence of a model collation process. It is a flowchart which shows the process sequence of an action recognition process. It is explanatory drawing for demonstrating the preparation procedure of an action estimation model. It is explanatory drawing for demonstrating a return point and an acceptance | permission area | region. It is explanatory drawing for demonstrating the operation example of an action estimation apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Action estimation apparatus 10 ... Image processor 10a ... ROM 10b ... RAM 10c ... CPU 15 ... Imaging device 21 ... Navigation apparatus 22 ... Air-conditioning control apparatus

Claims (5)

Captured image acquisition means for acquiring a captured image obtained by capturing a person every time the captured image is captured;
A feature that uses a preset point on a person's body as a feature point and detects a detection feature point that is a feature point of the person reflected in the photographed image each time the photographed image is acquired by the photographed image acquisition unit Point detection means;
A behavior in which a predefined feature point repeatedly moves around a feature point defined separately from the feature point is defined as a specific behavior, and a plurality of feature points in which a continuously changing feature point of the person taking the specific behavior is located A behavior estimation model that includes a point as a return point, and is provided for each return point, and a region where a feature point of a person who has taken the specific action may be located is an allowable region. Model storage means for storing
The detected feature points detected by the feature point detecting means are located within a predetermined time within a predetermined time within a permissible area of the behavior estimation model stored in the model storing means, and a predetermined number of times. An action estimation device comprising: an action estimation unit that estimates that a person reflected in the captured image is taking the specific action when in the authorized state. The behavior estimation means includes
The behavior estimation apparatus according to claim 1, wherein when the detected feature point is located in the allowable region continuously for the specified number of times or more, the authorized state is set.   When it is estimated that the action estimating means is taking a specific action, it is provided with a fatigue reducing means for executing fatigue reduction control for reducing fatigue of a person reflected in the photographed image. The behavior estimation apparatus according to claim 1 or 2. The fatigue reducing means includes
A break in which a person reflected in the captured image can take a break with respect to a navigation device that detects the current position, acquires information on facilities located around the detected current position, and can notify the acquired information The behavior estimation apparatus according to claim 3, wherein outputting the control command so as to notify the facility information is executed as the fatigue level reduction control. The fatigue reducing means includes
A control command is output as the fatigue reduction control to output a control command so as to blow cool air on a person reflected in the photographed image to at least an air conditioner control device having a cooling function. The behavior estimation device according to claim 3 or claim 4.