JP2022134088A - Posture discrimination apparatus - Google Patents

Abstract

To accurately discriminate a posture of a person while suppressing erroneous discrimination.SOLUTION: A control apparatus 10 as a posture discrimination apparatus 30 comprises a skeleton point detection unit 13 for detecting skeleton points SP of a person included in a captured image Vd. The control apparatus 10 comprises: a posture discrimination probability value calculation unit 21 for calculating a posture discrimination probability value X for a person based on the detected skeleton points SP; and a posture discrimination unit 22 for discriminating a posture PH of a person based on the posture discrimination probability value X. The posture discrimination probability value calculating unit 21 calculates a fall-over discrimination probability value XC indicating probability of "a fall-over posture" as the posture discrimination probability value X. Further, the posture discrimination unit 22 comprises a fall-over erroneous discrimination determining unit 41 for determining erroneous discrimination of "the fall-over posture", based on the fall-over discrimination probability value XC. The fall-over erroneous discrimination determining unit 41 determines that there is a possibility of occurrence of erroneous discrimination of "the fall-over posture" based on the fall-over discrimination probability value XC if a change speed of the fall-over discrimination probability value XC is a predetermined speed threshold or more.SELECTED DRAWING: Figure 3

Description

The present invention relates to a posture determination device.

Conventionally, there has been proposed a method of determining the posture of an imaged person by detecting skeletal points of the person included in the imaged image. For example, the posture detection device described in Patent Literature 1 detects the head and trunk as the skeletal points of a person appearing in a captured image. In addition, the position, size, orientation, etc. of the head are detected in this posture detection device. Furthermore, the positional relationship between the head and trunk is detected. Based on these parameters, the posture of the person in the photographed image is determined.

WO2016/143641

However, when detecting skeletal points of a person included in a photographed image, the accuracy of detecting skeletal points may decrease depending on the positional relationship between the person and the camera. As a result, there is a possibility that the person's posture is erroneously determined.

A posture discrimination apparatus for solving the above-mentioned problems includes a skeleton point detection unit that detects skeleton points of a person included in a photographed image, and a posture discrimination probability value that calculates a posture discrimination probability value of the person based on the detection of the skeleton points. and a posture determination unit that determines the posture of the person based on the posture determination probability value, and the posture determination probability value calculation unit determines that the posture of the person is a falling posture as the posture determination probability value. a fall discrimination probability value indicating the probability that the posture discrimination unit is a fall discrimination determination unit that determines misjudgment of the fall posture based on the fall discrimination probability value; The turnover erroneous determination determining unit determines that the erroneous determination may occur when the rate of change of the turnover determination probability value is equal to or greater than a predetermined speed threshold.

That is, it takes a certain amount of time for the person's posture to change from the "standing posture" to the "falling posture". Therefore, if the rate of change of the fall determination probability value is too fast, the value is highly likely to be an abnormal value. Thus, it can be determined that there is a possibility that an erroneous determination will occur in the determination of the "falling posture" based on the falling determination probability value.

Therefore, according to the above configuration, even in a situation in which the fall determination probability value cannot be calculated correctly due to a decrease in the detection accuracy of the skeletal points based on the positional relationship with the camera, it is possible to quickly detect a "fall over" based on the fall determination probability value. Possibility of erroneous determination of "posture" can be detected. As a result, it is possible to suppress erroneous determination of the "falling posture" and accurately determine the posture of the person.

In the posture discrimination apparatus for solving the above-described problems, the posture discrimination probability value calculation unit inputs the feature amount calculated based on the detection of the skeleton point to an inference model generated by machine learning, thereby calculating the posture discrimination probability value. is preferably calculated.

According to the above configuration, it is possible to accurately determine the posture of a person based on the detection of the skeleton points by the image analysis. Furthermore, there is an advantage that it is possible to determine the posture of a person using only a two-dimensional image. And thereby, cost reduction can be achieved.

In the posture determination device for solving the above problems, the detection of the skeleton points, the calculation of the posture determination probability value, and the posture determination based on the posture determination probability value are periodically executed. a previous value holding unit that holds the result of the posture discrimination in the previous value as a previous value, and the posture discrimination unit stores the previous value is preferably used as the result of the posture determination in the current cycle.

According to the above configuration, it is possible to determine the posture of a person so that erroneous determination of "falling posture" based on the falling determination probability value calculated in the current cycle does not become the final output of posture determination. Accordingly, it is possible to accurately determine the posture of a person.

In the posture determination device for solving the above-described problems, the posture determination unit determines the determination result of the fall erroneous determination determination unit when the number of consecutive cycles in which the erroneous determination of the overturned posture is confirmed reaches a predetermined consecutive threshold value. It is preferable that the posture is determined to be the overturned posture regardless of the position.

That is, while the cycle in which the erroneous determination of the "falling posture" is confirmed continues, the posture of the person determined in the cycle before the erroneous determination is confirmed is maintained as the final output of the posture determination. It will be. As a result, there is a possibility that the posture of the person indicated by the result of posture determination deviates from the actual posture. However, if the cycle in which the person's posture is determined to be "falling posture" based on the fall discrimination probability value continues for a certain period, there is a possibility that the actual posture has changed to "falling posture". is high. In such a case, by assuming that the actual posture has already changed to the "falling posture", it is possible to match the actual posture with the posture indicated in the final output of posture determination. can.

In the posture discrimination device that solves the above problems, the posture discrimination unit holds a history of the posture discrimination probability values calculated before the previous cycle, and stores the posture discrimination probability values calculated in the current cycle. and determining the posture of the person based on the posture discrimination probability value after executing a low-pass filter process that reflects the history of the posture discrimination probability value, wherein the erroneous discrimination of the falling posture is confirmed. It is preferable to initialize the history of the posture discrimination probability values used in the low-pass filter processing when the number of consecutive cycles reaches the consecutive threshold.

That is, by performing low-pass filter processing on the posture discrimination probability value calculated in the current cycle with the history of the posture discrimination probability value initialized, the fall discrimination probability value calculated in the current cycle is changed. Based on this, the posture of the person is determined to be the “falling posture”. As a result, the posture of the person shown in the final output of the posture determination can be made to match the actual posture, which is assumed to have already changed to the "falling posture".

In the posture discrimination device for solving the above-described problems, the fall erroneous discrimination determination unit may cause the erroneous discrimination when the height of the skeleton point appearing in the captured image is equal to or higher than a predetermined height threshold. It is preferable to determine that

That is, when the posture of the person changes from the "standing posture" to the "falling posture", the height of the skeletal points appearing in the photographed image is lowered. Therefore, if the height of the skeletal point is too high, the value is highly likely to be an abnormal value.

Therefore, according to the above configuration, even in a situation in which the fall determination probability value cannot be calculated correctly due to a decrease in the detection accuracy of the skeletal points based on the positional relationship with the camera, it is possible to quickly detect a "fall over" based on the fall determination probability value. Possibility of erroneous determination of "posture" can be detected. Accordingly, it is possible to accurately determine the posture of a person.

In the posture determination device for solving the above-described problems, the fall erroneous determination determining unit determines that there is a possibility of the erroneous determination based on the rate of change of the fall determination probability value, and the height of the skeletal point When it is determined that there is a possibility that the misjudgment may occur based on the above, it is preferable to confirm the misjudgment of the falling posture.

According to the above configuration, erroneous determination of "falling posture" can be accurately determined based on the falling determination probability value.
In the posture determination device for solving the above problems, it is preferable that the posture determination unit determines the posture of an occupant positioned inside a vehicle interior as the posture of the person.

According to the above configuration, it is possible to accurately determine the posture of the occupant positioned in the passenger compartment of the vehicle while suppressing erroneous determination of the "overturned posture". In particular, in vehicles where passengers ride in a "standing position" that requires determination of "overturned posture", in many cases, a camera is installed in a position such as the ceiling where the occupants can be photographed from above. be done. Therefore, when the occupant moves below the camera, the image of the occupant in the photographed image tends to be in a state where the lower half of the body is hidden behind the head and the upper half of the body. As a result, the detection accuracy of the skeletal points is lowered, and the calculation accuracy of the posture discrimination probability value is also lowered. For example, when the skeletal points of a person included in the captured image are detected densely, the fall determination probability value may increase. Therefore, by implementing the fall misjudgment determination of each of the above configurations in such a configuration, a more remarkable effect can be obtained.

A posture discrimination apparatus for solving the above-mentioned problems includes a skeleton point detection unit that detects skeleton points of a person included in a photographed image, and a posture discrimination probability value that calculates a posture discrimination probability value of the person based on the detection of the skeleton points. and a posture determination unit configured to determine the posture of the person based on the posture determination probability value, wherein the posture determination unit determines erroneous determination of the posture of the person based on the posture determination probability value. and the misjudgment judgment unit judges that the misjudgment is likely to occur when the rate of change of the posture discrimination probability value is greater than or equal to a predetermined speed threshold.

That is, it takes a certain amount of time until the person's posture changes. Therefore, if the rate of change of the posture discrimination probability value is too fast, the value is highly likely to be an abnormal value. Accordingly, it can be determined that there is a possibility that an erroneous determination will occur in the determination of the "human posture" based on the posture determination probability value.

Therefore, according to the above configuration, even in a situation where the posture discrimination probability value cannot be calculated correctly due to a decrease in the detection accuracy of the skeletal points based on the positional relationship with the camera, the "human It is possible to detect the possibility of erroneous discrimination of "posture". Accordingly, it is possible to accurately determine the posture of a person.

ADVANTAGE OF THE INVENTION According to this invention, misjudgment is suppressed and a person's posture can be accurately discriminated.

1 is a perspective view of a vehicle provided with a posture determination device; FIG. FIG. 2 is an explanatory diagram of an occupant in a vehicle and a camera that captures the occupant; FIG. 2 is a block diagram showing a schematic configuration of a control device that functions as a posture determination device; Explanatory drawing which shows a person's skeleton point. 5 is a flowchart showing a processing procedure for posture determination based on detection of skeleton points; 4 is a flow chart showing a processing procedure for misjudgment of overturn based on the rate of change of the overturn judgment probability value. 4 is a flow chart showing a processing procedure of fall misjudgment determination based on the height of skeleton points. 4 is a flowchart showing a processing procedure for confirming misjudgment of a falling posture based on the possibility of misjudgment. 4 is a flow chart showing a mode of processing at the time of erroneous fall determination.

An embodiment of a posture determination device will be described below with reference to the drawings.
As shown in FIGS. 1 and 2, a vehicle 1 of this embodiment has a substantially square box-shaped vehicle body 2 extending in the vehicle front-rear direction. Further, a side surface of the vehicle body 2 is provided with a door opening 3 serving as an entrance/exit for passengers. The door opening 3 is provided with a pair of sliding doors 4, 4 which open and close in opposite directions to each other in the longitudinal direction of the vehicle. Then, the occupant 5 of the vehicle 1 sits on a seat 7 provided in the passenger compartment 6 in a "sitting posture", or, for example, using a strap or a handrail (not shown) in a "standing posture". It is configured to ride on 1.

In addition, the vehicle 1 of this embodiment is provided with a camera 8 for capturing an image of the inside of the vehicle interior 6 . In the vehicle 1 of this embodiment, the camera 8 is provided on the ceiling portion 9 of the passenger compartment 6 . For the camera 8, for example, a visible light camera or the like is used. The camera 8 of the present embodiment is thus configured to photograph the person H in the vehicle interior 6, that is, the passenger 5 who gets on the vehicle 1 from above.

As shown in FIG. 3 , in the vehicle 1 of the present embodiment, a photographed image Vd of the vehicle interior 6 captured by the camera 8 is input to the control device 10 . Then, the control device 10 of the present embodiment cooperates with the image analysis unit 11 that analyzes the captured image Vd, and by cooperating with this image analysis unit 11, based on the result of the image analysis, the image of the vehicle interior reflected in the captured image Vd is determined. A person recognition unit 12 for recognizing a person H in the vehicle 1, that is, an occupant 5 of the vehicle 1 is provided.

Furthermore, as shown in FIGS. 3 and 4, the control device 10 of the present embodiment cooperates with the image analysis unit 11 and the human recognition unit 12 to generate the photographed image Vd based on the result of the image analysis. and a skeleton point detection unit 13 for detecting the skeleton points SP of the person H included in the. That is, the skeletal points SP are unique points that characterize the body of the person H, such as joints and points on the surface of the body. , knees, ankles, etc. The control device 10 of the present embodiment includes an occupant information acquisition section 20 that acquires information on the occupant 5 appearing in the captured image Vd based on the detection of the skeleton points SP by the skeleton point detection section 13 .

Specifically, the person recognition unit 12 of the present embodiment executes recognition processing of the person H by using an inference model generated by machine learning. Then, the skeleton point detection unit 13 also uses the inference model generated by machine learning to execute detection processing of the skeleton points SP.

The occupant information acquisition unit 20 also includes a posture discrimination probability value calculation unit 21 that calculates the posture discrimination probability value X of the person H appearing in the captured image Vd based on the detection of the skeleton point SP, and the posture discrimination probability value X and a posture determination unit 22 that determines the posture PH of the person H using the posture determination unit 22 . Thus, the control device 10 of this embodiment has a function as the posture determination device 30 .

Specifically, the posture determination probability value calculator 21 of the present embodiment calculates feature amounts based on the positions of skeleton points SP of the occupant 5 such as the head and shoulders detected from the captured image Vd. Specifically, the posture discrimination probability value calculation unit 21 of the present embodiment calculates a feature amount based on the position on the two-dimensional coordinates in the captured image Vd. Further, the posture discrimination probability value calculation unit 21 calculates a feature amount based on body dimensions indicated by the plurality of skeletal points SP, such as the shoulder width of the occupant 5, for example. Then, the posture discrimination probability value calculation unit 21 of the present embodiment calculates the posture discrimination probability value X by inputting these feature amounts into an inference model generated by machine learning.

More specifically, the posture determination probability value calculation unit 21 of the present embodiment determines that the posture PH of the person H reflected in the captured image Vd in the vehicle interior 6, that is, the posture PH of the passenger 5 who is the subject of posture determination is "standing". A standing position discrimination probability value calculation unit 31 is provided for calculating the probability of being in the "posture". The posture determination probability value calculation unit 21 also includes a sitting position determination probability value calculation unit 32 that calculates the probability that the posture PH of the target occupant 5 is the "sitting posture". The posture determination probability value calculation unit 21 of the present embodiment includes a fall determination probability value calculation unit 33 that calculates the probability that the posture PH of the target occupant 5 is "falling posture".

That is, the posture determination unit 22 of the present embodiment uses the standing determination probability value XA calculated by the standing determination probability value calculation unit 31 and the sitting determination probability value calculated by the sitting determination probability value calculation unit 32 as the posture determination probability value X. A probability value XB and a fall determination probability value XC calculated by the fall determination probability value calculation unit 33 are obtained. Furthermore, the posture discrimination probability value calculation unit 21 calculates the posture discrimination probability value so that the total value of the standing posture discrimination probability value XA, the sitting posture discrimination probability value XB, and the fall discrimination probability value XC becomes "1.0". Perform an operation on the value X. Accordingly, the posture determination unit 22 of the present embodiment can determine the posture PH of the occupant 5 based on the posture determination probability value X without contradiction.

The standing discrimination probability value XA calculated by the standing discrimination probability value calculating unit 31 of the present embodiment further includes the probability that the occupant 5 in the "standing posture" is in the "moving state", the "stationary state and the probability of being in a state of using a strap or handrail. The posture determination unit 22 of the present embodiment is thus configured to be able to subdivide and determine the "standing posture".

(fall misjudgment judgment)
Next, in the control device 10 of the present embodiment, a fall misjudgment determination executed by the posture determination unit 22 and a mode of posture determination processing based on the determination result will be described.

As shown in FIG. 3 , the control device 10 of this embodiment includes a fall error determination determination section 41 provided in the posture determination section 22 . In the posture determination unit 22 of the present embodiment, the fall erroneous determination determination unit 41 determines erroneous determination of the "falling posture" based on the fall determination probability value XC. Then, the posture determination section 22 of the present embodiment is configured to change the mode of posture determination processing based on the determination result of the fall error determination determination section 41 .

That is, as shown in FIG. 2, in the vehicle 1 of this embodiment, when the occupant 5 in the passenger compartment 6 moves below the camera 8 provided on the ceiling 9, the occupant reflected in the captured image Vd The image of 5 tends to be in a state where the lower body is hidden behind the head and upper body. Furthermore, this lowers the detection accuracy of the skeleton point SP, and thus the calculation accuracy of the posture discrimination probability value X also lowers. For example, when the skeletal points SP of the person H included in the captured image Vd are detected densely, there is a possibility that the fall determination probability value XC increases. As a result, in the control device 10 of the present embodiment, in such a case, there is a problem that misjudgment of the "overturned posture" based on the overturn determination probability value XC is likely to occur.

Based on this point, in the control device 10 of the present embodiment, as described above, the fall misjudgment determination unit 41 provided in the posture determination unit 22 determines the "fall posture" based on the fall determination probability value XC. erroneous discrimination is determined. Thus, the control device 10 of the present embodiment suppresses the erroneous determination of the "falling posture" from becoming the final output of the posture determination, and accurately determines the posture PH of the occupant 5 reflected in the captured image Vd. It has a recognizable structure.

More specifically, as shown in the flowchart of FIG. 5, in the control device 10 of the present embodiment, an image Vd of the vehicle interior 6 captured by the camera 8 is periodically obtained (step 101). Subsequently, the skeleton point detection unit 13 detects the skeleton point SP (step 102), and the posture discrimination probability value calculation unit 21 calculates the feature amount FV based on the detection of the skeleton point SP. (step 103). In addition, in the control device 10 of the present embodiment, at this timing, the current time Tc and the execution time Tb of the previous cycle are acquired (step 104), and the elapsed time ΔT from the previous cycle to the current cycle is obtained. is calculated (ΔT=Tc−Tb, step 105). Furthermore, the execution time Tb of the previous period held in the storage area (not shown) is updated by the current time Tc calculated in step 103 (Tb=Tc, step 106). Then, the posture discrimination probability value calculation unit 21 inputs the feature value FV calculated in step 103 to the inference model generated by machine learning, thereby calculating the posture discrimination probability value X ( step 107).

Next, in the control device 10 of the present embodiment, the fall erroneous determination determination is performed by the fall erroneous determination determination unit 41 (step 108). Further, in the posture determination unit 22, it is determined whether or not the determination result by the fall erroneous determination determination unit 41 indicates erroneous determination of the "falling posture" based on the fall determination probability value XC ( step 109). Then, if the determination result of the fall erroneous determination determination unit 41 is "no erroneous determination" (step 109: YES), the posture determination unit 22 of the present embodiment calculates the posture determination probability calculated in the current cycle. Ordinary attitude determination processing based on the value X is executed.

Specifically, as shown in FIG. 3, the posture determination unit 22 of the present embodiment is a filter processing unit that performs low-pass filtering on the posture determination probability value X calculated by the overturn determination probability value calculation unit 33. 42. The low-pass filter processing executed by the filter processing unit 42 of this embodiment can be represented by the following equation (1).

即ち、本実施形態の姿勢判別部２２において、このフィルタ処理部４２が実行するローパスフィルタ処理は、その骨格点ＳＰの検出に同期して周期的に演算される姿勢判別確率値Ｘの履歴を反映する。つまりは、前回以前の周期において演算された姿勢判別確率値Ｘの値を利用するものとなっている。

That is, in the posture determination unit 22 of the present embodiment, the low-pass filter processing executed by the filter processing unit 42 reflects the history of the posture determination probability value X that is periodically calculated in synchronization with the detection of the skeleton point SP. do. In other words, the value of the posture discrimination probability value X calculated in the cycle before the previous cycle is used.

As shown in FIG. 5, the posture determination unit 22 of the present embodiment first performs this low-pass filter processing on the posture determination probability value X calculated in step 107 as the posture determination processing in the normal state ( step 110). Further, the posture determination unit 22 determines the posture of the passenger 5, who is the subject, based on the posture determination probability value X' after the low-pass filter processing (step 111). Thus, the posture determination unit 22 of the present embodiment is configured such that the result of posture determination based on the posture determination probability value X is less likely to be blurred such as chattering.

On the other hand, if the determination result of the fall erroneous determination determining unit 41 in step 109 is "misdetermined" (step 109: NO), the posture determining unit 22 of the present embodiment performs step 110 and The process of step 111 is not executed. Then, when the "overturned posture" is determined to be erroneously determined, the overturning erroneous determination processing is executed (step 112).

More specifically, as shown in FIG. 3, the overturning erroneous determination determination unit 41 of the present embodiment determines the "falling posture" based on the overturning determination probability value XC based on the rate of change of the overturning determination probability value XC. A changing speed determination unit 51 is provided for determining the possibility of erroneous determination in determination.

Specifically, as shown in the flowchart of FIG. 6, the change speed determination unit 51 of the present embodiment first acquires the fall determination probability value XC calculated in the current cycle (step 201). Next, when the change speed determination unit 51 acquires the previous value XCb of the fall determination probability value calculated in the previous cycle (step 202), it then compares the previous value XCb with the current fall determination probability value XC. A difference value ΔXC is calculated (ΔXC=│XCb−XC│, step 203). In addition, in the control device 10 of the present embodiment, the previous value XCb of the overturn determination probability value is held in a storage area (not shown). Then, the change speed determination unit 51 of the present embodiment divides the difference value ΔXC calculated in step 103 by the elapsed time ΔT from the previous cycle to the current cycle, thereby determining the change in the fall determination probability value XC. A velocity α is calculated (α=ΔXC/ΔT, step 204).

Next, the change speed determination unit 51 of the present embodiment compares the change speed α of the fall determination probability value XC with a predetermined speed threshold value αth (step 205). Then, when the change speed α is equal to or greater than the speed threshold value αth (α≧αth, step 205: YES), there is an erroneous determination of the “falling posture” based on the overturning determination probability value XC calculated in the current cycle. It is determined that there is a possibility of occurrence (step 206).

That is, it takes a certain amount of time for the posture PH of the person H to change from the "standing posture" to the "falling posture". For example, the following reference states that when a person H falls while walking, it takes about 0.3 seconds for the knee to touch the ground. Then, the maximum speed when the required time until the overturn is converted into the change speed α of the overturn determination probability value XC is "3.3/second".

(Reference) "E.T. Hsiao and S.N. Robinovitch, "Common protective movements govern unexpected falls from standing height." Journal of biomechanics, vol.31, no.1, pp.1-9, 1997."

Based on this point, in the change speed determination unit 51 of the present embodiment, when the occupant 5 inside the vehicle interior 6 shown in the captured image Vd actually falls, the assumed value that occurs in the change speed α of the fall determination probability value XC is The velocity threshold value αth is set corresponding to the . Then, when the change speed α equal to or greater than the speed threshold value αth occurs, the change speed determination unit 51 of the present embodiment regards the change speed α as an abnormal value, and there is a possibility that an erroneous determination of the “falling posture” will occur. It is configured to determine that there is

At step 205, when the rate of change α of the overturn determination probability value XC is lower than the speed threshold value αth (α<αth, step 205: NO), the speed threshold value αth is determined to be a normal value. Then, the changing speed determination unit 51 of the present embodiment is configured to determine that there is no erroneous determination of the "falling posture" based on the overturning determination probability value XC calculated in the current cycle ( step 207).

Further, as shown in FIG. 3, the fall erroneous determination determination unit 41 of the present embodiment determines the "fall posture" based on the fall determination probability value XC based on the height β of the skeleton point SP appearing in the captured image Vd. A skeletal point height determination unit 52 is provided for determining the possibility of erroneous determination in the determination of .

Specifically, as shown in the flowchart of FIG. 7, the skeletal point height determination unit 52 of this embodiment first acquires the height β of the skeletal point SP from the captured image Vd of the passenger 5 who is the subject. (step 301).

Specifically, as shown in FIG. 4, in the skeletal point height determination unit 52 of the present embodiment, a predetermined skeletal point SPx is set in advance as the skeletal point SP for obtaining the height β in the captured image Vd. is set. Specifically, as the predetermined skeleton point SPx, for example, a skeleton point SP located in the lower body of the passenger 5, such as the waist, knees, or ankles, is used. Further, in the vehicle 1 of the present embodiment, the Y-axis direction of the two-dimensional coordinates (X, Y) in the captured image Vd captured by the camera 8 is set to be the height direction of the compartment 6. . In other words, in the posture determination device 30 configured by the control device 10 of this embodiment, the Y axis of the captured image Vd is the height coordinate. The skeletal point height determination unit 52 of the present embodiment is thus configured to obtain the height β of the skeletal point SP appearing on the Y-axis coordinates in the captured image Vd.

As shown in FIG. 7, the skeletal point height determination unit 52 of the present embodiment then compares the height β of the skeletal point SP obtained from the captured image Vd with a predetermined height threshold value βth (step 302). ). Then, when the height β of the skeleton point SP is equal to or greater than the height threshold βth (β≧βth, step 302: YES), the “falling posture” is determined based on the falling determination probability value XC calculated in the current cycle. It is determined that there is a possibility that an erroneous determination may occur (step 303).

That is, when the posture PH of the person H changes from the "standing posture" to the "falling posture", the height β of the skeleton point SP appearing in the captured image Vd is reduced. Based on this point, in the skeletal point height determination unit 52 of the present embodiment, the predetermined skeletal point SPx corresponds to an assumed value when the occupant 5 inside the vehicle interior 6 reflected in the captured image Vd actually falls over. Then, the height threshold βth is set. Then, when the height β of the acquired skeletal point SP is equal to or greater than the height threshold value βth, the skeletal point height determination unit 52 of the present embodiment regards the value of the height β as an abnormal value, It is configured to determine that there is a possibility of erroneous determination of "falling posture".

In step 302, if the height β of the skeleton point SP is lower than the height threshold βth (β<βth, step 302: NO), it is determined that the height threshold βth is a normal value. Then, the skeletal point height determination unit 52 of the present embodiment is configured to determine that there is no erroneous determination of the "falling posture" based on the fall determination probability value XC calculated in the current cycle. (step 304).

More specifically, as shown in the flowchart of FIG. 8, in the turnover erroneous determination determination unit 41 of the present embodiment, first, the change speed determination unit 51 executes change speed determination for the turnover determination probability value XC. (Step 401). Further, when it is determined that there is a possibility of erroneous determination of the “falling posture” in the change rate determination of the overturning determination probability value XC (step 402: YES), the skeletal point height determining unit 52 Height determination of the point SP is performed (step 403). When it is determined that there is a possibility of erroneous determination of the "falling posture" in the height determination of the skeleton point SP (step 404: YES), the fall erroneous determination determination unit 41 of the present embodiment determines that the fall The erroneous discrimination of "falling posture" based on the discrimination probability value XC is determined (step 405).

Further, when the erroneous determination of the "falling posture" is denied based on the change speed α of the fall determination probability value XC (step 402: NO), the fall determination probability value XC is used as the fall determination probability value XC. Determine whether the "overturned posture" is normal (step 406). Then, even if the erroneous determination of the "falling posture" is denied based on the height β of the skeleton point SP (step 404: NO), the fall determination determination unit 41 of the present embodiment performs the fall determination in step 406. It is configured to determine the normal determination of the "falling posture" based on the probability value XC.

Further, when the erroneous determination of "falling posture" is determined in step 405, the fall erroneous determination determination unit 41 of the present embodiment increments a counter (not shown) (N=N+1, step 407). On the other hand, if it is determined that the "overturned posture" is normal in step 406, the counter (not shown) is cleared (N=0, step 408). In other words, the overturning erroneous determination determining unit 41 of the present embodiment has a function of counting the number of consecutive times N in which the erroneous determination of the "falling posture" is confirmed. Then, the posture determination unit 22 of the present embodiment executes the overturning erroneous determination processing corresponding to the case where the erroneous determination of the "falling posture" is determined based on the consecutive number N of the cycles in which the erroneous determination is confirmed. (See FIG. 5, step 112).

More specifically, as shown in FIG. 9, the posture determination unit 22 of the present embodiment obtains the number of consecutive times N of cycles in which the erroneous determination of the "falling posture" has been determined (step 501). It is compared with a predetermined continuity threshold Nth (step 502). In the posture determination unit 22 of the present embodiment, the continuation threshold Nth is set to, for example, about "three times". Then, when the number of consecutive times N is less than the consecutive threshold value Nth, erroneous determination of the "falling posture" based on the overturning discrimination probability value XC calculated in the current cycle does not become the final output of this posture discrimination. , the posture PH of the occupant 5 is determined.

Specifically, as shown in FIG. 3, the posture determination unit 22 of the present embodiment converts the posture PH of the occupant 5 determined in the previous cycle, that is, the posture determination result in the previous cycle, into the previous value PHb is provided with a previous value holding unit 55 that holds the value as . Then, as described above, when the erroneous determination of the "falling posture" is confirmed in the overturning erroneous determination determination unit 41 (see FIG. 8, step 405), the previous value PHb is used as the posture determination result in the current cycle. It is configured to

That is, as shown in FIG. 9, the posture determination unit 22 of the present embodiment, when executing the fall misjudgment process, if the number of consecutive misjudgments N is less than the consecutive threshold value Nth (N<Nth, step 502 : YES), the previous value PHb is read from the previous value holding unit 55 (step 503). Then, in the subsequent step 504, the result of posture discrimination in the previous cycle indicated by the previous value PHb is used for posture discrimination in the current cycle. That is, the posture PH of the occupant 5 determined in the previous cycle is set to the posture PH of the occupant 5 determined in the current cycle (PH=PHb, step 504).

Further, in step 502, the posture determination unit 22 of the present embodiment, when the number N of consecutive erroneous determinations has reached the continuous threshold value Nth (N≧Nth, step 502: NO), Take no action. Then, the posture determination unit 22 of the present embodiment does not depend on the determination result of the overturning erroneous determination determination unit 41 (see FIG. 8 , step 405) that confirms the erroneous determination of the “falling posture”, and the posture PH of the occupant 5 is It is configured to determine that it is in a "falling posture".

Specifically, in this case, the posture determination section 22 of the present embodiment initializes the low-pass filter processing executed by the filter processing section 42 (step 505). Specifically, the initialization of the low-pass filter in step 401 is performed by initializing the history of posture discrimination probability values X used in the low-pass filter processing. In other words, on the right side of the above equation (1), the first term "i = 1 to P" is set to "zero matrix", and the second term "j = 1 to Q" is set to "zero matrix". do. Furthermore, in the posture discrimination section 22 of the present embodiment, in the state initialized by this, the posture discrimination probability value X (see FIG. 5, step 107) calculated in the current cycle is processed by the filter processing section 42. performs low-pass filtering (step 506). Then, the posture determination unit 22 of the present embodiment determines the posture of the occupant 5 based on the posture determination probability value X′ after the low-pass filter processing executed in step 506, so that the posture PH is the “falling posture”. (step 507).

That is, while the cycle in which the erroneous determination of the “falling posture” is confirmed continues, the posture PH of the occupant 5 determined in the cycle before the erroneous determination is confirmed for the first time is used as the final output of the posture determination unit 22. will be maintained. Further, during this period, the history of posture discrimination probability value X used for low-pass filtering is also not updated. As a result, the posture PH of the occupant 5 indicated by the final output of the posture determination section 22 may deviate from the actual posture PH.

Based on this point, the posture determination unit 22 of the present embodiment, as described above, determines that the actual posture is It is assumed that the PH has already changed to the "falling posture". That is, when the cycle in which the posture PH of the occupant 5 is determined to be the "falling posture" based on the overturning determination probability value XC continues to a certain extent, the actual posture PH changes to the "falling posture". likely to be Furthermore, by performing low-pass filter processing on the posture discrimination probability value X calculated in the current cycle with the history of the posture discrimination probability value X initialized, the fall discrimination probability calculated in the current cycle Based on the value XC, the posture PH of the occupant 5 is determined to be the "overturned posture". In the control device 10 of the present embodiment, the posture PH of the occupant 5 indicated by the final output of the posture determination section 22 is thereby matched with the actual posture PH.

Next, the operation of this embodiment will be described.
In the control device 10 functioning as the posture determination device 30, the skeletal points SP of the occupant 5 appearing in the photographed image Vd inside the vehicle interior 6 are detected, and the skeletal points SP of the occupant 5 are detected based on the detection of the skeletal points SP. A posture determination probability value X is calculated. Based on this posture determination probability value X, the posture PH of the occupant 5 is determined.

Further, in the control device 10, as the posture discrimination probability value X, a fall discrimination probability value XC indicating the probability that the posture PH of the occupant 5 is a "falling posture" is calculated, and this fall discrimination probability value XC is calculated. is calculated. If the rate of change α of the overturn determination probability value XC is equal to or greater than a predetermined speed threshold value αth, there is a possibility that an erroneous determination may occur in the determination of the "overturned posture" based on the overturn determination probability value XC. be judged.

Next, the effects of this embodiment will be described.
(1) The control device 10 as the posture determination device 30 includes the skeleton point detection unit 13 that detects the skeleton points SP of the person H included in the captured image Vd. The control device 10 also includes a posture discrimination probability value calculation unit 21 that calculates a posture discrimination probability value X of the person H based on the detection of the skeleton point SP, and a posture discrimination probability value X of the person H based on the posture discrimination probability value X. and a posture determination unit 22 for determination. In addition, the posture discrimination probability value calculation unit 21 calculates, as the posture discrimination probability value X, a fall discrimination probability value XC that indicates the probability that the posture PH of the person H is "falling posture". Further, the posture determination unit 22 includes a fall misjudgment determination unit 41 that determines misdetermination of the “fall posture” based on the fall determination probability value XC. Then, when the rate of change α of the fall determination probability value XC is equal to or greater than a predetermined speed threshold value αth, the fall determination probability value XC causes an erroneous determination of the “fall posture” based on the fall determination probability value XC. determine that it is possible.

That is, it takes a certain amount of time for the posture PH of the person H to change from the "standing posture" to the "falling posture". Therefore, if the rate of change α of the fall determination probability value XC is too fast, the value is highly likely to be an abnormal value. Accordingly, it can be determined that there is a possibility that an erroneous determination will occur in the determination of the "falling posture" based on the overturning determination probability value XC.

Therefore, according to the above configuration, even in a situation where the detection accuracy of the skeletal point SP is lowered based on the positional relationship with the camera 8 and the overturn determination probability value XC cannot be calculated correctly, the overturn determination probability value XC can be quickly detected. It is possible to detect the possibility of misjudgment of "falling posture". As a result, it is possible to suppress erroneous determination of the "falling posture" and accurately determine the posture PH of the person H.

(2) Posture discrimination probability value calculation unit 21 calculates posture discrimination probability value X by inputting feature quantity FV calculated based on detection of skeleton point SP to an inference model generated by machine learning.

According to the above configuration, it is possible to accurately determine the posture PH of the person H based on the detection of the skeleton points SP by the image analysis. Furthermore, there is an advantage that the posture of the person H can be determined using only the two-dimensional image. And thereby, cost reduction can be achieved.

(3) The control device 10 performs posture determination based on the detection of the skeleton point SP by the skeleton point detection unit 13, the posture determination probability value X by the posture determination probability value calculation unit 21, and the posture determination probability value X by the posture determination unit 22. cyclically. The posture determination unit 22 also includes a previous value holding unit 55 that holds, as the previous value PHb, the posture PH of the person H determined in the previous cycle, that is, the posture determination result in the previous cycle. Then, based on the possibility of the erroneous determination, if the erroneous determination of "falling posture" is confirmed by the erroneous determination of overturning determination unit 41, the posture determination unit 22 sets the previous value PHb to the posture determination in the current cycle. be the result of

According to the above configuration, it is possible to determine the posture PH of the person H so that erroneous determination of the "falling posture" based on the overturning determination probability value XC calculated in the current cycle does not become the final output of the posture determination. can. Accordingly, the posture PH of the person H can be accurately determined.

(4) When the number of consecutive times N in which the erroneous determination of the “falling posture” is confirmed reaches a predetermined consecutive threshold value Nth, the posture determination unit 22 does not depend on the determination result of the fall erroneous determination determination unit 41, It is configured to determine that the posture PH of the person H is the "falling posture".

That is, while the cycle in which the erroneous determination of the "falling posture" is confirmed continues, the posture PH of the person H determined in the cycle before the erroneous determination is confirmed for the first time is maintained as the final output of posture determination. will be As a result, there is a possibility that the posture PH of the person H indicated by the posture determination result deviates from the actual posture PH. However, when the cycle in which the posture PH of the person H is determined to be the "falling posture" based on the fall determination probability value XC continues to a certain extent, the actual posture PH changes to the "falling posture". likely to be In such a case, it is assumed that the actual posture PH has already changed to the "falling posture", so that the posture PH of the person H shown in the final output of the posture determination and the actual posture PH are compared. match can be achieved.

(5) Posture determination unit 22 holds a history of posture determination probability values X calculated before the previous cycle. In addition, the posture determination unit 22 performs a low-pass filtering process on the posture determination probability value X calculated in the current cycle to reflect the history of the posture determination probability value X, and converts the posture determination probability value X′ to Based on this, the posture PH of the person H is determined. Then, when the number of consecutive times N of the cycle in which the erroneous determination of the “falling posture” is determined reaches the continuous threshold value Nth, the posture determination unit 22 initializes the history of the posture determination probability value X used in the low-pass filtering process. do.

That is, by performing low-pass filter processing on the posture discrimination probability value X calculated in the current cycle in a state in which the history of the posture discrimination probability value X is initialized, the fall discrimination probability calculated in the current cycle Based on the value XC, the posture PH of the person H is determined to be the “falling posture”. As a result, the posture PH of the person H indicated in the final output of posture determination can be matched with the actual posture PH, which is assumed to have already changed to the "falling posture".

(6) When the height β of the skeletal point SP appearing in the captured image Vd is equal to or greater than a predetermined height threshold value βth, the fall erroneous determination determination unit 41 determines the “fall posture” based on the fall determination probability value XC. It is determined that misclassification may occur.

That is, when the posture PH of the person H changes from the "standing posture" to the "falling posture", the height β of the skeleton point SP appearing in the captured image Vd is reduced. Therefore, if the height β of the skeletal point SP is too high, there is a high possibility that the value is an abnormal value.

Therefore, according to the above configuration, even in a situation where the detection accuracy of the skeletal point SP is lowered based on the positional relationship with the camera 8 and the overturn determination probability value XC cannot be calculated correctly, the overturn determination probability value XC can be quickly detected. It is possible to detect the possibility of misjudgment of "falling posture". Accordingly, the posture PH of the person H can be accurately determined.

(7) The fall discrimination determination unit 41 determines that there is a possibility of misjudgment based on the rate of change α of the fall discrimination probability value XC, and based on the height β of the skeleton point SP. If it is determined that there is a possibility, erroneous determination of the "falling posture" based on the overturning determination probability value XC is determined. As a result, erroneous determination of the "falling posture" can be accurately determined based on the overturning determination probability value XC.

(8) The posture determination unit 22 determines the posture PH of the occupant 5 positioned inside the passenger compartment 6 of the vehicle 1 as the posture PH of the person H.
According to the above configuration, the posture PH of the occupant 5 positioned in the passenger compartment 6 of the vehicle 1 can be determined with high accuracy while suppressing erroneous determination of the "overturned posture". In particular, in a vehicle 1 in which an occupant 5 rides in a "standing posture" that requires determination of an "overturned posture", in many cases, the occupant 5 in the passenger compartment 6 is often viewed from above, for example, the ceiling portion 9 or the like. A camera 8 is installed at a position for photographing. Therefore, when the occupant 5 moves below the camera 8, the image of the occupant 5 captured in the captured image Vd tends to be in a state where the lower half of the body is hidden behind the head and upper half of the body. As a result, the detection accuracy of the skeleton point SP is lowered, and the calculation accuracy of the posture discrimination probability value X is also lowered. For example, when the skeletal points SP of the person H included in the captured image Vd are detected densely, there is a possibility that the fall determination probability value XC increases. Therefore, with such a configuration, a more remarkable effect can be obtained by carrying out the fall erroneous determination as described above.

It should be noted that the above embodiment can be implemented with the following modifications. The above embodiments and the following modifications can be combined with each other within a technically consistent range.

- In the above embodiment, the feature quantity FV of the person H is calculated based on the positions of the skeleton points SP included in the captured image Vd and the body dimensions indicated by the plurality of skeleton points SP. Then, by inputting the feature quantity FV to an inference model generated by machine learning, the posture determination probability value X of the person H appearing in the captured image Vd is calculated. However, the present invention is not limited to this, and may be applied to a configuration in which the posture determination probability value X is calculated based on the detection of the skeleton point SP by the image analysis using a statistical method, map calculation, or the like.

- The format of the camera 8 may be arbitrarily changed, for example, using an infrared camera or the like. Furthermore, the position where the camera 8 is installed may also be changed arbitrarily. However, it is desirable to install it at a high position so that the whole body of the person H, who is the target person, can be seen. A configuration using two or more cameras 8 may also be used.

- Also, the skeleton points SP and body dimensions used to calculate the posture discrimination probability value X may be set arbitrarily.
- In the above-described embodiment, the posture PH of the occupant 5 positioned in the passenger compartment 6 of the vehicle 1 is determined as the posture PH of the person H appearing in the captured image Vd. However, the present invention is not limited to this, and may be applied to, for example, a configuration for determining the posture PH of a person H appearing in the photographed image Vd on the street or in the interior of a building.

- In the above-described embodiment, the posture PH of the person H reflected in the captured image Vd is determined as "standing posture", "sitting posture", or "falling posture". The "standing posture" is further subdivided into "moving state", "stationary state", and "state in which straps or handrails are used". However, the present invention is not limited to this, and for example, the "sitting posture" may also be applied to a configuration in which more subdivided determination is performed, such as determination of a plurality of states. Then, for example, it may be applied to a configuration with a small number of determinations, such as determining which one of the "standing posture" and "falling posture".

In the above embodiment, it is determined that there is a possibility of misclassification based on the rate of change α of the fall determination probability value XC, and that there is a possibility of misclassification based on the height β of the skeleton point SP. In this case, the erroneous determination of the "falling posture" is decided based on the falling determination probability value XC. However, not limited to this, it is determined that there is a possibility of misclassification based on the change rate α of the fall discrimination probability value XC, or it is determined that there is a possibility of misclassification based on the height β of the skeleton point SP. A configuration may be adopted in which, when a determination is made, erroneous determination of the "falling posture" based on the overturning determination probability value XC is determined. Further, it is also possible to combine the results of fall erroneous determination by other methods to determine the erroneous determination of the "falling posture". Then, the erroneous determination of the "falling posture" may be confirmed based on the rate of change α of the fall determination probability value XC, without executing the erroneous determination of the "falling posture" based on the height β of the skeletal point SP. .

- The speed threshold αth may be changed arbitrarily. Also, the height threshold βth may be set arbitrarily. The predetermined skeleton point SPx for obtaining the height β of the skeleton point SP may be set arbitrarily. Furthermore, the method of obtaining the height β of the skeleton point SP may also be changed. Then, the continuity threshold Nth may also be changed arbitrarily.

- In the above-described embodiment, the filter processing unit 42 that performs low-pass filtering on the posture determination probability value X is provided.

In the above embodiment, the posture discrimination probability value calculation unit 21 calculates, as the posture discrimination probability value X, the fall discrimination probability value XC that indicates the probability that the posture PH of the person H is "falling posture". Furthermore, the posture determination unit 22 determines erroneous determination of the “falling posture” based on the fall determination probability value XC as an erroneous determination determination unit that determines erroneous determination of the posture PH of the person H based on the posture determination probability value. A fall erroneous determination determination unit 41 is provided. Then, when the change speed α of the fall determination probability value XC is equal to or greater than a predetermined speed threshold value αth, the fall determination probability value XC is used as the fall determination probability value XC. It was decided that there is a possibility that this could occur.

However, not limited to this, the posture determination section 22 includes an erroneous determination determination section other than the fall erroneous determination determination section 41 . Then, the erroneous determination determination unit may determine the erroneous determination of the posture PH of the person H based on the comparison between the change speed of the posture determination probability value X and a predetermined speed threshold.

That is, the determination determination unit determines whether or not the rate of change of the posture determination probability value X corresponding to the posture P of the person H is equal to or greater than a predetermined speed threshold for the posture PH of the person H other than the "falling posture". do. Then, it may be determined that there is a possibility that the posture PH of the person H is erroneously determined based on the posture determination probability value X when the change speed is equal to or greater than a predetermined speed threshold.

In this case, the posture PH of the person H who performs the erroneous determination may be set arbitrarily. Then, regarding the postures PH of a plurality of persons H, including misjudgment of "falling posture", misjudgment is performed based on comparison between the change speed of the posture discrimination probability value X and a predetermined speed threshold. good too.

・In addition, when misjudgment of the posture PH of the person H is confirmed, or when the number of consecutive times of the cycle in which the misjudgment is confirmed reaches the continuous threshold, or the use of a low-pass filter, etc. It may be applied to determination of postures other than the "falling posture".

REFERENCE SIGNS LIST 10 control device 13 skeletal point detection unit 21 posture determination probability value calculation unit 22 posture determination unit 30 posture determination device 41 fall error determination determination unit Vd photographed image H person SP skeleton point X posture determination Probability value PH: Posture XC: Fall determination probability value α: Change speed αth: Speed threshold

Claims (9)

a skeletal point detection unit that detects skeletal points of a person included in a captured image;
a posture discrimination probability value calculation unit that calculates a posture discrimination probability value of the person based on the detection of the skeleton points;
a posture determination unit that determines the posture of the person based on the posture determination probability value;
The posture discrimination probability value calculation unit calculates a fall discrimination probability value indicating a probability that the posture of the person is a posture of falling as the posture discrimination probability value,
The posture determination unit includes a fall erroneous determination determination unit that determines erroneous determination of the fall posture based on the fall determination probability value,
The fall erroneous determination determining unit determines that there is a possibility of the erroneous determination when a change speed of the fall determination probability value is equal to or greater than a predetermined speed threshold. The posture determination device according to claim 1,
The posture discrimination probability value calculation unit calculates the posture discrimination probability value by inputting the feature amount calculated based on the detection of the skeleton point into an inference model generated by machine learning;
A posture determination device characterized by: In the posture determination device according to claim 1 or claim 2,
The detection of the skeletal points, the calculation of the posture discrimination probability value, and the posture discrimination based on the posture discrimination probability value are periodically executed,
a previous value holding unit that holds, as a previous value, the result of posture determination in a previous cycle;
The posture determination unit uses the previous value as the posture determination result in the current cycle when the misjudgment of the falling posture is confirmed based on the possibility of occurrence of the misjudgment.
A posture determination device characterized by: In the posture determination device according to claim 3,
When the number of consecutive cycles in which the erroneous determination of the overturned posture is confirmed reaches a predetermined consecutive threshold value, the posture determination unit determines whether the posture is the overturned posture regardless of the determination result of the overturning erroneous determination determination unit. A posture determination device characterized by being configured to determine that In the posture determination device according to claim 4,
The posture determination unit retains a history of the posture determination probability values calculated before the previous cycle, and reflects the history of the posture determination probability values in the posture determination probability values calculated in the current cycle. The posture of the person is determined based on the posture determination probability value after performing the low-pass filtering process,
initializing the history of the posture discrimination probability value used in the low-pass filter processing when the number of consecutive cycles in which the misjudgment of the falling posture is confirmed reaches the consecutive threshold;
A posture determination device characterized by: In the posture determination device according to any one of claims 1 to 5,
the fall misjudgment determining unit determines that the misjudgment may occur when the height of the skeletal point appearing in the captured image is equal to or greater than a predetermined height threshold;
A posture determination device characterized by: In the posture determination device according to claim 6,
The fall misjudgment determining unit determines that the misjudgment may occur based on the rate of change of the fall discrimination probability value, and determines the possibility that the misjudgment may occur based on the height of the skeleton point. Determining misjudgment of the falling posture when it is determined that there is
A posture determination device characterized by: In the posture determination device according to any one of claims 1 to 7,
The posture determination device, wherein the posture determination unit determines the posture of an occupant positioned in a vehicle interior as the posture of the person. a skeletal point detection unit that detects skeletal points of a person included in a captured image;
a posture discrimination probability value calculation unit that calculates a posture discrimination probability value of the person based on the detection of the skeleton points;
a posture determination unit that determines the posture of the person based on the posture determination probability value;
The posture determination unit includes an erroneous determination determination unit that determines erroneous determination of the posture of the person based on the posture determination probability value,
The erroneous determination determining unit determines that the erroneous determination may occur when a change speed of the posture determination probability value is equal to or greater than a predetermined speed threshold.

Images