JP2008146583A - Attitude detector and behavior detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an attitude detector capable of highly accurately detecting an attitude of a person to be observed. <P>SOLUTION: The attitude detector is provided with: a human body area information extraction means for extracting three-dimensional human body area information on the basis of a photographed image of the person to be observed; a calculation means for calculating the height, width and depth information of a three-dimensional human body area from the three-dimensional human body area information; and a decision means for deciding the attitude of the person to be observed on the basis of the ratio of the width to the height of the three-dimensional human body area and the ratio of the depth to the height of the three-dimensional human body area. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a posture detection device that detects the posture of a person to be observed and a behavior detection device that detects a behavior of the person to be observed such as a fall.

  As a device for detecting the fall of the person to be observed, the person to be observed is photographed with a camera, the motion vector of the person to be observed is detected from the photographed image, and the motion vector thus detected and the fall vector of the person to be observed stored in advance Has already been developed to determine whether or not the person to be observed has fallen (see Japanese Patent Laid-Open No. 2002-232870).

  In addition, as a device for detecting the fall of the person to be observed, the person to be observed is photographed with a camera, the difference area between the photographed image and the reference image photographed when there is no person to be observed is extracted, and the area of the difference area Based on the above, what has been developed to determine whether or not the subject has fallen has been developed (see Japanese Patent Application Laid-Open No. 2000-207664).

In addition, as a device for determining whether the observer is in a moving state or a stationary state, the observer is photographed with a camera, a person area is extracted from the photographed image, and the image coordinates of the top of the head are extracted from the extracted person area, Using the height data of the person to be observed, the image coordinates of the top of the head are converted into real world coordinates, and the distance between the real world coordinates of the top of the obtained head and the real world coordinates of the top of the head for a predetermined time is calculated and obtained. A device that determines whether the observer is in a moving state or a stationary state by comparing the obtained distance with a predetermined threshold has been developed (see Japanese Patent Application Laid-Open No. 2002-197463).
JP 2002-232870 A JP 2000-207664 A JP 2002-197463 A

  An object of this invention is to provide the attitude | position detection apparatus which can detect an observer's attitude | position with high precision.

  Another object of the present invention is to provide a behavior detection device that can detect the occurrence of a behavior such as a fall of the person to be observed with high accuracy.

  According to the first aspect of the present invention, in the posture detection device, the person area information extracting means for extracting the three-dimensional person area information based on the photographed image of the observer, the height of the three-dimensional person area from the three-dimensional person area information. Calculating means for calculating width, depth information, and determining means for determining the posture of the person to be observed based on the ratio of the width to the height of the three-dimensional person area and the ratio of the depth to the height of the three-dimensional person area It is characterized by having.

  According to a second aspect of the present invention, in the posture detection device, the person area information extracting means for extracting the three-dimensional person area information based on the photographed image of the person to be observed, the back muscle of the person to be observed from the three-dimensional person area information. It is characterized by comprising a calculating means for calculating the orientation and a judging means for determining the posture of the subject based on the orientation of the back muscle of the subject.

  According to a third aspect of the present invention, in the posture detection apparatus according to the second aspect, the calculation means uses the height direction of the three-dimensional person region as the Y axis, the width direction as the X axis, and the depth direction as the Z axis. A first projection image is generated by projecting 3D human area information onto the XY plane, and a second projected image is generated by projecting 3D human area information onto the YZ plane. A means for calculating a center of gravity for each projection image, a means for correcting all coordinate values of each projection image to a coordinate value having the center of gravity as the origin, and a principal component analysis from the first projection image after the coordinate value correction. To calculate a first vector representing the slope of the first principal component, and to compute a second vector representing the slope of the first principal component from the second projected image after the coordinate value correction using principal component analysis. And means for combining the first vector and the second vector It makes obtains a vector indicating the direction of the spine of the observed person, characterized in that from the obtained vector is provided with means for determining the orientation of the back muscles of the observer.

  According to a fourth aspect of the present invention, in the posture detection apparatus according to the second aspect, the calculation means uses the height direction of the three-dimensional person region as the Y axis, the width direction as the X axis, and the depth direction as the Z axis. A first projection image is generated by projecting 3D human area information onto the XY plane, and a second projected image is generated by projecting 3D human area information onto the YZ plane. Means for calculating the center of gravity for each projected image, means for correcting all coordinate values of each projected image to coordinate values having the center of gravity as the origin, and passing through the origin from the first projected image after the coordinate value correction. In addition, the inclination of the first line segment having the maximum length among the line segments extending to the outline of the person region is obtained, and the second projection image after the coordinate value correction is passed through the origin and the both ends are The length of the line segment extending to the outline of the person area is the maximum. It means determining the slope of the second line, and on the basis of the inclination of the first line segment of slope and a second segment, characterized in that it comprises a means for determining the orientation of the spine of the observed person.

  According to a fifth aspect of the present invention, in the behavior detection apparatus, the person area information extracting means for extracting the three-dimensional person area information based on the photographed image of the person to be observed, based on the three-dimensional person area information. Attitude determination means for determining attitude Based on the determination result of the attitude determination means, behavior determination data generation means for generating behavior determination data for determining the behavior of the detection target from the three-dimensional human region information, Corresponding to the storage means that accumulates the data for behavior determination during the period when the behavior may have occurred, the time series data obtained from the data for behavior determination accumulated by the storage means, and the behavior of the detection target It is characterized by comprising determination means for determining whether or not the behavior of the detection target has occurred by comparing with a model created in advance from learning data.

  According to a sixth aspect of the present invention, in the behavior detection apparatus according to the fifth aspect, the behavior determination data generation means is for behavior determination including the height, width, and depth of the three-dimensional human region from the three-dimensional human region information. Generating data, wherein the time-series data is time-series data of height, width and depth of the three-dimensional person area accumulated by the accumulation means, and the model is an HMM model. .

  According to a seventh aspect of the present invention, in the behavior detection apparatus according to the fifth aspect, the behavior determination data generation means is for behavior determination including the height, width, and depth of the three-dimensional human region from the three-dimensional human region information. Data for generating data, wherein the time-series data is calculated from the height, width, and depth of the three-dimensional person area accumulated by the accumulation means, and is a symbolized amount of movement of the observer And the model is a DP matching model.

  According to the present invention, the posture of the person to be observed can be detected with high accuracy. Moreover, according to this invention, generation | occurrence | production of behavior, such as a to-be-observed person's fall, can be detected now with high precision.

  Hereinafter, with reference to the drawings, an embodiment when the present invention is applied to a fall detection device for detecting a fall will be described.

[1] Overall configuration of the fall detection system

FIG. 1 shows the overall configuration of the fall detection system.
The fall detection system includes a fall detection device 10 including stereo cameras 11 and 12 that captures the room of the person to be observed, a monitoring device 20 connected to the fall detection device 10 via the network 40, and a network to the fall detection device 10. The mobile communication terminal 30 connected through the terminal 40 is provided.

  The fall detection system includes a fall detection device 10 including stereo cameras 11 and 12 that capture an image of the room 50 of the person to be observed, a monitoring device 20 connected to the fall detection device 10 via the network 40, and the fall detection device 10. A mobile communication terminal 30 connected via a network 40 is provided.

  The fall detection device 10 detects the occurrence of a fall of the person to be observed. Connected to the network 40 are a plurality of fall detection devices 10 provided corresponding to the rooms 50 of the plurality of persons to be observed. The monitoring device 20 is installed in a monitoring center packed with a monitor. In the monitoring device 20 and the mobile communication terminal 30, a room screen is displayed that can identify the room 50 of each person to be observed. When the fall detection device 10 detects a fall, this is the case. The monitoring device 20 and the mobile communication terminal 30 notify the mobile communication terminal 30 and generate an alarm, and also display in which room the fall is detected on the room screen. When the monitor sees this display and selects the room display in which the fall is detected, the monitoring device 20 and the mobile communication terminal 30 receive the current image of the room from the fall detection device 10 that has detected the fall. Displayed on the monitoring device 20 and the mobile communication terminal 30.

[2] Fall detection processing procedure

FIG. 2 shows the procedure of the fall detection process executed by the fall detection device 10.
In the initial setting, it is assumed that the flag F that stores the data accumulation period for falling determination is reset (F = 0).

  First, images (left and right images) are captured from the stereo cameras 11 and 12 (step S11). The time interval for capturing an image differs depending on the timing of returning to step S11, but is about 200 msec. Based on the acquired left and right images, a three-dimensional human region extraction process is performed (step S12). Thereby, three-dimensional human region information (a human region plot group in a three-dimensional space) in the camera coordinate system is obtained. Next, posture estimation processing is performed based on the obtained three-dimensional person area information (step S13). That is, the current posture (standing position, sitting position, and lying position) of the person to be observed is estimated.

  Thereafter, the data group relating to the frame read this time is stored in the first buffer (step S14). The data group includes time information, input image data, three-dimensional human region information, the width, height, depth information, and posture estimation result of the subject obtained by posture estimation processing. The first buffer is a buffer for storing a data group for several frames. When a data group for a predetermined frame is accumulated, the oldest data group for one frame is erased and a new data group is created. A data group for one frame is accumulated.

  Next, it is determined whether or not the flag F is set (step S15). When the flag F is not set, it is determined whether or not it is the start point of the fall determination data accumulation period based on the history data of the posture estimation result (step S16). Details of this determination processing will be described later. If it is determined that it is not the start point of the fall determination data accumulation period, the process returns to step S11.

  If it is determined in step S16 that it is the start point of the fall determination data accumulation period, after setting the flag F (step S17), the data group relating to the currently acquired frame is accumulated in the second buffer (step S16). S18). In this case, elapsed time information from the start point (expressed by the number of frames from the start point) is added as a data group. And it is discriminate | determined whether it is the end point of the data accumulation period for fall determination (step S19). Details of this determination processing will be described later. If it is determined that it is not the end point of the fall determination data storage period, the process returns to step S11.

  When the process returns from step S19 to S11, since F = 1 in the next step S15, the process proceeds from step S15 to step S18, and the data group is not only in the first buffer but also in the second buffer. Accumulated.

  If it is determined in step S19 that the end point of the fall determination data storage period is reached, after the flag F is reset (step S20), the fall determination process is performed based on the data group stored in the second buffer. Is performed (step S21). As a result of the fall determination process, when it is determined that it is not a fall (step S22), the process returns to step S11. As a result of the fall determination process, when it is determined that a fall has occurred (step S22), the monitoring device 20 is notified that a fall has occurred (step S23). And it returns to step S11

[3] Three-dimensional human region extraction process in step S12 of FIG.

  FIG. 3 shows a detailed procedure of the three-dimensional person region extraction process in step S12 of FIG.

  As a pre-process, an image obtained by photographing the room with one of the stereo cameras 11 and 12 is acquired in a state where no observer is present in the room 50, and the acquired image is converted into a gray scale. The obtained image is stored in the storage device of the fall detection device 10 as a background image.

  In the three-dimensional human region extraction process, the left-and-right images acquired from the stereo cameras 11 and 12 are two-dimensionally obtained by using the background difference method using the image acquired this time from the camera 11 that captured the background image and the background image. Extract the person area. Further, depth information is calculated for each pixel from two images acquired from the stereo cameras 11 and 12 by a three-dimensional surveying method, and coordinate information (three-dimensional position information) that can be plotted in a three-dimensional space is acquired. Then, by superimposing the two-dimensional person area information and the three-dimensional position information, data corresponding to the person area in the three-dimensional space (three-dimensional person area information) is extracted.

  Specifically, the image acquired this time from the camera 11 that captured the background image is converted to a gray scale (step S31).

  By calculating the absolute value of the difference between the pixel values for each corresponding pixel of the image obtained in step S31 and the background image stored in advance, a difference image between the two images is created (step S32). Two-dimensional person area information is extracted by binarizing the obtained difference image (step S33).

  On the other hand, three-dimensional position information is calculated from the left and right images acquired this time from the stereo cameras 11 and 12 using a known stereo method (step S34). Based on the two-dimensional person area information extracted in step S33 and the three-dimensional position information calculated in step S34, three-dimensional person area information is extracted (step S35).

  FIG. 4 shows an example of a three-dimensional person area image represented by the three-dimensional person area information. In FIG. 4, a rectangular parallelepiped 101 is a rectangular parallelepiped having a plane parallel to each of the XY plane, the YZ plane, and the ZX plane and circumscribing a three-dimensional person region. The upper drawing (reference numeral 102) of the rectangular parallelepiped 101 is a plan view of the three-dimensional human region viewed from above the rectangular parallelepiped 101, and the right figure (reference numeral 103) of the rectangular parallelepiped 101 is a three-dimensional view from the right of the rectangular parallelepiped 101. It is the side view which looked at the person area.

[4] Posture estimation processing in step S13 of FIG.

  FIG. 5 shows a detailed procedure of the posture estimation process in step S13 of FIG.

  Based on the three-dimensional human region information obtained by the three-dimensional human region extraction process, as shown in FIG. 4, the surface has parallel surfaces to the XY plane, the YZ plane, and the ZX plane, and 3 The width lx, height ly, and depth lz (the width, height, and depth of the person to be observed) of the rectangular parallelepiped 101 circumscribing the three-dimensional person region are calculated (step S41). lx is the absolute value of the difference between the maximum value and the minimum value of the x coordinate of the three-dimensional person area, and ly is the absolute value of the difference between the maximum value and the minimum value of the y coordinate of the three-dimensional person area Lz can be obtained by calculating the absolute value of the difference between the maximum value and the minimum value of the z coordinate of the three-dimensional human region.

  Next, the aspect ratios lx / ly and lz / ly are calculated (step S42). Then, based on the calculated aspect ratios lx / ly and lz / ly and a predetermined rule, the posture of the person to be observed is estimated (step S43).

Specifically, the posture of the person to be observed is estimated based on the following rules.
(A) If lx / ly <0.4 or lz / ly <0.4, the posture of the observer is estimated as “standing”.
(B) If lx / ly> 1.5 or lz / ly> 1.5, the observer's posture is estimated to be “recumbent”.
(C) Otherwise, the observer's posture is estimated as “sitting”.

[5] Start point or end point determination processing in step S16 and step S19 in FIG.

  In the start point determination process in step S16 in FIG. 2, “a state where at least one of the aspect ratios lx / ly and lz / ly is 0.7 or more continues for 1 second (corresponding to about 5 frames) or more. It is determined whether or not the start point condition “is present” is satisfied. If the start point condition is satisfied, it is determined that the currently captured frame is the start point of the fall determination data storage period.

  In the end point determination process in step S19 of FIG. 2, ““ at least one of the aspect ratios lx / ly and lz / ly is 0.7 or less is 1.4 seconds (corresponding to about 7 frames) or more. "Continuing" or "The data group stored in the second buffer is equal to or greater than the predetermined number of frames" or "Elapsed time from the start point has reached the predetermined time or more" It is determined whether or not the end point condition is satisfied. If the end point condition is satisfied, it is determined that the frame captured this time is the end point of the fall determination data storage period.

[6] Falling determination process in step S21 of FIG.

  Prepare models in advance for each type of behavior. In this example, the types of behavior include “forward-falling”, “shiri-mochi”, “lie down”, and “sit down”. Here, a hidden Markov model (HMM) is used as the model. A model for each behavior is created based on learning data corresponding to the behavior. For example, “forward fall” is created based on the learning data of the “forward fall” operation. As the learning data, time-series data including elapsed time information (expressed by the number of frames from the start point) and the width, height, and depth information (lx, ly, lz) of the observer is used. Among the behaviors of “forward falling”, “shirimochi”, “lie down” and “sit”, “forward falling” and “shirimochi” correspond to the overturning motion.

  Of the data group stored in the second buffer, the elapsed time information (expressed by the number of frames from the start point) and the width, height, and depth information (lx, ly, lz) of the observer are time-series. The probability (likelihood) that the time series data can be reproduced as data is calculated for each model. Then, the model with the highest likelihood is obtained. The behavior corresponding to the model with the highest likelihood is the behavior of the observer. In this embodiment, since it is desired to detect a fall, if the model with the highest likelihood is a “forward fall” or “shirimochi” model, it is determined that the person to be observed has fallen.

[7] Modification of posture estimation processing

  In the above embodiment, the posture of the observer is estimated based on the aspect ratios lx / ly and lz / ly obtained from the width, height, and depth of the observer. The posture of the person to be observed may be estimated based on the direction of a vector (person main axis) corresponding to the direction.

  That is, as shown in FIG. 6, the angle γ (the direction of the person main axis) formed by the XZ plane of the person main axis Q is obtained, and the posture of the person to be observed is estimated based on the following rules.

(A) If γ> π / 2 × 0.6, the observer's posture is estimated as “standing”.
(B) If γ <π / 2 × 0.3, the observer's posture is estimated to be “prone”.
(C) Otherwise, the observer's posture is estimated as “sitting”.

  FIG. 7 shows an example of the orientation of the person main axis Q and the posture estimation result, FIG. 7A shows an example in which “standing” is estimated, and FIG. 7B shows an example in which “sitting” is estimated. FIG. 7C shows an example in which it is estimated that the user is in the “recumbent position”.

  Hereinafter, a method for obtaining the orientation of the person principal axis will be described. There are a first method and a second method for obtaining the orientation of the person main axis.

  First, the first method will be described. The first method is a method for obtaining the orientation of the person principal axis using principal component analysis.

  FIG. 8 shows the procedure of the method of calculating the orientation of the person principal axis (first method).

  The first projection of the UV coordinate system in which the X axis is the U axis and the Y axis is the V axis by projecting the 3D human area information (a plot group of the human area in the 3D space) onto the XY plane. An image is obtained, and the 3D human region information is projected onto the YZ plane, thereby obtaining a second projected image in the UV coordinate system having the Z axis as the U axis and the Y axis as the V axis (step S51). . Next, the center of gravity is calculated for each projected image (step S52). Then, all coordinate values of each projected image are corrected to coordinate values having the center of gravity as the origin (step S53).

  Next, an eigenvector (a vector representing the inclination of the first principal component) is calculated from the first projection image after the coordinate value correction using the principal component analysis, and the principal component is calculated from the second projection image after the coordinate value correction. An eigenvector (a vector representing the inclination of the first principal component) is calculated using analysis (step S54). Then, a vector indicating the direction of the person principal axis is obtained by combining the two eigenvectors calculated in step S54 (step S55). Then, the orientation γ of the person principal axis is obtained from the vector indicating the orientation of the person principal axis (step S56).

A method for calculating the eigenvector will be described. Since the calculation method of each eigenvector is the same, the calculation method of the first eigenvector will be described. First, based on the first projected image after the coordinate value correction, the variance _{u of} the variable _u , the variance s _v of the variable _v, and the covariance s _uv of the variables u and v are calculated. Then, the eigenvalue λ is calculated using the variances s _u and s _v and the covariance s _uv . Since the method for calculating the eigenvalue λ using the variances s _u and s _v and the covariance s _uv is well known in principal component analysis, its details are omitted. Next, an eigenvector is calculated using the eigenvalue λ. Since the method for calculating the eigenvector using the eigenvalue λ is well known in the principal component analysis, its details are omitted.

  FIG. 9 shows the procedure of the method for calculating the orientation of the person main axis (second method).

  By projecting the three-dimensional human region information (a plot group of the human region in the three-dimensional space) onto the XY plane, a first projected image is obtained and the three-dimensional human region information is projected onto the YZ plane. Thus, a second projected image is obtained (step S61). Next, the center of gravity is calculated for each projected image (step S62). Then, all coordinate values of each projected image are corrected to coordinate values having the center of gravity as the origin (step S63).

  Next, from the first projected image after the coordinate value correction, among the line segments extending through the origin and extending to the contour of the person area, the inclination of the first line segment having the maximum length is obtained, From the second projected image after the coordinate value correction, the inclination of the second line segment having the maximum length among the line segments passing through the origin and extending to the contour of the person area is obtained (step S64). Then, based on the inclinations of the two line segments obtained in step S64, the orientation γ of the person principal axis is obtained (step S65).

  As shown in FIG. 10, when the slope of the first line segment obtained from the first projection image is α and the slope of the second line segment obtained from the second projection image is β, each line segment is The unit vectors v1 and v2 indicating the direction are expressed by the following equation (1).

v1 = (cos α, sin α)
v2 = (cosβ, sinβ) (1)

  Based on the following equation (2), when v1 and v2 are combined, a vector V indicating the orientation of the person principal axis is obtained.

  V = (cos α, sin α + sin β, cos β) (2)

  The orientation γ of the person principal axis is obtained based on the following equation (3).

tan γ = (sin α + sin β) / (cos α ² + cos β ² ) ^1/2 (3)

  A method for obtaining the first line segment and the second line segment will be described. Since the method for obtaining each line segment is the same, the method for obtaining the first line segment will be described.

  FIG. 11 shows how to obtain the first line segment.

  First, as shown in FIG. 12, the projected image is divided into a plurality of regions in the X-axis direction by dividing the first projected image obtained in step S3 by a plurality of dividing lines parallel to the Y-axis having a predetermined interval. (Step S71). A point corresponding to the maximum value and the minimum value of the y coordinate of the person area is specified as the contour point Pi for each divided area (step S72).

  Of the contour points, the contour point having the largest x coordinate and the y coordinate closest to 0 is defined as a point A, and a divided region including the point A is defined as a region of interest (step S73). In the example of FIG. 12, the point P1 is specified as the point A. Among contour points other than point A, a contour point closest to the straight line connecting point A and the origin is obtained and set as point B (step S74). In the example of FIG. 12, when the point A is the point P1, the point P7 is the point B. The distance between the line connecting point A and point B (distance between A and B) is calculated and held (step S75).

  Next, it is determined whether or not the attention area is a divided area (final processing area) including an outline point having the smallest x coordinate among the outline points (step S76). If the current attention area is not the final processing area, the attention area is updated to a divided area adjacent to the current attention area in the direction in which the x-coordinate becomes smaller (step S77). Then, in the updated attention area, the contour point having the largest y coordinate is set as a point A (step S78). And it returns to step S74 and performs the process after step S74 again.

  If it is determined in step S76 that the current region of interest is the final processing region, the line segment having the maximum distance between A and B is specified as the first line segment (step S79). In the example of FIG. 12, the line segment connecting the point P3 and the point P9 is specified as the first line segment.

[8] Modification of the fall determination process in step S21 of FIG.

  A learning model is prepared in advance for each type of behavior. In this example, the types of behavior include “forward-falling”, “shiri-mochi”, “lie down”, and “sit down”. A learning model for each behavior is created based on learning data corresponding to the behavior. For example, “forward fall” is created based on the learning data of the “forward fall” operation. As the learning data, time series data including time information and the width, height, and depth information (lx, ly, lz) of the observer is used. Based on the learning data, a learning model is created that includes the degree of movement (many, medium, and small) of the observer at a plurality of points in time. Among the behaviors of “forward falling”, “shirimochi”, “lie down” and “sit”, “forward falling” and “shirimochi” correspond to the overturning motion.

  FIG. 13 shows the procedure of the fall determination process. FIG. 14 is a schematic diagram showing the procedure of the fall determination process.

  First, based on the width, height, depth information (lx, ly, lz) and time information (shown as time-series data 201 in FIG. 14) of a plurality of frames stored in the second buffer, Motion amount data (dlx / dt, dly / dt, dlz / dt) per unit time at a plurality of time points is obtained (step S111).

  By dividing the differences dlx, lyy, and dlz of lx, ly, and lz between adjacent frames in the time series data 201 by the time difference between the frames, motion amount data per unit time at a plurality of time points is obtained. It is done.

  Next, based on the maximum value of the three types of motion amount data at each time point and two preset threshold values, the motion amount at each time point is symbolized into three types (many, medium, and small). (Step S112). As a result, a pattern 202 composed of symbolized data at a plurality of time points as shown in FIG. 14 is obtained.

  The distance (similarity) between the obtained pattern 202 and the learning model for each behavior (shown as 211 to 214 in FIG. 14) is calculated by DP matching (step S113). Then, the behavior corresponding to the learning model having the shortest distance between both patterns is determined as the behavior of the observer (step S114). In this embodiment, since it is desired to detect a fall, if the learning model with the shortest distance between the two patterns is a “forward-facing fall” or “Sirimochi” learning model, it is determined that the subject has fallen. .

[9] Utilization of detection history

  The monitoring device 20 (see FIG. 1) may evaluate and rank the fall risk for each person to be observed from the number of fall detections to date, the occurrence interval of the fall detection, and the like. This ranking may be set freely by the supervisor. Further, the type of alarm output when a fall is detected may be changed according to the height of the fall risk. In addition, in the case where a plurality of observers are detected to fall at the same time, the monitoring device 20 can detect the occurrence of a fall and the subject to be able to check the situation with priority given to a high-risk observer. It is preferable to display both the observer's fall risk.

  In the above embodiment, each fall detection device 10 performs processing related to behavior detection such as a fall, but it may be performed collectively on the monitoring device 20 side. However, if it does in this way, processing load may concentrate on the monitoring apparatus 20, and the monitoring apparatus 20 may go down. From this point of view, it is preferable that each fall detection device 10 performs processing related to behavior detection such as fall. In preparation for the occurrence of a power failure or the like, it is preferable to install a lamp or an alarm that falls at the entrance of each room when a fall is detected, and to have a battery built in each fall detection device 10. Further, it is preferable that a remote family member or the like can be notified directly from the behavior detection device 10 without using the monitoring device 20.

  In the above-described embodiment, the fall detection device 10 includes an imaging device such as a stereo camera in order to extract 3D human region information. However, the fall detection device 10 is not limited to this, and the fall detection device 10 has a 3D human region. In order to extract information, for example, a combination of a sensor including an imaging device such as a combination of a monocular imaging device and a laser range finder, a combination of a monocular imaging device and a plurality of ultrasonic sensors may be provided. Good.

It is a block diagram which shows the whole structure of a fall detection system. 4 is a flowchart illustrating a procedure of a fall detection process executed by the fall detection device 10. It is a flowchart which shows the detailed procedure of the three-dimensional person area extraction process of step S12 of FIG. It is a schematic diagram which shows an example of the three-dimensional person area image corresponding to three-dimensional person area information. It is a flowchart which shows the detailed procedure of the attitude | position estimation process of step S13 of FIG. It is a schematic diagram which shows angle | corner (γ) which makes the person main axis Q and the XZ plane of the person main axis Q. It is a schematic diagram which shows the direction of the person main axis | shaft Q and the example of an attitude | position estimation result. It is a flowchart which shows the procedure of the calculation method (1st method) of a person principal axis. It is a flowchart which shows the procedure of the calculation method (2nd method) of a person principal axis. It is a schematic diagram which shows the inclination (alpha) of the 1st line segment obtained from the 1st projection image, and the inclination (beta) of the 2nd line segment obtained from the 2nd projection image. It is a flowchart which shows how to obtain | require a 1st line segment. It is a schematic diagram for demonstrating how to obtain | require a 1st line segment. It is a flowchart which shows the modification of the procedure of the fall determination process of step S21 of FIG. It is a schematic diagram for demonstrating the procedure of the fall determination process of FIG.

Explanation of symbols

10 fall detection device 11, 12 stereo camera 20 monitoring device 30 mobile communication terminal 40 network

Claims (7)

Person area information extracting means for extracting three-dimensional person area information based on a photographed image of the person to be observed;
Based on the calculation means for calculating the height, width and depth information of the 3D person area from the 3D person area information, and the ratio of the width to the height of the 3D person area and the ratio of the depth to the height of the 3D person area Determining means for determining the posture of the person to be observed;
An attitude detection device comprising: Person area information extracting means for extracting three-dimensional person area information based on a photographed image of the person to be observed;
Calculating means for calculating the orientation of the back muscle of the subject from the three-dimensional person area information, and determination means for determining the posture of the subject based on the orientation of the back muscle of the observer,
An attitude detection device comprising: The calculation means is
The first projected image is generated by projecting the 3D human area information onto the XY plane with the Y axis as the height direction of the 3D human area, the X axis as the width direction, and the Z axis as the depth direction. And means for generating a second projected image by projecting the three-dimensional human region information onto the YZ plane,
Means for calculating the center of gravity for each projected image, and correcting all coordinate values of each projected image to coordinate values having the center of gravity as the origin;
A first vector representing the inclination of the first principal component is calculated from the first projection image after the coordinate value correction using the principal component analysis, and the principal component analysis is used from the second projection image after the coordinate value correction. Means for calculating a second vector representing the inclination of the first principal component, and by combining the first vector and the second vector, a vector indicating the orientation of the back muscle of the observer is obtained, and the obtained vector From which the direction of the back muscle of the person to be observed is determined,
The posture detection apparatus according to claim 2, comprising: The calculation means is
The first projected image is generated by projecting the 3D human area information onto the XY plane with the Y axis as the height direction of the 3D human area, the X axis as the width direction, and the Z axis as the depth direction. And means for generating a second projected image by projecting the three-dimensional human region information onto the YZ plane,
Means for calculating the center of gravity for each projected image, and correcting all coordinate values of each projected image to coordinate values having the center of gravity as the origin;
From the first projected image after the coordinate value correction, the inclination of the first line segment having the maximum length among the line segments passing through the origin and extending to the contour of the person area is obtained, and the coordinate value correction is performed. Means for obtaining the slope of the second line segment having the maximum length among the line segments that pass through the origin and extend to the contour of the person area from the second projected image later; and the first line segment Means for determining the direction of the spine of the person to be observed based on the inclination of the second line segment and the inclination of the second line segment;
The posture detection apparatus according to claim 2, comprising: Person area information extracting means for extracting three-dimensional person area information based on a photographed image of the person to be observed;
Posture determination means for determining the posture of the person to be observed based on the three-dimensional person area information;
Behavior determination data generating means for generating behavior determination data for determining the behavior of the detection target from the three-dimensional person area information;
Based on the determination result of the posture determination means, it is obtained from the storage means for accumulating behavior determination data during the period when the behavior of the detection target may have occurred, and the behavior determination data accumulated by the storage means. Determining means for determining whether or not the behavior of the detection target has occurred by comparing the time series data and a model created in advance from learning data corresponding to the behavior of the detection target,
A behavior detection device comprising: The behavior determination data generation means generates behavior determination data including the height, width, and depth of the three-dimensional person area from the three-dimensional person area information. The time-series data is stored by the storage means. 6. The behavior detection apparatus according to claim 5, wherein the behavior detection apparatus is time series data of height, width, and depth of a three-dimensional person region, and the model is an HMM model. The behavior determination data generation means generates behavior determination data including the height, width, and depth of the three-dimensional person area from the three-dimensional person area information. The time-series data is stored by the storage means. Time-series data of data obtained by symbolizing the magnitude of the amount of movement of the observer calculated from the height, width, and depth of the three-dimensional person region, and the model is a model for DP matching The behavior detection apparatus according to claim 5. .

Images