JP2015079339A - Posture estimation device - Google Patents

Abstract

Provided is a posture estimation device that generates a posture evaluation formula using a plurality of feature amounts and estimates a posture using the posture evaluation formula when estimating a posture of a person. Model image generation for generating a first model image simulating a person in a first posture photographed by an imaging unit and a second model image simulating a person in a second posture photographed by an imaging unit A feature amount calculating means for calculating a plurality of types of feature amounts from the image, and a weight for posture evaluation according to a difference between the feature amount of the first model image and the feature amount of the second model image for each feature amount A posture evaluation formula generating means for generating a posture evaluation formula and a human region extraction means for extracting a human area image from the input image, and estimating a posture from the feature amount calculated for the human area image based on the posture evaluation formula Posture determination means for [Selection] Figure 1

Description

  The present invention relates to an apparatus for estimating a posture of a person based on a feature amount calculated from an image of the person.

  An apparatus for estimating the posture of a person from an image of the person has been proposed. For the estimation of the posture, a region of a person shown in the input image is obtained, and various features are extracted from the region and threshold processing is performed.

  For example, a technique is disclosed that uses, as a feature amount, an angle formed by a long axis direction of a human region in an input image and a horizontal axis (X axis) of the input image (Patent Document 1).

  In addition, an apparatus is disclosed that uses the area of a person region obtained by background difference as a feature amount and determines that a person is lying on the floor surface when the area is equal to or greater than a threshold value (Patent Document 2).

JP 2002-197463 A JP 2000-207664 A

  FIGS. 11A to 11C show a diagram (right diagram) simulating an actual space and an image (left diagram) taken by the camera 120. FIG. FIGS. 11A to 11C are images obtained by photographing a person from obliquely above with the camera 120 installed in the upper left.

  In FIG. 11A, the standing person 121 is captured by the camera 120. FIG. 11B is a diagram in which a person 122 lying in a direction along the vertical plane 101 including the person and the camera 120 is photographed by the camera 120. FIG. 11C is a diagram in which a person 123 lying in a direction perpendicular to the vertical plane 101 including the person and the camera 120 is photographed by the camera 120.

  In Patent Document 1, the angle formed by the long axis direction (direction connecting the foot and the head) of the human region in the image and the horizontal axis (X axis) of the image is calculated as the main axis angle θ. The posture of the person is judged as close to 0 ° if close to the side.

  For this reason, the standing human region 103 in FIG. 11A and the lying human region 105 in FIG. 11C are the angle formed by the long axis direction of the human region and the horizontal axis (X axis) of the image. It is easy to determine whether the main axis angle θ is substantially 90 degrees or 0 degrees and whether it is standing or lying. However, the standing person area 103 in FIG. 11A and the lying person area 104 in FIG. 11B are the angle formed between the long axis direction of the person area and the horizontal axis (X axis) of the image. Are both in the vicinity of 90 °, making discrimination difficult. Therefore, with the method of Patent Document 1, it is difficult to discriminate between standing and lying.

  In Patent Literature 2, it is detected whether a person has fallen from the area of the person region. Since the area of the standing person area 103 in FIG. 11 (a) and the lying person area 104 in FIG. 11 (b) vary greatly as apparent from the drawing, the distinction between standing and lying is made. Is possible. However, since the area of the standing person area 103 in FIG. 11A and the lying person area 105 in FIG. 11C are substantially the same area, it cannot be determined whether they are standing or lying. Therefore, with the method of Patent Document 2, it is difficult to discriminate between standing and lying.

  In this way, when the posture of a person is estimated using only one feature amount, it is difficult to estimate with high accuracy. Furthermore, even if a plurality of feature quantities are used, it is difficult to estimate with high accuracy simply by using them together.

  In view of the above, an object of the present invention is to provide a posture estimation apparatus that uses a plurality of feature amounts to estimate a posture of a person and weights the feature amounts so that a correct posture is estimated to determine the posture. .

  An attitude estimation apparatus according to the present invention is an attitude estimation apparatus that calculates a plurality of types of feature amounts from a person region image in an input image captured by an imaging unit installed at a predetermined position and estimates a person's attitude. A model image generating means for generating a first model image imitating a person in a first posture photographed by an imaging unit, a second model image imitating a person in a second posture photographed by an imaging unit, and an image A feature amount calculation means for calculating a plurality of types of feature amounts from the feature amount, and for each feature amount, a weight for posture evaluation is determined according to a difference between the feature amount of the first model image and the feature amount of the second model image Attitude evaluation expression generating means for generating an attitude evaluation expression, human area extraction means for extracting a person area image from an input image, and attitude determination means for estimating an attitude based on an attitude evaluation expression from a feature amount calculated for the person area image It is characterized by having .

  Since the posture estimation apparatus according to the present invention generates and uses a posture evaluation formula by weighting according to a difference in feature amount due to a difference in posture, the posture can be estimated with high accuracy.

  Further, the model image generation means enlarges or reduces a predetermined person model stored in advance so as to match the physique of the person estimated from the person region image in the imaging space of the imaging unit, and the person model is stored in the imaging unit. It is preferable to generate the first model image and the second model image on the assumption that the images have been taken.

  Thereby, since the weight is determined by reflecting the physique of the person, the posture of the person can be estimated with higher accuracy.

  Further, the model image generation means estimates the person position in the imaging space of the imaging unit from the person area image, places a predetermined person model stored in advance at the position, and assumes that the imaging unit has captured the person model. It is preferable to generate one model image and a second model image.

  As a result, the weight is determined by reflecting the difference in how the image is taken depending on the position of the person in the shooting space, so that the posture of the person can be estimated with higher accuracy.

  Further, the predetermined person model is a single-shaped person model, and the model image generation means estimates a person's foot position in the imaging space of the imaging unit from the person area image, and matches the foot position to a standing posture. And a recumbent posture model, a first model image is generated on the assumption that the imaging unit has captured a human model in a standing posture, and a second model on the assumption that the imaging unit has captured a human model in a recumbent posture. It is preferable to generate a model image.

  As a result, when the first posture is the standing posture and the second posture is the recumbent posture, a weighted posture evaluation formula is generated that reflects the difference in the feature amount between the standing posture and the recumbent posture. Therefore, the standing posture or the lying posture can be estimated with high accuracy.

  Further, the predetermined person model is preferably a person model having a first posture and a person model having a second posture having a shape different from that of the person model having the first posture.

  Thereby, since weighting is performed according to the difference in feature amount due to the difference in shape, it is possible to accurately estimate the posture in which the difference in shape of the person occurs.

  According to the present invention, it is possible to provide a posture estimation apparatus that generates a posture evaluation formula using a plurality of feature amounts and estimates a posture using the posture evaluation formula when estimating the posture of a person.

It is a block diagram which shows the structure of the recumbent judgment apparatus in embodiment of this invention. It is a figure for demonstrating the calculation method of the feature-value in embodiment of this invention. It is a figure which illustrates the relationship between the actual position in the person area of the input image in the embodiment of this invention, and the actual position in a monitoring area | region, and a magnitude | size relationship. It is a figure which illustrates the relationship of the position when lying down in the person area | region and monitoring area | region of the input image in embodiment of this invention. It is a figure which illustrates the state assumed that the standing-up model and recumbent model in embodiment of this invention have been arrange | positioned in the monitoring area | region. It is a figure which illustrates the standing model image and recumbent model image in embodiment of this invention. It is a figure which shows the main flowchart of the recumbency estimation method in embodiment of this invention. It is a figure which shows the subflowchart of the calculation method of the weight in embodiment of this invention. It is a figure for demonstrating the calculation method of the feature-value in the modification 1 of this invention. It is a figure for demonstrating the calculation method of the feature-value in the modification 2 of this invention. It is a figure for demonstrating a prior art.

A mode in which the posture estimation device of the present invention is applied to a recumbent determination device that identifies a standing posture and a recumbent posture will be described.
As shown in FIG. 1, the recumbent determination device 1 according to the embodiment of the present invention includes an imaging unit 10, a storage unit 20, an image processing unit 30, and an output unit 40. The recumbent determination device 1 identifies and determines a standing posture and a recumbent posture of a person captured in an image captured by the image capturing unit 10.

  In the present embodiment, each unit will be described as the lying down determination device 1, but the imaging unit 10, the storage unit 20, the image processing unit 30, and the output unit 40 are each a separate case, making full use of communication technology. Necessary control signals or the like may be communicated. Further, these may be appropriately combined and incorporated into the housing.

The imaging unit 10 outputs an image obtained by imaging the imaging space to the image processing unit 30 as an input image. The imaging unit 10 includes an optical system, an imaging element such as a CCD element or a C-MOS element, optical system parts, an analog / digital converter, and the like. These image sensors may be of a visible light type that converts brightness into an electrical signal, or may be a thermal image type that converts temperature into an electrical signal in response to infrared rays. Photographing is performed at predetermined time intervals.
The input image is temporarily stored in the storage unit 20 via the image processing unit 30 every time it is taken from the imaging unit 10.

  In the present embodiment, the imaging unit 10 is installed above the floor of a place (monitoring area) where the posture of a person is to be estimated, for example, on the ceiling. The imaging unit 10 may be attached to a support. It is assumed that the imaging unit 10 is installed with the imaging direction directed obliquely downward.

  The storage unit 20 includes a semiconductor device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and a memory device such as a hard disk. The storage unit 20 is accessible from the image processing unit 30, and stores camera information 21, tracking information 22, and a person model 23, as well as data such as a program (not shown), an input image, a background image, and a threshold value used for determination processing. To do.

  The camera information 21 is information used to calculate the relationship between the area in the input image and the actual position of the monitoring area by a geometric method. The camera information 21 includes various parameters such as information on the angle of view of the imaging unit 10, the height from the floor, and the depression angle.

  The tracking information 22 is the current time when the imaging unit 10 acquires the input image, the position information (center of gravity position coordinates) of the person area image extracted from the input image, the number of tracking times that is the number of frames in which the person area image is tracked, and will be described later. This is information in which physique information (height and width) estimated in the physique estimation process is associated. The position information of the person area image is used for tracking processing when there are a plurality of persons in the input image and for estimating the physique of each person. The tracking number is set to 1 as an initial value when it is determined that a new appearance appears by the tracking means described later.

The person model 23 is information that represents a three-dimensional shape of a person to be photographed using a polygon model, a surface model, or the like, and connects at least a standard human height (170 cm), width (60 cm), and the top and feet. This is a model that expresses the axis (main axis). The person model 23 is enlarged or reduced in accordance with the physique estimated in the physique estimation process described later. In the present embodiment, it is assumed that the “careful” posture is stored as the person model 23.
The background image stored in advance is an image when no person is present in the imaging range of the input image.

  The image processing unit 30 is realized by an arithmetic device (computer) including a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The image processing unit 30 functions as each unit described below by reading and executing various programs stored in the storage unit 20.

  The image processing unit 30 sequentially processes input images taken by the imaging unit 10. Note that the image to be processed may be processed in order of photographing time for each frame, or may be processed every several frames.

  The image processing unit 30 includes modules of a person area extraction unit 301, a tracking unit 302, a feature amount calculation unit 303, a posture evaluation formula generation unit 304, an evaluation value calculation unit 311, and a posture determination unit 312.

  The person area extraction unit 301 is a unit that extracts a change area generated in the input image temporarily stored in the storage unit 20 and cuts out the inside thereof. Specifically, a luminance difference between corresponding pixels in the input image and the background image stored in the storage unit 20 is obtained, and a pixel whose difference value is equal to or greater than a luminance threshold is extracted as a change region. Note that the change region extraction method is not limited to this, and a method may be employed in which a region extracted by inter-frame difference processing of temporally adjacent input images is extracted as a change region.

The person area extraction unit 301 performs a labeling process on the areas that are considered to be the same object for the pixel group specified as the change area. A known process may be applied to the labeling process. The change area is usually extracted in various sizes, for example, due to illumination fluctuations or noise superimposition, but with reference to the resolution of the imaging unit 10 or the camera information 21, the change area has a size that satisfies human characteristics, It is assumed that a person is reflected in a change area having an aspect ratio and is to be processed later. The change area to be processed is hereinafter referred to as a “person area”, and a partial image obtained by cutting out the interior from the input image is hereinafter referred to as a “person area image”.
Note that the method of extracting a person area image is not limited to this, and is similar in a method of extracting a person area image by a learning discriminator that has learned a person image that is a detection target, and a matching process with a person template. Various methods such as a method of extracting a region as a human region image may be employed.

  In addition, the person area extraction unit 301 obtains the position of the center of gravity in the input image of the person area image and outputs it to the tracking unit 302 for tracking processing.

The tracking unit 302 refers to the position of the person area image at the previous time stored as the tracking information 22 in the storage unit 20, and uses the same person as the person area image at the current time within a predetermined distance from the position. The tracking process is performed by regarding them as associations. Further, the barycentric position of the person area image at the current time associated with the person area image at the previous time is stored as tracking information 22 in the storage unit 20 in association with the current time. When the person area image at the current time corresponds to the person area image at the previous time, the tracking number of the tracking information 22 is incremented by one.
If there is no person area image at the previous time stored in the storage unit 20 corresponding to the person area image at the current time, it is assumed that a new person starts to be imaged by the imaging unit 10 at the current time, and the next time Targeted for tracking.
Conversely, if the person area image stored at the previous time is stored in the storage unit 20 and there is no corresponding person area image at the current time, the person who has been captured by the imaging unit 10 is captured in the input image. Since it is no longer present, it is removed from the subsequent processing target and erased from the storage unit 20.
Since these tracking processes may apply a known method, the details are omitted.

  Note that the tracking process by the tracking unit 302 is not necessary when it is assumed that only one person is present in the monitoring area, or when a change in the feature amount due to a change in time is not used.

  The feature amount calculation unit 303 obtains a physique feature amount and a recumbent determination feature amount from the person region image extracted by the person region extraction unit 301. Further, a standing model image and a recumbent model image, which will be described later, are acquired from the posture evaluation formula generation unit 304, and values of a standing model feature and a recumbent model feature for determining weights described later are obtained. The physique feature amount, the recumbent determination feature amount, the standing model feature amount, and the recumbent model feature amount are calculated for each extracted person region image.

With reference to FIG. 2, processing in the feature amount calculation unit 303 will be described.
FIG. 2A shows a circumscribed rectangle 202 of the person area image 201 including the person 200 along the vertical axis (y axis) and the horizontal axis (x axis) in the input image. The circumscribed rectangle 202 has a lateral width p and a height q.
FIG. 2B shows a circumscribed rectangle 203 in which the direction of the long side coincides with the direction of the main axis 204 of the person region image 201. The circumscribed rectangle 203 has a horizontal width w and a height h.

The feature quantity calculation means 303 obtains the following two as physique feature quantities for estimating the physique of a person.
(A) The height q of the circumscribed rectangle 202 corresponding to the height of the person
(B) The width p of the circumscribed rectangle 202 corresponding to the width of the person
Then, the obtained physique feature amount is output to the posture evaluation formula generation unit 304.

In addition, the feature amount calculating unit 303 calculates the following five values as the feature amount for recumbency determination to be used for recumbent detection.
(1) circumscribed rectangle area fj1 = lateral width p × height q of circumscribed rectangle 202
(2) circumscribed rectangle aspect ratio fj2 = width p / height q of circumscribed rectangle 202
(3) circumscribed rectangle height fj3 = height q of circumscribed rectangle 202
(4) Aspect ratio fj4 in the principal axis direction = width w and height h of the circumscribed rectangle 203 (shorter / longer)
(5) Main axis direction long side length fj5 = the longer one of the lateral width w and the height h of the circumscribed rectangle 203. Then, the obtained feature value for recumbent judgment is output to the evaluation value calculation means 311.

Further, as will be described later, the feature amount calculation unit 303 uses a circumscribed rectangle of the standing model image generated by the model image generation unit 313 of the posture evaluation formula generation unit 304, as in (1) to (5) above. The standing model feature amount (circumscribed rectangle area fs1, circumscribed rectangle aspect ratio fs2, circumscribed rectangle height fs3, main axis aspect ratio fs4, main axis direction long side length fs5) is calculated.
Then, the obtained standing model feature amount is output to the posture evaluation formula generation unit 304.

In addition, as described later, the feature amount calculation unit 303 uses the circumscribed rectangle of the recumbent model image generated by the model image generation unit 313 of the posture evaluation formula generation unit 304, as described in (1) to (5) above. A model feature amount (circumscribed rectangle area ft1, circumscribed rectangle aspect ratio ft2, circumscribed rectangle height ft3, main axis aspect ratio ft4, main axis direction long side length ft5) is calculated.
Then, the obtained recumbent model feature amount is output to the posture evaluation formula generation unit 304.

  The above (1) to (5) are feature amounts related to the shape of the image. In addition to this, the feature amount calculation means 303 is a feature based on pixel values of pixels included therein, such as an average luminance value and texture information. An amount may be calculated and used.

The posture evaluation formula generation unit 304 generates a posture evaluation formula for calculating a determination evaluation value used to identify whether the person shown in the person area image is standing or lying.
The posture evaluation formula generation unit 304 obtains physique information (height and width) when the person area image is tracked for 10 frames, and each feature amount calculated by the feature amount calculation unit 303 is stored in the storage unit 20. A posture evaluation formula is generated with reference to the model 23 and the like.
The posture evaluation formula generation unit 304 includes a model image generation unit 313, a weight calculation process 309, and an evaluation formula determination process 310. Hereinafter, each processing will be described.

The model image generating means 313 is a recumbent model when it is assumed that the posture of the person having the same physique as the person shown in the person area image is recumbent with the standing model image when recognizing that the person is standing. An image is generated and output to the weight calculation process 309.
The model image generation unit 313 includes a 2D / 3D conversion process 305, a physique estimation process 306, a 3D model generation process 307, and a model region generation process 308.

The two-dimensional / three-dimensional conversion process 305 is a process for mutually converting a two-dimensional coordinate value in the input image and a three-dimensional position representing the monitoring area in the world coordinate system.
The conversion process is realized by a known geometric method based on information such as the angle of view, the installation height, and the depression angle of the imaging unit 10 stored as camera information 21 in the storage unit 20.
The position of the person existing in the monitoring area can be calculated by the 2D / 3D conversion process 305 based on the coordinate information of the person area image in the input image, for example. An example of this will be described with reference to FIG.

  FIG. 3A shows a state in which a person area image 201 generated by the person 200 is extracted from the input image. The two-dimensional / three-dimensional conversion processing 305 regards the lowest point 205 of the person region image 201 as the foot position (person position) of the person 200, and converts the coordinate value of the lowest point 205 in the input image to that shown in FIG. ), The coordinate values in the world coordinate system separately defined in the monitoring area are converted. FIG. 3B shows a person 206 actually standing in the monitoring area, and the actual foot position 207 in the monitoring area corresponds to the lowest point 205 of the person area image 201.

  The physique estimation process 306 estimates the physique of the person shown in the person area image. The physique is the height and width, and the estimated height and width are referred to as “estimated height” and “estimated width”.

In the physique estimation process 306, the height and width of the person area image 201 shown in FIG. 3A, that is, the height q and the width p of the circumscribed rectangle 202 of the person area image 201 are converted into two-dimensional / three-dimensional conversion processes 305. As shown in FIG. 3B, the estimated height q ′ and the estimated width p ′ of the person 206 actually standing at the foot position 207 in the monitoring area are obtained.
The obtained estimated height and estimated width values are stored in the tracking information 22 of the storage unit 20 in association with the information of the person area image at the current time.

In the present embodiment, it is assumed that the predetermined number of frames after the person begins to appear in the image is a standing person who is standing and walking. The physique estimation process 306 calculates an estimated height and estimated width for a predetermined number of frames after the person starts to appear in the input image, stores the average value in the tracking information 22 as physique information, and then further estimates the estimated height and estimated width. The calculation process is omitted. The predetermined number of frames is an appropriate number of frames for improving the accuracy when estimating the physique of the person from the image, and is determined by the installation conditions of the recumbent determination device 1. In this embodiment, 10 frames are used.
The physique estimation process 306 may use the estimated height and the estimated width calculated for each frame as they are, instead of using the average value for 10 frames.

  The three-dimensional model generation process 307 uses the estimated height and estimated width calculated in the physique estimation process 306 to read and deform the person model 23 in the storage unit 20 to generate a standing model and a recumbent model.

First, the three-dimensional model generation processing 307 first determines a floor body direction for the person area image extracted by the person area extraction unit 301. This will be described with reference to FIG. The floor body direction is the body direction when it is assumed that the person shown in the person area image is lying on the floor.
As shown in FIG. 4A, the three-dimensional model generation processing 307 obtains the main axis 350 of the person region image 201, and as two points on the screen on the main axis 350, the end point A that is the top and the end point that is the foot. Find the coordinates of B. Then, as shown in FIG. 4B, the three-dimensional model generation process 307 converts the end points A and B into points A ′ and B ′ on the floor surface of the monitoring area by the two-dimensional / three-dimensional conversion process 305. Calculated as the coordinates (world coordinates).

  The three-dimensional model generation processing 307 includes an intersection between the visual axis 351 of the imaging unit 10 and the floor surface, a v-axis passing through a floor surface position below the imaging unit 10, and an intersection between the visual axis 351 of the imaging unit 10 and the floor surface. 4 is considered as the origin, and the u-axis orthogonal to the v-axis is considered. As shown in FIG. 4C, the angle formed by the straight line 352 connecting the points A ′ and B ′ with the u-axis on the floor surface is Obtained as body direction 353. Note that the foot position 207 shown in FIG. 3B may be used as the point B ′ on the floor surface.

  Subsequently, the three-dimensional model generation process 307 calculates a “standing model” and a “recumbent model” and performs a process of virtually arranging both models on the floor surface in the monitoring area.

  With reference to FIG. 5, the process of virtually arranging both models will be described. The three-dimensional model generation process 307 reads the person model 23 stored in the storage unit 20 and enlarges or reduces the person model 23 so as to match the estimated height and estimated width estimated by the physique estimation process 306. Then, the three-dimensional model generation processing 307 virtually arranges the standing model 400 by matching the position of the foot of the person model 23 with the point B ′ on the floor surface of the monitoring area. It is assumed that the standing model 400 stands upright on the floor surface. Further, the position of the foot is matched with the point B ′, and the recumbent model 401 having the same size as the standing model 400 is virtually arranged. The direction in which the recumbent model 401 connects the top of the head and the feet is made to coincide with the floor body direction 353 in FIG.

  The model region generation processing 308 is a model region generation process and a recumbent model in an image when it is assumed that the standing model 400 and the recumbent model 401 that are virtually arranged on the floor of the monitoring region are captured by the imaging unit 10. The area is calculated.

FIG. 6A shows a standing model region that represents the contour of the standing model 400 that is assumed to be captured by the imaging unit 10 when the standing model 400 is placed at the position of the point B ′ on the floor surface of the monitoring region. Is a standing model image 500 cut out by the model image generating means 313. Further, a circumscribed rectangle 501 including the standing model image 500 and a circumscribed rectangle 502 in which the direction of the long side coincides with the main axis direction (not shown) are shown along the vertical axis (y axis) and the horizontal axis (x axis) in the input image. ing. In the case of the standing model, the main axis direction coincides with the y-axis.
FIG. 6B shows the inside of the recumbent model region representing the contour of the recumbent model 401 that the image capturing unit 10 assumed to have captured when the recumbent model 401 was placed at the position of the point B ′ on the floor of the monitoring region. A recumbent model image 503 cut out by the model image generation means 313 is shown. Further, along the vertical axis (y-axis) and the horizontal axis (x-axis) in the input image, the circumscribed rectangle 501 including the recumbent model image 503 and the circumscribed direction in which the direction of the long side coincides with the main axis direction (not shown) of the recumbent model image 503. A rectangle 502 is shown.

As described above, the feature amount calculation unit 303 obtains the standing model feature amount from the standing model image 500 generated by the model region generation processing 308 and the recumbent model feature amount from the recumbent model image 503.
That is, the feature amount calculation unit 303 calculates the standing model feature amount (the circumscribed rectangle area fs1, the circumscribed rectangle aspect ratio fs2, the circumscribed rectangle height fs3, the main axis aspect ratio) from the circumscribed rectangle 501 and the circumscribed rectangle 502 of the standing model image 500. fs4, main axis direction long side length fs5) is calculated.
Also, the feature amount calculation means 303 calculates the horizontal model feature amount (circumscribed rectangle area ft1, circumscribed rectangle aspect ratio ft2, circumscribed rectangle height ft3, main axis aspect ratio ft4, main axis from the circumscribed rectangle 501 and circumscribed rectangle 502 of the recumbent model image 503. The direction long side length ft5) is calculated.
The calculation method of the standing model feature quantity and the recumbent model feature quantity is as described in the description of the feature quantity calculation unit 303. The height q of the circumscribed rectangle 501, the lateral width p, the height h of the circumscribed rectangle 502, Calculated using the width w.

  The standing model feature value fsi (i = 1 to 5) is obtained when the same person as the person's physique shown in the input image is standing at the foot position 207 of the person in FIG. Each characteristic quantity obtained is represented. Similarly, the recumbent model feature amount fti (i = 1 to 5) is obtained when the same person as the person's physique shown in the input image is lying on the person's foot position 207 in FIG. Each feature quantity obtained is shown in FIG.

The weight calculation processing 309 calculates the weights w1 to w5 for each of the recumbent determination feature amounts based on the above-described standing model feature amount and recumbent model feature amount. The weight is calculated for each feature amount for recumbency determination for each of the extracted person regions.
Specifically, the weights w1 to w5 are determined according to the following conditional expressions. In the following, i is an integer of 1 to 5.

  That is, if the condition | fsi−fti | ≦ thi is satisfied, the weight wi = 0.2, and if not, the weight wi = 0.8. thi is a value in which the difference between fsi and fti is recognized as a significant difference between the standing position and the recumbent position, and is a threshold value that is determined experimentally and empirically.

That is, the small difference between the standing model feature value fsi and the recumbent model feature value fti means that the change in the value of the feature value due to the difference between the standing position and the recumbent is small, and for the recumbency determination corresponding to the feature value. This means that it is difficult to distinguish the standing position and the recumbent position using the feature amount fji. Such a recumbent determination feature quantity fji should not be emphasized when obtaining an evaluation value for posture estimation. Therefore, the smaller the difference between the standing model feature quantity fsi and the recumbent model feature quantity fti, the more weight wi. Let be a small value.
On the contrary, the fact that the difference between the standing model feature value fsi and the recumbent model feature value fti is large means that the change in the value of the feature value due to the difference between the standing position and the recumbent is large and the recumbent determination feature corresponding to the feature value. It means that it is easy to distinguish standing and recumbent using the quantity fji. Such a recumbent determination feature value fji should be emphasized when obtaining an evaluation value for posture estimation. Therefore, the greater the difference between the standing model feature value fsi and the recumbent model feature value fti, the greater the weight wi. Let be a large value.

  Note that the feature amount used for the determination is not limited to these feature amounts, and may be any feature amount that is considered to cause a difference between the determined postures. In the present embodiment, any feature value may be used as long as it is considered that a difference in value occurs between standing and lying.

  The numerical value of the weight wi is an example, and the weight wi may be set so that the difference between the standing model feature quantity and the recumbent model feature quantity is smaller than the threshold value and smaller than when the difference is larger than the threshold value. Is preferred.

  For example, the weight wi = 0 may be set when the condition that the difference between the standing model feature quantity and the recumbent model feature quantity is equal to or less than the threshold value, and the weight wi = 1 may be set when the condition is not established. In this case, the feature quantity having the weight wi of 0 is not reflected in the evaluation value for determination described later, and the feature quantity having the weight wi of 1 is completely reflected in the evaluation value for determination described later.

  Further, the weight wi may be a continuous value corresponding to the difference between the standing model feature quantity and the recumbent model feature quantity. For example, the minimum value of | fsi−fti | is 0 when both are equal, and the maximum value can be calculated based on the camera information 21. Therefore, even if the weight wi is set to a value proportional to the value of | fsi−fti | so that the weight wi is 0 when the minimum | fsi−fti | Good. Here, the maximum | fsi-fti | means the maximum | fsi-fti | obtained by arranging a large number of standing models and recumbent models in the monitoring area.

The evaluation formula determination process 310 uses the wi calculated in the weight calculation process 309 to determine a posture evaluation formula for calculating an evaluation value for determination for distinguishing whether a person in the input image is standing or lying.
In the present embodiment, the following equation is used as the posture evaluation equation.

  In Equation (1), vmi is an intermediate evaluation value, and the difference between the recumbent determination feature quantity fji and the standing model feature fsi and the difference between the recumbent determination feature quantity fji and the recumbent model feature quantity fti If the former is small, the intermediate evaluation value vmi is set to +1 if the former is small, and if the latter is small, the intermediate evaluation value vmi is set to −1.

  The evaluation value calculation unit 311 uses the recumbent determination feature amount fji (i = 1 to 5) calculated by the feature amount calculation unit 303 according to the equation (1) determined by the evaluation equation determination process 310. An evaluation value Vj for determination for determining the presence or absence of a lying person is calculated. Then, the evaluation value calculation unit 311 outputs the calculated evaluation value Vj for determination to the posture determination unit 312.

  For example, the difference between the circumscribed rectangular area fj1 and the circumscribed rectangular area fs1 is compared with the difference between the circumscribed rectangular area fj1 and the circumscribed rectangular area ft1, and if the former is small, the circumscribed rectangular area The intermediate evaluation value vm1 is set to +1 assuming that the person is standing, and if the latter is small, the person is lying down and the intermediate evaluation value vm1 is set to -1. Similarly, it is determined whether the person is standing or lying on the circumscribed rectangle aspect ratio, circumscribed rectangle height, principal axis direction aspect ratio, and principal axis direction long side length, and the intermediate evaluation value vm2− Set vm5 to +1 or -1.

  This is because the smaller the difference between the recumbent determination feature value fji and the standing model feature value fsi, the more the same person as the person's physique in the person area can be standing when the posture determination is performed using the recumbent determination feature value fji. The smaller the difference between the recumbent determination feature quantity fji and the recumbent model feature quantity fti, the smaller the difference between the recumbent determination feature quantity fji, This means that there is a high possibility that The intermediate evaluation value vmi is set to a large value (for example, +1) when the possibility that the person is standing is high, and is set to a small value (for example, -1) when the possibility that the person is lying is high.

  The evaluation value Vj for determination obtained by the evaluation value calculation means 311 becomes a larger value as the person area to be evaluated is closer to the standing model area, and becomes a smaller value as it is closer to the recumbent model area. In other words, the evaluation value Vj for determination becomes larger as the person in the person area to be evaluated is closer to the standing position of the person model, and the evaluation value Vj for determination becomes smaller as the person model is closer to lying.

  The posture determination means 312 determines that the person in the input image is standing when the evaluation value Vj for determination exceeds a predetermined determination threshold value, and determines that the person is lying down otherwise. Is output to the output unit 40. Here, the determination threshold is a value experimentally determined as a value that can identify when the person is standing and when the person is lying down.

  The output unit 40 is an interface for connecting to an external device. For example, when connected to a control device of a security device (not shown) and receives a signal representing recumbent from the posture determination means 312, it is converted into a signal of a format that can be received by the control device.

  For example, when the recumbent determination device 1 is installed in an office building or a commercial building, it is reported to the security device installed in the guard room that an elderly person has fallen down due to a sudden illness.

  Next, the operation of the recumbent determination device 1 will be described with reference to the flowchart shown in FIG. The flow in FIG. 7 is executed every time one frame of the input image is acquired.

  In step S <b> 100, the imaging unit 10 acquires an input image and outputs the input image to the image processing unit 30, and the image processing unit 30 temporarily stores the input image in the storage unit 20. The input image is subjected to processing by the image processing unit 30. In step S120, the person area extraction unit 301 extracts a person area image from the input image.

  Here, the following steps S130 to S420 will be described for the case where one person area image is extracted. However, when a plurality of person area images are extracted, they are performed for each person area image.

In step S130, the tracking unit 302 performs tracking processing of the extracted person area image.
In step S140, the posture evaluation formula generation unit 304 refers to the tracking information 22 stored in the storage unit 20, and determines whether or not the number of tracking of the extracted person area image is less than “11”. When the number of tracking is less than “11”, the process proceeds to step S160 to perform the physique estimation process (No branch), and when the number of tracking is “11” or more, the posture evaluation formulas of steps S260 to S340 are calculated. The process proceeds to the process of determining and the process of calculating the feature amount for recumbent determination in S240 (Yes branch).

First, in step S160, the two-dimensional / three-dimensional conversion process 305 and the physique estimation process 306 refer to the camera information 21 in the storage unit 20 and perform a process of estimating the height and width of the person shown in the person area image. .
When the tracking number of the tracking information 22 is “10” at the current time (Yes in step S180), the physique estimation process 306 obtains an average of the estimated height and estimated width for 10 frames, and the physique information (Step S200).
If the number of tracking of the tracking information 22 is not “10” at the current time, the process for obtaining the average of the estimated height and the estimated width (step S200) is not performed.

In step S220, the estimated height and estimated width obtained by the physique estimation process 306 in step S160 are stored in the tracking information 22 as one piece of information related to the person area image until the number of tracking of the tracking information 22 is “9”. When the tracking number of the tracking information 22 becomes “10”, the average value of the estimated height and the estimated width obtained by the physique estimation process 306 in step S200 is stored in the tracking information 22 as physique information.
Then, the process for the person area image at the current time is terminated.

If Yes in step S140, the process proceeds to a process for determining an evaluation formula.
In step S260, the three-dimensional model generation process 307 reads the person model 23 and the physique information (average value of estimated height and estimated width) stored in step S220.

In step S280, the three-dimensional model generation processing 307 generates a three-dimensional model by scaling the person model 23 so as to match the physique information.
In step S300, the model region generation processing 308 causes the three-dimensional model to stand in the monitoring region and the standing model region that is the region of the standing model in the image assumed to be captured by the imaging unit 10, and the three-dimensional The model is laid down on the monitoring region, and a recumbent model region that is a region of a recumbent model in an image assumed to be captured by the imaging unit 10 is obtained. Then, the model image generating unit 313 cuts out the interiors of the standing model region and the recumbent model region, and generates a standing model image and a recumbent model image, respectively.

In step S320, the weight calculation process 309 uses the standing model image and the recumbent model image to calculate the weights of the circumscribed rectangle area, circumscribed rectangle aspect ratio, circumscribed rectangle height, principal axis aspect ratio, and major axis direction long side length. Ask.
The weight calculation process will be described in detail with reference to FIG.

In step S3201, the weight calculation processing 309 selects one of the five feature amounts, and determines the value of the standing model feature amount obtained from the standing model region and the value of the recumbent model feature amount obtained from the recumbent model region. Find the difference.
If the difference exceeds the threshold (Yes branch in step S3203), the weight wi for the feature amount is set to a large value, for example, 0.8 (step S3203), and the weight wi is set to a small value, for example, 0. .2 (step S3204).
When the weight values are determined for all the feature values, the process returns to step S340 in FIG.

  In step S340, the evaluation formula determination process 310 performs a process of determining a posture evaluation formula for calculating a determination evaluation value Vj for determining whether the person is standing or lying. In the present embodiment, the weight wi calculated in step S320 is substituted into the mathematical formula (1).

  On the other hand, in step S240, the feature amount calculation unit 303 calculates a recumbent determination feature amount from the height and width of the person region image.

  In step S360, the evaluation value calculation unit 311 substitutes the value of the feature value for recumbent determination obtained in step S240 into the posture evaluation formula determined in step S340 to obtain the evaluation value Vj for determination.

In step S380, the posture determination unit 312 compares the evaluation value Vj for determination with the threshold for determination, and determines whether the person shown in the person area image is standing or lying.
In step S400, when the posture determination means 312 determines the recumbency of the person shown in the person area image (Yes branch), the posture determination unit 312 outputs that fact to the output unit 40. In step S420, the output unit 40 notifies an external device.
When the posture determination unit 312 determines standing for any of the person area images, no processing is performed (No branch in step S400).

<Modification 1>
In the above-described embodiment, the aspect of estimating the posture when the person who entered the monitoring area is standing and lying is described, but the scope of application of the present invention is not limited to this. For example, when the posture changes from a standing position to a squatting posture, it may not be as important as an unconsciousness due to a sudden illness, but the guard may be alerted.

  The processing here will be described with reference to FIG. FIG. 9A shows a person area image 800 in an input image captured by the imaging unit 10 when a person stands on the floor surface of the monitoring area, and a circumscribed rectangle 801 thereof. FIG. 9B shows a person area image 804 and its circumscribed rectangle 805 in the input image captured by the imaging unit 10 when a person is crouching on the floor surface of the monitoring area. Further, in the first modification, the feature amount calculating unit 303 obtains the main axes 802 and 806 of the person area images 800 and 804. In the example of FIG. 9, the circumscribed rectangles 801 and 805 of the person area images 800 and 804 have a horizontal width p and a height q along the vertical axis (y axis) and the horizontal axis (x axis) in the input image.

Further, as in the case shown in FIG. 5, the model area generation processing 308 virtually arranges the standing model and the crouching model on the floor of the monitoring area, and represents the standing model area representing the outline of the standing model. A crouching model area representing the outline of the crouching model is obtained, and the model image generating means 313 generates a standing model image and a crouching model image cut out from each inside.
Here, the crouching model is information that expresses a posture in which a person to be imaged is crouching with a polygon model or a surface model.

  Then, the feature amount calculation means 303 calculates the squat detection feature amount from the person area in the input image, and the standing model feature amount and the crouch model feature amount value for determining the weight from the standing model image and the crouching model image. Ask.

The feature amount calculation unit 303 calculates the following five as values of a crouch detection feature amount used for crouch detection, a standing model feature amount, and a crouch model feature amount.
(11) Height q of circumscribed rectangle
(12) circumscribed rectangle aspect ratio = width p / height q
(13) Ratio of the number of pixels in the person area to the number of pixels of the circumscribed rectangle (14) Number of pixels in the person area with respect to the number of pixels in the outer periphery of the person area (15) Angles of principal axes 803 and 807 with respect to the x axis of the image

  The weight calculation process 309 calculates the weights w1 to w5 for each of the squatting detection feature values based on the standing model feature value and the squatting model feature value. The calculation method of the weight wi is the same as that in the above embodiment.

As in the present embodiment, the evaluation value calculation unit 311 uses the crouch detection feature amount and the calculated weight to determine a determination evaluation value for determining whether or not a person in the input image is crouching. calculate.
Hereinafter, as in the above-described embodiment, the evaluation value for determination may be compared with the determination threshold value, and thus the details are omitted.

<Modification 2>
In addition, a description will be given of a second modification of a mode for estimating the posture when a person entering the monitoring area is in a posture where both hands are lowered and when both are raised. In this example, even if you are standing, if you move from a position with both hands down to a position with both hands raised, you may be threatened by bandits, etc. Assumed.

  The processing here will be described with reference to FIG. FIG. 10A illustrates a person area image 810 in the input image captured by the imaging unit 10 when the person stands on the floor of the monitoring area with both hands lowered (careful attitude); The circumscribed rectangle 811 is shown. FIG. 10B illustrates a person area image 812 in the input image captured by the imaging unit 10 when the person stands with both hands on the floor surface of the monitoring area (an all-year-old position). The circumscribed rectangle 813 is shown. In the example of FIG. 10, the circumscribed rectangles 811 and 813 of the person area images 810 and 812 have a horizontal width p and a height q along the vertical axis (y axis) and the horizontal axis (x axis) in the input image.

In addition, the model area generation processing 308, as in the case shown in FIG. 5, takes care of the floor of the monitoring area and virtually arranges the model and the year-old model, and takes care to express the outline of the model. The model image generation unit 313 generates a model image and a model image that are cut out from the insides of the model region and the model model generation unit 313.
Here, the watchfulness model and the all-year-old model refer to a posture in which the person to be imaged stands in a careful posture with both hands lowered and a posture in which the both-handed posture with both hands raised is a polygon model or a service model. This is information expressed by a face model.

  Then, the feature amount calculation means 303 takes care of detecting the feature value from the person area image in the input image, the care model feature value and the feature model feature value for determining the weight from the care model image and the care model image. Find the value of.

The feature value calculation means 303 calculates the following three values as the feature values for the detection of the sensation for use in the detection of sensation, the careful model feature value, and the sensation model feature value.
(21) Height q of circumscribed rectangle
(22) circumscribed rectangle aspect ratio = width p / height q
(23) Ratio of the number of pixels in the upper half of the person area to the number of pixels in the upper half of the circumscribed rectangle (corresponding to the upper body)

  Hereinafter, as in the above embodiment and the first modification, the weight corresponding to each feature amount is calculated, the evaluation value for determination is obtained, and compared with the determination threshold value.

  In the embodiments described above and modifications thereof, information representing a three-dimensional shape such as a polygon model or a surface model is stored as a person model stored in the storage unit. Instead, the outline information of a two-dimensional person may be stored as a person model. In this case, the model region generation processing obtains a model region in the input image by enlarging / reducing and deforming the person model while referring to the person's foot position and camera information stored in the storage unit.

  In addition, in the physique estimation process or the 3D model generation process, instead of considering the lowest point of the person area image as the foot and setting it as the foot position in the monitoring area, the same as the foot position. You may ask for. This is because when the person lies on the front side with respect to the imaging unit, the highest point of the person area image corresponds to the foot in the input image.

  In addition, when it is desired to perform recumbent determination even in the first few frames (for example, 10 frames), standard height and width (for example, 170 cm for height and 60 cm for width) may be used as physique information.

  In addition, the person model stored in the storage means may be stored in advance as a person model having the standard dimensions of the person described above, and the person model may be used in common for estimating the posture. In this case, the physique estimation process can be omitted.

  In addition, when not using feature values using pixel values, such as average luminance and texture information, instead of using partial images cut out from inside the person region, etc., the person region itself is used to calculate the feature value. It may be used. In this case, the partial image cut-out process can be omitted.

  As described above, according to the embodiment and the modification example described above, a plurality of feature amounts are used for estimating the posture of a person, and the correct posture is determined by weighting the feature amounts so that the correct posture is estimated. A posture estimation device can be provided.

  DESCRIPTION OF SYMBOLS 1 Recumbent determination apparatus, 23 person model, 303 feature-value calculation means, 304 attitude | position evaluation formula generation means, 306 physique estimation process, 308 model area generation process, 309 weight calculation process, 311 evaluation value calculation means, 312 attitude determination means, 313 Model image generation means.

Claims (5)

An attitude estimation apparatus that calculates a plurality of types of feature amounts from a person area image in an input image captured by an imaging unit installed at a predetermined position and estimates a person's attitude,
A model image generating unit that generates a first model image imitating a person in a first posture photographed by the imaging unit and a second model image imitating a person in a second posture imaged by the imaging unit; ,
A feature amount calculating means for calculating a plurality of types of feature amounts from an image;
Posture evaluation formula generating means for determining a weight for posture evaluation according to the difference between the feature amount of the first model image and the feature amount of the second model image for each feature amount, and generating a posture evaluation formula;
Person area extracting means for extracting a person area image from the input image;
A posture estimation apparatus comprising posture determination means for estimating a posture based on the posture evaluation formula from a feature amount calculated for the person region image.   The model image generation means enlarges or reduces a predetermined person model stored in advance so as to match the physique of the person estimated in the shooting space of the imaging unit from the person area image, and the person model is The posture estimation apparatus according to claim 1, wherein the first model image and the second model image are generated on the assumption that the image is captured by the imaging unit.   The model image generation means estimates a person position in the imaging space of the imaging unit from the person area image, arranges a predetermined person model stored in advance at the position, and the imaging unit images the person model. The posture estimation apparatus according to claim 1, wherein the first model image and the second model image are generated on the assumption. The predetermined person model is a single-shaped person model,
The model image generation means estimates a person's foot position in the imaging space of the imaging unit from the person area image, and arranges the person model in a standing posture and a recumbent posture together with the foot position, The first model image is generated on the assumption that the image capturing unit has photographed the human model, and the second model image is generated on the assumption that the image capturing unit has captured the human model in a lying posture. 4. The posture estimation apparatus according to 3.   4. The posture estimation apparatus according to claim 2, wherein the predetermined person model is a person model having a first posture and a person model having a second posture having a shape different from that of the person model having the first posture. 5.

Images