JP2014229266A - Device for detecting specific operation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for detecting a specific operation capable of calculating a feature quantity having both characteristics of a movement and a shape with less calculation quantity and detecting the specific operation.SOLUTION: A device for detecting a specific operation includes: a camera 1 for outputting a captured image by a continuous image frame; a luminance gradient calculation part 2 for obtaining a luminance gradient for each extremely small region for the image frame; a luminance gradient difference calculation part 3 for extracting a difference between the luminance gradients obtained for the continuous image frame; a time series feature calculation part 4 for extracting a time change from difference information of the extracted luminance gradient and calculating an autocorrelation feature of a gradient strength in order to extract a movement characteristic from the luminance gradient difference; a determination part 5 for comparing the data in which an operation feature is previously numerically converted with a calculation result of the time series feature quantity calculation part 4 and determining an operation; and a result output part 6 for outputting an estimation result.

Description

  The present invention relates to a specific motion detection apparatus that quantifies specific motion and shape information from unknown video data and determines similarity by comparing with specific motion information of a specific object held as prior information.

To estimate a person's behavior from a video, we extract the person's characteristics from an image taken at a certain time, observe how the information changes over time, and The behavior is estimated by comparing the time-series change and the feature data of the image relating to the specific action accumulated in advance, and calculating the similarity.
In recent years, a CHLAC (see, for example, Patent Document 1) feature is known as a method for identifying such an operation. The CHLAC feature is not only a simple feature description, but the same description is possible regardless of the position of the object that enters the scene, and it can be expressed by the sum of feature amounts corresponding to a plurality of simultaneous operations. Widely adopted.

  On the other hand, in nursing homes and facilities for the elderly, there are an increasing number of cases where facility residents such as care recipients are seriously injured such as fractures due to falls. In this case, the most injured is said to occur when getting up and getting up from the bed to go to the toilet. For example, Non-Patent Document 1 discloses a technique using a camera as a measure for preventing such an injury. In this method, a person is imaged by a camera, the head is tracked, and a fall is distinguished from a change in acceleration in three axis directions.

JP 2006-079272 A JP 2008-269063 A

C. Rougier, J. Meunier, A. Saint-Arnaud, and J. Rousseau, “Monocular 3D head tracking to detect falls of elderly people,” In Proc. Of 28th AC. Rougier, J. Meunier, A. Saint-Arnaud , and J. Rousseau, “Monocular 3D head tracking to detect falls of elderly people,” In Proc. of 28th Annual International Confer-ence of the IEEE Engineering in Medicine and Biology Society, pp. 6384-6387, 2006.

However, although the CHLAC feature shows a certain response to actions other than the steady operation, it has been difficult to identify the action itself because it has model-free characteristics. Further, for example, there is a technique of Patent Document 2 as a technique applying CHLAC. However, in Patent Document 2, since the movement of a person is decomposed using multiple divided images, the position invariance characteristic of CHLAC is not utilized. However, the amount of calculation is large and the implementation cost is high.
The technique of Non-Patent Document 1 is an approach for detecting a fall, and is not a technique that can prevent the fall itself.

  Therefore, in view of such problems, the present invention calculates a feature amount having both characteristics of motion and shape with a small amount of calculation to detect a specific motion, and before a person stands up from the bed. It is an object of the present invention to provide a specific motion detection device that can detect and detect the motion of rising motion with a small amount of calculation.

In order to solve the above-described problem, the specific motion detection device according to the first aspect of the present invention includes an imaging unit that outputs a captured image in continuous image frames, and a luminance gradient that obtains a luminance gradient for each minute region with respect to the image frame. A calculation unit, a luminance gradient difference calculation unit that extracts a difference in luminance gradient obtained for a plurality of time-series image frames, and a time from the extracted luminance gradient difference information in order to extract a motion feature from the luminance gradient difference The time series feature calculation unit that extracts the change and calculates the autocorrelation feature of the gradient strength, the data obtained by digitizing the behavior feature of the specific object in advance and the calculation result of the time series feature calculation means It has the determination part to determine, and the result output part which outputs a determination result, It is characterized by the above-mentioned.
According to this configuration, the contour of a moving object can be grasped by obtaining a luminance gradient difference between image frames, and a behavior can be determined by extracting a motion feature from an autocorrelation calculated from the luminance gradient difference. Therefore, it is possible to calculate a feature amount having both characteristics of motion and shape with a relatively small amount of calculation, and it is possible to discriminate a person's specific motion and characteristic motion with an inexpensive device.

According to a second aspect of the present invention, in the configuration according to the first aspect, the image pickup means is a camera that picks up a person on the bed, and the data obtained by previously digitizing the behavioral characteristics of the specific object is a wake-up operation on the bed The determination unit determines a person's wake-up motion from a captured image of the camera.
According to this configuration, since an operation in which a person such as a patient who is lying on the bed rises based on the captured image of the camera, the determination can be made with an inexpensive device. And before the operation of the person to be monitored reaches a dangerous state, the person concerned such as a caregiver can be notified, which helps to prevent an accident.

According to the present invention, a contour of a moving object can be grasped by obtaining a luminance gradient difference between image frames, and a behavior can be determined by extracting a motion feature from an autocorrelation calculated from the luminance gradient difference. Therefore, it is possible to calculate a feature amount having both characteristics of motion and shape with a relatively small amount of calculation, and it is possible to discriminate a person's specific motion and characteristic motion with an inexpensive device.
Then, by using the data obtained by digitizing the action characteristics of the specific object in advance as the data of the person who wakes up on the bed, it is possible to determine the wake-up action of the person on the bed with an inexpensive device.

It is a block diagram which shows an example of the specific operation | movement detection apparatus which concerns on this invention. It is a figure which shows the outline | summary of the feature-value extraction in a time series information calculation part. It is explanatory drawing which shows the flow which determines specific operation | movement from an image, (a) is a time series image frame of the camera picked-up image to discriminate | determine, (b) is a brightness | luminance gradient histogram of each image frame, (c) is an image of a brightness | luminance gradient. The figure which extracted the difference between frames, (d) is the histogram figure which calculated the autocorrelation characteristic from the difference between the frames of a brightness | luminance gradient.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention are described in detail below with reference to the drawings. FIG. 1 is a block diagram showing an example of a specific motion detection apparatus according to the present invention, where 1 is a camera for imaging a monitoring target, and 2 is a luminance gradient calculation for calculating luminance gradient data from video data captured by the camera 1. 3 is a luminance gradient difference calculation unit that extracts a difference between image frames of the luminance gradient, 4 is a time series information calculation unit that calculates autocorrelation, and 5 is a determination unit that identifies an operation using a cascade type discriminator based on AdaBoost. , 6 is a result output unit.
The luminance gradient extraction unit 2, the luminance gradient difference extraction unit 3, the time series information calculation unit 4, and the determination unit 5 are integrally configured by a CPU or DSP in which a predetermined program is installed. The specific operation detected here is specifically an operation in which a person such as a patient who is lying on the bed gets up, and a configuration in which notification is made when an operation in which the person to be monitored rises is detected will be described.

  The camera 1 is installed above the bed and at a position where it is possible to satisfactorily capture at least the head of a person on the bed. For example, the camera 1 generates and outputs an image frame (still image) every 0.03 seconds.

  The luminance gradient calculation unit 2 calculates luminance gradient strength and gradient direction. Specifically, first, downsampling is performed to a predetermined patch size for each input image frame, and gradient intensity m and gradient direction θ are calculated from the luminance of each pixel by the following equations (Equation 1 and Equation 2). An Nth-order histogram (luminance gradient histogram) is generated with pixels.

  dx (x, y) and dy (x, y) are luminance differences and are calculated by the following equation (Equation 3).

  Note that I (x, y) is the luminance on the (x, y) coordinate.

The luminance gradient difference calculation unit 3 calculates the luminance gradient difference between adjacent image frames in time series with respect to the luminance gradient data calculated by the luminance gradient calculation unit 2, and sets the luminance gradient direction θ to the number of bins K. Fold it up. Specifically, the angle information of the approximate pixel in the S × S pixel region is convolved with a gradient histogram having the number K of bins.
A luminance gradient histogram hk (x, y), which is a score of each bin that has been convolved, is expressed by Equation 4.

θ ′ (x, y) is a value obtained by convolving the angular direction θ (x, y) with the number K of bins, δ is a Kronecker delta, and 1 if k is equal to θ ′ (x, y). Otherwise 0 is returned.
Finally, the difference between the luminance gradient histograms hk (x, y) between the adjacent frames obtained by Equation 4 is obtained. The luminance gradient intensity histogram difference hKsub (x, y) at time t is obtained by Equation 5.

The time series information calculation unit 4 calculates the autocorrelation from the temporal change amount of the luminance gradient that is the luminance gradient difference feature. An autocorrelation of the gradient strength is calculated using a predetermined mask pattern formed by adjacent cells with respect to the difference information between the image frames of the luminance gradient. Here, the autocorrelation of the four-dimensional information by the combination of the coordinate position, time, and luminance gradient direction is acquired.
FIG. 2 shows an outline of feature extraction. An image frame FL is arranged on the XY plane (cell positions are defined on the XY plane), and time series information is defined on the t-axis.
CL denotes a cell, and is an aggregate of pixels (for example, composed of 5 × 5 pixels) obtained by dividing the image frame FL into a plurality of parts. The luminance gradient direction is defined by the number of bins for each cell of the histogram, and the gradient intensity is a four-dimensional vector composed of these coordinate positions, time, and luminance gradient direction.

Here, the mask pattern will be described. The N-dimensional mask pattern is obtained from a three-dimensional vector (x, y, t) in a mask block defined by N × N × N cells. For example, when the number of dimensions of the mask pattern is 3, 81 (= 9 × 1 × 9) mask patterns are defined in the mask block. At this time, the cell position at the time of the base point is fixed at one point, and the total number of extracted features is the number of mask blocks scanned by shifting the mask blocks by one cell from the image divided into cell area sizes. . For example, when there are H × W cells in one image, (W−N + 1) × (H−N + 1) mask blocks are obtained per image. Further, (W−N + 1) × (H−N + 1) × K features can be obtained by combining the angular directions.
On the other hand, if the cell position (x, y) is a position vector r and the luminance gradient difference is f (r, k), the Nth-order autocorrelation feature X calculated by the time series information calculation unit 4 is obtained by Equation 6. Can do.

“an” is a displacement vector, “k” and “kn” are gradient directions, “N” is an autocorrelation order, “α” is a difference of convolved angle information, “G (•)” is a Gaussian kernel, and “C” is the number of mask blocks.
As described above, the geometric feature of the present invention calculates the autocorrelation of the gradient strength using the mask pattern, but it is not necessary to define a complicated mask pattern in advance, so that the calculation is simplified compared to the conventional CHLAC. And easy to implement.

  The determination unit 5 includes a learning sample classified according to a person and a background, and a strong discriminator learned using the calculated feature amount, and a statistical feature amount is calculated using a well-known cascade booster of Adaboost. The similarity is determined by calculation.

  The result output unit 6 includes a sound report unit for reporting an alarm, a display unit for displaying the detected video, a report unit for reporting to the outside, and the determination unit 5 performs a similar evaluation to notify the sound, Displays the image captured by the camera.

  Next, the detection operation of the specific operation detection device described above, specifically, the operation of detecting the wake-up operation in which a person lying on the bed gets up will be described. FIG. 3 is an explanatory diagram showing a flow of detecting a person's wake-up motion from an image, and will be described with reference to FIG. 3A is a time-series image frame output from the camera 1, FIG. 3B is a luminance gradient histogram (HOG) extracted from each image frame output from the camera 1, and FIG. 3C is an extracted luminance gradient histogram. Image inter-frame difference information, (d) shows the result of calculating the autocorrelation feature from the inter-frame difference of the luminance gradient histogram.

As shown in FIG. 3B, the gradient strength m and the gradient direction θ are calculated from the luminance of each pixel of the four-frame image extracted by the luminance gradient calculation unit 2 in units of, for example, 2 seconds output from the camera 1. Generate a brightness gradient histogram.
Next, the luminance gradient difference calculation unit 3 calculates the difference in the luminance gradient histogram between adjacent image frames in time series based on the luminance gradient data output from the luminance gradient calculation unit 2. By this calculation, the contour of the fluctuating part is extracted as shown in FIG.
Then, the time series information calculation unit 4 calculates an autocorrelation from the calculated luminance gradient difference feature. As a result, it is possible to extract only the displacement of the object of interest while eliminating uncorrelated displacements such as illumination fluctuations and movements of other objects with different movement directions.
In this way, the inter-frame difference is calculated, further autocorrelation is obtained, and a two-stage feature extraction process is performed to obtain a statistical feature value as a final result.

  Finally, the determination unit 5 obtains a statistical feature amount using Adaboost as shown in FIG. 3D from the autocorrelation features of three consecutive frames, and determines whether it is a wake-up operation. When the wake-up operation is determined in this way, a notification operation such as a warning sound is performed in the result output unit 6, and when the monitor is provided, the captured image of the camera 1 is displayed.

Thus, the contour of the moving object can be grasped by obtaining the inter-frame difference of the luminance gradient, and the action can be estimated by extracting the temporal change of the inter-frame difference. Therefore, it is possible to calculate a feature amount having both characteristics of motion and shape with a relatively small amount of calculation, and it is possible to discriminate a human specific action and a characteristic action with an inexpensive apparatus.
Further, since the specific motion of the object is detected based on the autocorrelation feature obtained from the gradient strength and the gradient direction of the edge, the specific motion can be detected with a small amount of calculation.
Based on the captured image of the camera 1 disposed above the bed, it is possible to detect the movement of a person who is lying on the bed, and the rising can be determined with an inexpensive device. Therefore, it is possible to notify related persons such as caregivers before the monitored person's movement becomes dangerous, which helps prevent accidents and does not emit sound or light to the monitored person. There will be no burden on you.

Although the said embodiment demonstrated regarding the operation | movement which detects the operation | movement which the person who is lying in the bed rises, the specific operation | movement detection apparatus of this invention does not limit the operation | movement detected. The specific operation can be widely changed according to the contents of the learning samples to be compared in the determination unit 5, and when considered on the extension line of getting up, the operation to wake up is detected and notified without being reacted. You may let them.
In addition, when it is desired to detect a suspicious person incorporated in a crime prevention facility, it is possible to use it to detect a suspicious person by accumulating learning samples of an operation of looking into a window as a learning sample.
Furthermore, it is not necessary to limit the detection target to a person, and it is possible to discriminate widely moving objects by changing the contents of the learning sample.

  DESCRIPTION OF SYMBOLS 1 ... Camera (imaging means) 2. Brightness gradient calculation part 3 .... Brightness gradient difference calculation part 4 .... Time series information calculation part 5 .... Determination part 6 .... Result output part

Claims (2)

Imaging means for outputting captured images in successive image frames;
A luminance gradient calculating unit for obtaining a luminance gradient for each minute region with respect to the image frame;
A luminance gradient difference calculation unit for extracting the difference of the luminance gradient obtained for a plurality of time-series image frames;
In order to extract a motion feature from the brightness gradient difference, a time-series feature calculation unit that extracts a time change from the extracted difference information of the brightness gradient and calculates an autocorrelation feature of the gradient strength;
A determination unit that compares the data obtained by quantifying the behavioral characteristics of the specific object in advance with the calculation result of the time-series characteristic calculation unit to determine the similarity;
And a result output unit that outputs a determination result. The imaging means is a camera that images a person on the bed, and data obtained by digitizing behavioral characteristics of a specific object in advance is data of a person who wakes up on the bed, and the determination unit includes the camera The specific motion detection device according to claim 1, wherein a wake-up motion of a person is determined from the captured video.

Images