WO2016031313A1

WO2016031313A1 - Physical condition-detecting apparatus, physical condition-detecting method, and physical condition-detecting program

Info

Publication number: WO2016031313A1
Application number: PCT/JP2015/064255
Authority: WO
Inventors: 安川　徹; 石井　智之
Original assignee: Ｎｋワークス株式会社
Priority date: 2014-08-25
Filing date: 2015-05-19
Publication date: 2016-03-03
Also published as: JP2017205134A

Abstract

Provided is a physical condition-detecting apparatus capable of being introduced relatively easily and, after completely extracting a user's walking characteristics, capable of detecting the physical condition of said user. The physical condition-detecting apparatus according to one aspect of the present invention is provided with: a storage unit for storing walking characteristic data representing a user's walking characteristics; an image-acquiring unit for acquiring filmed images that comprise depth data; a walking characteristic analysis unit for acquiring analysis data representing the walking characteristics of a user appearing in filmed images by extracting the region in the filmed images in which the user appears and analyzing the walking movement of the user appearing in said region in real space on the basis of the depth data; and a physical condition-specifying unit for assessing whether or not changes satisfying prescribed conditions have occurred in the user's walking characteristics by comparing the analysis data with walking characteristic data and specifying the user's physical condition on the basis of the results of said assessment.

Description

Physical condition detection device, physical condition detection method, and physical condition detection program

The present invention relates to a physical condition detection device, a physical condition detection method, and a physical condition detection program.

Patent Document 1 discloses a physical condition detection device that estimates a user's physical condition based on a change in walking pitch. Specifically, this physical condition detection device measures a long-time average and a short-time average of a user's walking pitch using a vibration sensor, a microphone, or a human body potential sensor, and measures the long-time average and short-time average of the measured walking pitch. The user's physical condition is detected by comparing with the average.

Patent Document 2 discloses a health management system that estimates the health status of a user using data obtained from a sensor that detects the user's daily motion. Specifically, this health management system calculates the position of the user by using a floor load sensor and a handrail load sensor as sensors for detecting the daily motion of the user. Then, the health management system obtains gaits such as a stride and a step from the position change of the user, and estimates the health state of the user based on the gait. Note that Patent Document 2 includes a TV camera and an infrared sensor as examples of other sensors.

JP 2003-290175 A JP 2007-037851 A

For example, in Patent Document 1, the physical condition detection apparatus causes a user to have a measuring instrument such as a vibration sensor, and estimates the walking pitch of the user based on data obtained from the measuring instrument. For this reason, it is impossible to measure the health state of a user who does not carry the measuring instrument, and there is a problem that the convenience of the apparatus is low.

On the other hand, in the health management system according to Patent Document 2, a floor load sensor provided on a floor of a house and a handrail load sensor provided on a handrail are used to measure a user's gait. Therefore, even if a user does not carry a measuring instrument etc., a user's health condition can be estimated. Therefore, according to the health management system according to Patent Document 2, the convenience for the user can be enhanced.

However, in order to accurately grasp the user's gait, it is required to appropriately arrange the floor load sensor and the handrail load sensor according to the gait. In addition, as the number of sensors is increased in order to acquire data accurately, the cost for introducing the system increases. For this reason, there is a problem that it is difficult to introduce the system.

On the other hand, in the health management system according to Patent Document 2, a TV camera or an infrared sensor that is relatively easy to introduce without using a floor load sensor and a handrail load sensor as a sensor for detecting a daily operation of the user is used. A method to use is conceivable.

However, since the image obtained by the TV camera is a two-dimensional image, whether or not the user's walking characteristics can be analyzed based on this image depends on the traveling direction of the user with respect to the TV camera. Therefore, there is a possibility that the user's walking characteristics cannot be sufficiently extracted depending on the traveling direction of the user.

Also, the infrared sensor detects the user's action by projecting and receiving infrared rays or receiving infrared rays emitted from the human body and detecting changes in the amount of received light. For this reason, such an infrared sensor can basically only detect the presence or absence of a human body, and the amount of information obtained by the infrared sensor is small. Therefore, even with this method using an infrared sensor, there is a possibility that the user's walking characteristics cannot be extracted sufficiently, as with a TV camera.

That is, although a method using a TV camera or an infrared sensor is relatively easy to introduce, there is a possibility that the user's walking characteristics cannot be extracted sufficiently. Therefore, this method has a problem that the user's health condition may not be specified with high accuracy.

In one aspect, the present invention has been made in consideration of such points, and can be introduced relatively easily. After sufficiently extracting the user's walking characteristics, the physical condition of the user can be improved. An object is to provide a technique for enabling detection.

The present invention adopts the following configuration in order to solve the above-described problems.

That is, the physical condition detection device according to one aspect of the present invention is photographed by a storage unit that stores walking characteristic data indicating walking characteristics in a user's real space and a photographing device installed to photograph the user's walking motion. An image acquisition unit that acquires a captured image including depth data indicating the depth of each pixel in the captured image; and an area in which the user is captured in the captured image, and the depth data Based on the depth of each pixel in the captured image indicated by, the analysis of the walking behavior in the real space of the user reflected in the captured image by analyzing the walking motion in the real space of the user captured in the region By comparing the walking characteristic indicated by the walking characteristic data and the walking characteristic indicated by the acquired analysis data by comparing the walking characteristic analysis unit that acquires data Determining whether a predetermined condition is satisfied change occurs in gait characteristic of over THE, based on the result of the determination, and a physical condition specifying section for specifying the physical condition of the user caught on the captured image.

According to the above configuration, the physical condition of the user is detected based on the user's walking characteristics. Here, the captured image acquired in order to analyze the user's walking characteristics includes depth data indicating the depth of each pixel. The depth of each pixel represents the depth from the photographing apparatus to the subject. Therefore, by using this depth, it is possible to analyze the state of the subject in the real space (three-dimensional space) regardless of the moving direction of the subject with respect to the photographing apparatus. Therefore, according to the above configuration, since the state in the real space of the user can be analyzed without depending on the installation location of the photographing apparatus, it can be introduced relatively easily, and the walking characteristics of the user are sufficiently In addition, the physical condition of the user can be detected.

Note that the walking characteristics are the properties of the user's walking motion, such as height, stride, step-to-step ratio, step, toe-up, toe-up, left-to-right ratio, heel up, heel Left / right ratio, cadence, body tilt, trunk lateral swing, pelvic side tilt, knee joint bending angle, knee joint bending angle left / right ratio, arm swing angle, arm swing angle left / right Items such as ratio and presence / absence of walking aids.

As another form of the physical condition detection device according to the above aspect, the physical condition specifying unit compares the acquired analysis data and the walking characteristic data as the analysis data is repeatedly acquired. Thus, it may be repeatedly determined whether or not a change satisfying a predetermined condition has occurred in the walking characteristics of the user. Then, while the physical condition specifying unit determines that the change satisfying the predetermined condition has not occurred in the walking characteristic of the user, the analysis data repeatedly acquired satisfies the predetermined condition on the walking characteristic of the user. It may be further determined whether or not it is changed in a direction in which it is determined that a satisfying change has occurred, and the analysis data obtained repeatedly causes a change that satisfies a predetermined condition in the walking characteristics of the user. When it is determined that the user is changing in the direction in which the user is determined to be present, a sign of the physical condition change of the user may be detected.

In this configuration, a sign of a change in the physical condition of the user is detected by referring to repeatedly acquired analysis data and grasping the change tendency of the analysis data. Thereby, the sign can be grasped in advance before the change of the user's physical condition occurs. That is, according to the said structure, it becomes possible to grasp | ascertain the change of a user's physical condition in steps.

Further, as another form of the physical condition detection device according to the above aspect, the storage unit may store the walking characteristic data for each individual. Then, the physical condition detection device compares the walking characteristic indicated by the walking characteristic data of each individual with the walking characteristic indicated by the acquired analysis data, so that the acquired analysis data is the walking characteristic of any individual. It is determined whether or not the data fits the data, and if the analysis data matches the walking characteristic data of any one of the individuals, the user shown in the captured image is a person indicated by the adapted walking characteristic data. A user identifying unit that identifies

According to this configuration, in addition to detecting the user's physical condition, the user can be identified based on the user's walking characteristics. Therefore, when targeting a plurality of users, it is possible to detect the physical condition of the user after identifying the user without separately providing a means for identifying the user.

As another form of the physical condition detection device according to the above aspect, the walking characteristic data stored in the storage unit may include a normal value and a current value of the user's walking specification, and the walking characteristic analysis unit The analysis data acquired by may include an analysis value of the walking characteristic. The user specifying unit may specify a user who appears in the captured image by comparing a current value included in the walking characteristic data with an analysis value included in the analysis data. Furthermore, the physical condition specifying unit may specify the physical condition of the user by comparing a normal value included in the walking characteristic data with an analysis value included in the analysis data.

The human walking characteristics can change over time. In the process of detecting the physical condition of the user, the change in the physical condition of the user is specified based on determining whether or not this change has occurred. On the other hand, in the process of identifying the user, the user is specified based on the degree of matching between the stored data and the analysis result. Therefore, if each process is continued to be executed based on the same data, there is a high possibility that a problem will occur in any of the processes.

On the other hand, in this configuration, two types of attribute values are held as normal values used for detecting physical condition and current values used for identifying users as attribute values of walking characteristics. The normal value indicates the normal walking motion of the user. On the other hand, the current value is obtained, for example, immediately after executing the physical condition detection process, and indicates the user's walking motion at the present time. Therefore, according to the said structure, the attribute value utilized for a user's identification process and the attribute value utilized for a user's physical condition detection process are separated. For this reason, it is possible to prevent the same value from being used together in both processes, and thus it is possible to prevent the above process from occurring.

As another form of the physical condition detection device according to the one aspect, the storage unit replaces the current value of the walking characteristic included in the walking characteristic data of the user specified by the user specifying unit with the analysis value of the analysis data. Thus, the current value of the walking characteristic included in the identified walking characteristic data of the user may be updated.

According to this configuration, the current value indicated by the walking characteristic data is updated using the analysis data that has been successfully identified by the user. Therefore, a current value used for user identification can be obtained in response to a change in walking characteristics over time without separately providing a means for acquiring the current value of the user's walking motion.

As another form of the physical condition detection device according to the above aspect, the walking characteristic may be composed of a plurality of items. The user specifying unit compares the acquired analysis data with the walking characteristic data in at least one of a plurality of items included in the walking characteristic, thereby allowing the user to appear in the captured image. May be specified. Further, the physical condition specifying unit specifies the physical condition of the user by comparing the analysis data and the walking characteristic data in one or more items different from the items used by the user specifying unit. Also good.

As mentioned above, human walking characteristics can change over time. Therefore, if the same value for the same item continues to be used in both processes, a problem may occur in either process. On the other hand, in the said structure, the item of the walk characteristic utilized for a user's identification process and the item of the walk characteristic utilized for a user's physical condition detection process are separated. Therefore, it is possible to prevent the above processing from occurring.

Further, as another form of the physical condition detection device according to the above aspect, the walking characteristic analysis unit calculates the foreground region of the captured image from a difference between a background image set as a background of the captured image and the captured image. You may extract as an area | region where a user shows. Then, the walking characteristic analysis unit identifies the depth of each pixel in the foreground region by referring to the depth data, and is reflected in the foreground region based on the identified depth of each pixel in the foreground region. The walking motion of the user in real space may be analyzed.

According to this configuration, the foreground area of the photographed image is specified by extracting the difference between the background image and the photographed image. That is, this foreground area is an area where a change has occurred from the background image. Therefore, when the user is walking, the foreground area shows the user who performs the walking motion. Therefore, by using this foreground area, it is possible to specify the area in which the user appears. Here, the process of extracting the foreground region is merely a difference calculation between the background image and the captured image. Therefore, according to the said structure, the area | region where a user is imaged can be narrowed down by a simple process.

In addition, as another form of the physical condition detection apparatus according to each of the above forms, an information processing system that realizes each of the above configurations, an information processing method, or a program may be used. It may be a storage medium that can be read by a computer, other devices, machines, or the like in which such a program is recorded. Here, the computer-readable recording medium is a medium that stores information such as programs by electrical, magnetic, optical, mechanical, or chemical action. The information processing system may be realized by one or a plurality of information processing devices.

For example, in the physical condition detection method according to one aspect of the present invention, a computer provided with a storage unit that stores walking characteristic data indicating walking characteristics in a user's real space is set to capture the user's walking motion. A captured image captured by the apparatus, the captured image including depth data indicating the depth of each pixel in the captured image, and a step of extracting an area in which the user is captured in the captured image; Based on the depth of each pixel in the captured image indicated by the depth data, the walking characteristics of the user in the real space captured in the captured image are analyzed by analyzing the walking motion of the user in the real space captured in the region. The step of acquiring the analysis data indicating the gait characteristic indicated by the gait characteristic data and the gait characteristic indicated by the acquired analysis data are compared. Determining whether or not a change satisfying a predetermined condition has occurred in the walking characteristics of the user, and identifying the physical condition of the user in the captured image based on the result of the determination. This is an information processing method to be executed.

In addition, for example, a program according to an aspect of the present invention is a shooting that is installed in a computer including a storage unit that stores walking characteristic data indicating walking characteristics in a user's real space so as to capture the user's walking motion. A captured image captured by the apparatus, the captured image including depth data indicating the depth of each pixel in the captured image, and a step of extracting an area in which the user is captured in the captured image; Based on the depth of each pixel in the captured image indicated by the depth data, the walking characteristics of the user in the real space captured in the captured image are analyzed by analyzing the walking motion of the user in the real space captured in the region. The analysis data indicating the gait characteristics indicated by the gait characteristics data and the gait characteristics indicated by the acquired analysis data Comparing the step of determining whether or not a change satisfying a predetermined condition has occurred in the walking characteristics of the user, and identifying the physical condition of the user in the captured image based on the result of the determination; It is a physical condition detection program for executing.

According to the present invention, it can be introduced relatively easily, and the physical condition of the user can be detected after sufficiently extracting the user's walking characteristics.

FIG. 1 shows an example of a scene where the present invention is applied. FIG. 2 illustrates a hardware configuration of the physical condition detection device according to the embodiment. FIG. 3 illustrates the relationship between the depth acquired by the camera according to the embodiment and the subject. FIG. 4 illustrates the walking characteristic data according to the embodiment. FIG. 5 illustrates a functional configuration of the physical condition detection device according to the embodiment. FIG. 6 illustrates analysis data according to the embodiment. FIG. 7 illustrates a processing procedure related to the user's physical condition detection by the physical condition detection device according to the embodiment. FIG. 8 illustrates a captured image acquired by the camera according to the embodiment.

Hereinafter, an embodiment according to one aspect of the present invention (hereinafter also referred to as “this embodiment”) will be described with reference to the drawings. However, this embodiment described below is only an illustration of the present invention in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. That is, in implementing the present invention, a specific configuration according to the embodiment may be adopted as appropriate. Although data appearing in the present embodiment is described in a natural language, more specifically, it is specified by a pseudo language, a command, a parameter, a machine language, or the like that can be recognized by a computer.

§1 Application scene First, the scene where the present invention is applied will be described with reference to FIG. FIG. 1 illustrates a scene in which the physical condition detection device 1 according to the present embodiment is used. The physical condition detection apparatus 1 according to the present embodiment estimates the physical condition of the user by photographing the walking motion of the user and analyzing the walking characteristics of the user shown in the captured image 3 obtained thereby. Therefore, the physical condition detection apparatus 1 according to the present embodiment can be widely used in scenes where the physical condition of the user is detected.

Specifically, the physical condition detection device 1 according to the present embodiment photographs a walking user with the camera 2. The camera 2 corresponds to the photographing apparatus of the present invention. The camera 2 is installed to capture a user's walking motion. Therefore, the camera 2 may be disposed at any location as long as the user's walking motion can be measured.

The camera 2 is configured to be able to acquire the depth corresponding to each pixel in the captured image 3. In the present embodiment, the camera 2 includes a depth sensor (a depth sensor 21 described later) that measures the depth of the subject so that the depth of each pixel can be acquired. However, the configuration of the camera 2 is not limited to such an example as long as the depth can be acquired, and can be appropriately selected according to the embodiment.

For example, the camera 2 may be a stereo camera so that the depth of the subject within the shooting range can be specified. Since the stereo camera shoots the subject within the shooting range from a plurality of different directions, the depth of the subject can be recorded. The camera 2 may be replaced with a single depth sensor as long as the depth of the subject within the shooting range can be specified.

The physical condition detection apparatus 1 according to the present embodiment acquires a captured image 3 captured by such a camera 2 as an information processing apparatus. The captured image 3 includes depth data indicating the depth obtained for each pixel, as illustrated in FIG. 8 described later. The captured image 3 may be data indicating the depth of the subject within the imaging range, and may be data (for example, a depth map) in which the depth of the subject within the imaging range is distributed two-dimensionally. The captured image 3 may include an RGB image together with the depth data. Further, the captured image 3 may be a moving image or a still image.

Subsequently, the physical condition detection device 1 extracts a region in which the user is captured in the acquired captured image 3 and analyzes the walking characteristics of the user in the extracted region. At this time, the physical condition detection device 1 analyzes the state of the user in the real space using the depth of each pixel in the captured image 3 indicated by the depth data. Thereby, the physical condition detection apparatus 1 acquires analysis data (analysis data 40 described later) indicating walking characteristics in the real space of the user shown in the captured image 3.

Furthermore, the physical condition detection device 1 compares the walking characteristic data registered in advance (walking characteristic data 121 described later) with the analysis data to determine whether or not a change satisfying a predetermined condition has occurred in the user's walking characteristic. Determine. Then, the physical condition detection device 1 identifies the physical condition of the user shown in the captured image 3 based on the determination result.

Thus, according to the present embodiment, the physical condition detection device 1 acquires depth data indicating the depth of each pixel together with the captured image 3 in order to analyze the walking characteristics of the user. When this depth data is used, as illustrated in FIG. 1, the position of the subject in the captured image in the real space (three-dimensional space) can be specified. Therefore, the physical condition detection device 1 can analyze the state of the user in the real space regardless of the moving direction of the user with respect to the camera 2 by using the depth data. Therefore, according to the present embodiment, it is possible to analyze the state of the user in the real space without depending on the installation location of the camera 2, so that it can be introduced relatively easily, and the walking characteristics of the user can be improved. The user's physical condition can be detected after sufficiently extracting.

In addition, if the arrangement | positioning location of the physical condition detection apparatus 1 can acquire the picked-up image 3 from the camera 2, it can be suitably determined according to embodiment. For example, the physical condition detection device 1 may be disposed so as to be close to the camera 2 as illustrated in FIG. Moreover, the physical condition detection apparatus 1 may be connected to the camera 2 via a network, and may be disposed at a place completely different from the camera 2.

§2 Configuration example <Hardware configuration>
Next, the hardware configuration of the physical condition detection device 1 will be described with reference to FIG. FIG. 2 illustrates a hardware configuration of the physical condition detection device 1 according to the present embodiment. As illustrated in FIG. 2, the physical condition detection device 1 stores a control unit 11 including a CPU, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, a program 5 executed by the control unit 11, and the like. Unit 12, a touch panel display 13 for displaying and inputting images, a speaker 14 for outputting sound, an external interface 15 for connecting to an external device, a communication interface 16 for communicating via a network, and This is a computer to which a drive 17 for reading a program stored in the storage medium 6 is electrically connected. In FIG. 2, the communication interface and the external interface are described as “communication I / F” and “external I / F”, respectively.

In addition, regarding the specific hardware configuration of the physical condition detection device 1, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, the control unit 11 may include a plurality of processors. In addition, for example, the touch panel display 13 may be replaced with an input device and a display device that are separately connected independently. For example, the speaker 14 may be omitted. Further, for example, the speaker 14 may be connected to the physical condition detection device 1 as an external device, not as an internal device of the physical condition detection device 1. Moreover, the physical condition detection apparatus 1 may incorporate the camera 2.

The physical condition detection device 1 may include a plurality of external interfaces 15 and may be connected to a plurality of external devices. In the present embodiment, the physical condition detection device 1 is connected to the camera 2 via the external interface 15. The camera 2 according to the present embodiment is installed to capture a user's walking motion. The camera 2 includes a depth sensor 21 for measuring the depth of the subject. The type and measurement method of the depth sensor 21 may be appropriately selected according to the embodiment. For example, the depth sensor 21 may be a sensor of TOF (TimeＦOf Flight) method or the like.

Note that the place where the user is photographed may be dark. Therefore, an infrared depth sensor that measures the depth based on infrared irradiation may be used as the depth sensor 21 so that the depth can be acquired without being affected by the brightness of the shooting location. Examples of relatively inexpensive imaging apparatuses including such an infrared depth sensor include Kinect from Microsoft, Xtion from ASUS, and CARMINE from PrimeSense.

Here, the depth measured by the depth sensor 21 according to the present embodiment will be described in detail with reference to FIG. FIG. 3 shows an example of a distance that can be handled as the depth according to the present embodiment. The depth represents the depth of the subject. As exemplified in FIG. 3, the depth of the subject may be expressed by, for example, a straight line distance A between the camera 2 and the object, or a perpendicular distance B from the horizontal axis with respect to the subject of the camera 2. It may be expressed as

That is, the depth according to the present embodiment may be the distance A or the distance B. In the present embodiment, the distance B is treated as the depth. However, the distance A and the distance B can be converted into each other by using, for example, the three-square theorem. Therefore, the following description using the distance B can be applied to the distance A as it is. By using such depth, the physical condition detection device 1 according to the present embodiment analyzes the walking characteristics of the user in real space.

Moreover, the physical condition detection apparatus 1 according to the present embodiment stores the walking characteristic data 121 indicating the walking characteristics in the real space of the user for each individual in the storage unit 12. The walking characteristic data 121 will be described with reference to FIG.

FIG. 4 illustrates walking characteristic data 121 according to the present embodiment. The walking characteristic data 121 is data indicating a user's walking characteristic. In this embodiment, one walking characteristic data 121 corresponds to one individual data. Therefore, in this embodiment, the number of walking characteristic data 121 corresponding to the number of persons whose physical condition is to be detected is retained.

As illustrated in FIG. 4, the walking characteristic data 121 according to the present embodiment includes information indicating a data ID, a person name, and a walking characteristic. The data ID is an identifier for identifying the walking characteristic data 121 of each individual. The person name indicates the name of the person targeted by each walking characteristic data 121.

Further, the walking characteristic indicates the property of the walking action of the person targeted by each walking characteristic data 121. In the present embodiment, the walking characteristics include stride, step-to-step ratio, step, toe rise, heel rise, body tilt, trunk lateral swing, pelvic side tilt, knee joint Items such as bending angle, arm swing angle, presence / absence of walking aids are included.

The stride indicates the distance traveled in one step. This one step refers to an operation until the heel of one foot is grounded and then the heel of the other foot is grounded. The left / right ratio of the stride indicates the ratio between the stride when the right foot is stepped forward and the stride when the left foot is stepped forward. The step length indicates the width between the left and right legs. The way of raising the toes indicates how the right or / and left toes are raised during the walking motion. The way of raising the heel indicates how the heel of the right foot and / or the left foot is raised during the walking motion. The inclination of the body indicates the angle of the body axis during the walking motion. The lateral shaking of the trunk indicates how the trunk moves outside during walking. The lateral inclination of the pelvis indicates the degree of inclination of the pelvis in the vertical direction during the walking motion. The bending angle of the knee joint indicates the degree of bending of the right knee and / or the left knee during the walking motion. The arm swing angle indicates the degree of bending of the right arm and / or the left arm during the walking motion. The presence / absence of a walking assist device indicates whether the target individual uses the walking assist device. Note that the walking assist device is a device for stabilizing walking, such as a walking stick, a walker, and the like.

In this embodiment, the walking characteristic data 121 includes two types of attribute values, a normal value and a current value, as attribute values of each item of the walking characteristic. The normal value indicates the normal walking motion of the user. On the other hand, the current value is obtained, for example, immediately after executing the physical condition detection process, and indicates the user's walking motion at the present time. As described above, since the walking characteristic data 121 holds two types of attribute values for each item of the walking characteristic, it is possible to prevent a problem from occurring in any of the user identification process and the physical condition detection process described later. In addition, the cause which a malfunction produces in either of both processes is based on the change of a user's walk characteristic with progress of time. Therefore, in the initial setting, the normal value and the current value may be handled as the same data. In other words, in the initial setting, the walking characteristic data 121 includes only one attribute value for each item of the walking characteristic, and may be configured to include two types of attribute values in the process described later. .

Each attribute value of each item is given so as to be compatible with the analysis processing described later. For example, the attribute value of an item that is relatively less likely to change over time, such as the stride, the step-to-step ratio, the step, the body inclination, and the presence / absence of a walking aid, may be given as a single numerical value. On the other hand, attribute values such as how to raise the toes, how to raise the heel, lateral movement of the trunk, lateral inclination of the pelvis, bending angle of the knee joint, arm swing angle, etc. are relatively likely to change over time. For example, it changes like a waveform pattern according to the walking cycle. Therefore, the attribute value of such an item may be given as a maximum value (single numerical value), may be given as an array representing the waveform pattern, or may be given as two numerical values: a period and a maximum value. It may be granted. The format of the attribute value of each item can be appropriately selected according to the embodiment.

Note that the item of walking characteristics is not limited to these examples, and can be appropriately selected according to the embodiment. For example, in addition to the items described above, the walking characteristics include items such as height, left / right ratio of how to raise the toe, left / right ratio of how to raise the heel, cadence, left / right ratio of the knee joint bending angle, left / right ratio of the arm swing angle, etc. Can be included. The cadence is also called a walking rate and indicates the number of steps in a unit time.

Also, the number of items of walking characteristics can be appropriately selected according to the embodiment. The number of items of walking characteristics may be singular or plural. When the walking characteristic indicated by the walking characteristic data 121 includes a plurality of items, the physical condition detection device 1 accepts a selection of an item to be used for physical condition detection from the plurality of items of the walking characteristic from the user. May be.

Note that the data format of the walking characteristic data 121 is not limited to such an example as long as the walking characteristics of each individual can be stored, and can be appropriately selected according to the embodiment. For example, the walking characteristic data 121 may be held as one record of a table format database.

Further, each attribute value related to the walking characteristics of the walking characteristics data 121 may be acquired by analyzing a captured image obtained by capturing the walking motion of each individual by a walking characteristics analysis process described later. In addition, each attribute value may be directly input by the user, or may be determined by selecting from a plurality of candidates. Such walking characteristic data 121 may be acquired from another information processing apparatus via a network.

The program 5 stored in the storage unit 12 is a program for causing the physical condition detection device 1 to execute each process related to physical condition detection described later, and corresponds to the “program” of the present invention. The program 5 may be recorded on the storage medium 6. The storage medium 6 stores information such as a program by an electrical, magnetic, optical, mechanical, or chemical action so that information such as a program recorded by a computer or other device or machine can be read. It is a medium to do. The storage medium 6 corresponds to the “storage medium” of the present invention. 2 illustrates a disk-type storage medium such as a CD (Compact Disk) or a DVD (Digital Versatile Disk) as an example of the storage medium 6. However, the type of the storage medium 6 is not limited to the disk type and may be other than the disk type. Examples of the storage medium other than the disk type include a semiconductor memory such as a flash memory.

Further, as the physical condition detection device 1, for example, a general-purpose device such as a PC (Personal Computer), a tablet terminal, etc. may be used in addition to a device designed exclusively for the provided service. Furthermore, the physical condition detection device 1 may be implemented by one or a plurality of computers.

<Functional configuration example>
Next, the functional configuration of the physical condition detection device 1 will be described with reference to FIG. FIG. 5 illustrates a functional configuration of the physical condition detection device 1 according to the present embodiment. In this embodiment, the control part 11 of the physical condition detection apparatus 1 expand | deploys the program 5 memorize | stored in the memory | storage part 12 to RAM. And the control part 11 interprets and runs the program 5 expand | deployed by RAM by CPU, and controls each component. Thereby, the physical condition detection device 1 functions as a computer including the image acquisition unit 31, the walking characteristic analysis unit 32, the physical condition specifying unit 33, and the user specifying unit 34.

The image acquisition unit 31 acquires a captured image 3 captured by the camera 2. As described above, the captured image 3 includes depth data indicating the depth of each pixel. Therefore, the walking characteristic analysis unit 32 extracts an area in which the user is captured in the captured image 3. And based on the depth of each pixel in the picked-up image 3 shown by depth data, the walk characteristic in the real space of the user reflected in the extracted area | region is analyzed. Thereby, the walking characteristic analysis part 32 acquires the analysis data 40 illustrated in FIG.

FIG. 6 illustrates analysis data 40 according to this embodiment. The analysis data 40 is obtained based on the result of the analysis by the walking characteristic analysis unit 32, and indicates the walking characteristic in the real space of the user shown in the captured image 3. The analysis data 40 is acquired corresponding to the walking characteristics included in the walking characteristics data 121.

That is, the walking characteristic analysis unit 32 analyzes the walking action of the user in the captured image 3 with respect to each item of the walking characteristic included in the walking characteristic data 121. Therefore, in the analysis data 40 according to the present embodiment, as the walking characteristics of the user shown in the captured image 3, the stride, the step-to-step ratio, the step, the toe rise, the heel rise, the body tilt, the trunk Analysis values of items such as lateral movement of the pelvis, lateral inclination of the pelvis, bending angle of the knee joint, arm swing angle, presence / absence of walking aids, and the like are included.

However, if the walking characteristics indicated by the walking characteristics data 121 and the walking characteristics indicated by the analysis data 40 can be compared, the items of the walking characteristics included in the walking characteristics data 121 and the items of the walking characteristics included in the analysis data 40 It does not have to match completely. That is, the walking characteristic data 121 and the analysis data 40 only need to include a common walking characteristic item, and a walking characteristic item that is not included in one data may be included in the other data.

The physical condition specifying unit 33 compares the analysis data 40 acquired in this way with the walking characteristic data 121 held in the storage unit 12 to determine whether or not a change satisfying a predetermined condition occurs in the user's walking characteristic. Determine whether. Then, the physical condition specifying unit 33 specifies the physical condition of the user shown in the captured image 3 based on the result of the determination.

Here, in the case where a plurality of users are to be subjected to physical condition detection, after the user shown in the captured image 3 is specified, the physical condition detection process of each user by the physical condition specifying unit 33 can be executed. Therefore, the physical condition detection device 1 according to the present embodiment further includes a user specifying unit 34 in order to specify the user shown in the captured image 3.

The user specifying unit 34 collates the analysis data 40 with the walking characteristic data 121. In other words, the user specifying unit 34 compares the walking characteristic indicated by the walking characteristic data 121 with the walking characteristic indicated by the acquired analysis data 40 to determine whether the acquired analysis data 40 matches the walking characteristic data 121. Determine. And the user specific | specification part 34 specifies the user who appears in the picked-up image 3 based on the result of the said determination. However, the method for specifying the user may not be limited to the method based on the walking characteristics of the user, and may be appropriately selected according to the embodiment. For example, the user may be specified by face authentication.

In the present embodiment, an example is described in which all of these functions are realized by a general-purpose CPU. However, some or all of these functions may be realized by one or more dedicated processors. In addition, regarding the functional configuration of the physical condition detection device 1, the omission, replacement, and addition of functions may be appropriately performed according to the embodiment. For example, when the user need not be identified, the user specifying unit 34 may be omitted. Each function will be described in detail in an operation example described later.

§3 Example of Operation Next, an example of the operation of the physical condition detection device 1 will be described with reference to FIG. FIG. 7 illustrates a processing procedure related to the physical condition detection of the physical condition detection device 1. In addition, the process procedure regarding the physical condition detection demonstrated below is only an example, and each process may be changed as much as possible. Further, in the processing procedure described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.

(Step S101)
In step S 101, the control unit 11 functions as the image acquisition unit 31 and acquires the captured image 3 captured by the camera 2. Then, the control part 11 advances a process to following step S102. Here, in the present embodiment, the camera 2 includes a depth sensor 21. Therefore, the captured image 3 acquired in step S101 includes depth data indicating the depth of each pixel. For example, the control unit 11 acquires the captured image 3 illustrated in FIG. 8 as the captured image 3 including the depth data.

FIG. 8 shows an example of a photographed image 3 including depth data. The captured image 3 illustrated in FIG. 8 is an image in which the gray value of each pixel is determined according to the depth of each pixel. A black pixel is closer to the camera 2. On the other hand, a white pixel is farther from the camera 2. Based on the depth data, the control unit 11 can specify the position of each pixel in the real space. That is, the control unit 11 can specify the position in the three-dimensional space (real space) of the subject captured in each pixel from the coordinates (two-dimensional information) and the depth of each pixel in the captured image 3. .

Note that FIG. 8 illustrates depth data acquired by the camera 2 oriented in the horizontal direction with respect to the floor surface. However, the orientation of the camera 2 is not limited to such an example, and can be set as appropriate according to the embodiment. When the camera 2 is inclined with respect to the floor surface, a three-dimensional distribution in the real space of the subject shown in the captured image 3 is obtained by performing projective transformation using the roll angle, pitch angle, and yaw angle of the camera 2. Obtainable.

Further, the control unit 11 may acquire the captured image 3 in synchronization with the video signal of the camera 2 in order to detect the physical condition of the user in real time. Then, at the stage where one or a plurality of photographed images 3 are acquired, the control unit 11 may immediately execute the processing of steps S102 to S107 described later on the one or a plurality of photographed images 3 obtained. Good. The physical condition detection apparatus 1 can perform real-time image processing by continuously executing such an operation continuously, and can detect the physical condition of the user existing in the imaging range of the camera 2 in real time.

(Step S102)
Returning to FIG. 7, in the next step S 102, the control unit 11 functions as the walking characteristic analysis unit 32 and extracts a region in which the user is captured in the captured image 3 acquired in step S 101. Then, the control part 11 advances a process to following step S103. Hereinafter, for convenience of explanation, an area in which the user is captured in the captured image 3 is also referred to as a “person area”.

The method for extracting the person area can be appropriately selected according to the embodiment. For example, the control unit 11 may extract a person region in which the user is photographed in the captured image 3 by performing image analysis such as pattern detection and graphic element detection based on the shape of the person. When a plurality of users are shown in the captured image 3, the control unit 11 extracts a person area for each user.

(Step S103)
In the next step S103, the control unit 11 functions as the walking characteristic analysis unit 32, and based on the depth of each pixel in the captured image 3 indicated by the depth data, the real space of the user shown in the region extracted in step S102 Analyze walking characteristics in

Specifically, as described above, the control unit 11 can recognize the three-dimensional distribution of the subject by using the depth data. Therefore, the control unit 11 may specify the positional relationship in the real space of each body part of the user shown in the person area by referring to the depth of the person area extracted in step S102.

And the control part 11 may calculate the attribute value (analysis value) which concerns on each item of the said user's walk characteristic from the positional relationship in the real space of each body part. Thereby, the control part 11 can acquire the analysis data 40 which shows the walk characteristic in the user's real space reflected in the picked-up image 3, and can advance a process to following step S104.

It should be noted that the method for specifying the region of each body part in the person region can be appropriately selected according to the embodiment. For example, similarly to the extraction of a person region, the region of each body part can be specified in the person region by performing image analysis such as pattern detection and graphic element detection based on the shape of each body part.

Also, the analysis method for each item of walking characteristics can be selected as appropriate according to the embodiment. For example, it is possible to analyze the walking characteristics of the user shown in the captured image 3 and calculate the analysis value of each item as follows.

For example, the control unit 11 may calculate the height (analysis value) of the user by measuring the distance from the ground (heel or toe) to the head. Further, for example, the control unit 11 may calculate the user's stride (analysis value) by measuring the distance from the heel of one foot to the toe of the other foot. For example, the control unit 11 may calculate the user's step (analysis value) by measuring a horizontal distance between the left and right ankles.

Further, for example, the control unit 11 may calculate an analysis value related to how to raise the toes by continuously measuring the distance from the toes of at least one foot to the ground. Further, for example, the control unit 11 may calculate an analysis value related to how the heel rises by continuously measuring the distance from the heel of at least one foot to the ground. For example, the control unit 11 may calculate the cadence (analysis value) by measuring the number of steps per unit time.

For example, the control unit 11 may calculate the body inclination (analysis value) by measuring the inclination of the upper body with respect to a straight line perpendicular to the ground. Further, for example, the control unit 11 may calculate an analysis value related to the lateral movement of the body by continuously measuring the inclination of the body in the left-right direction with respect to a straight line perpendicular to the ground. Further, for example, the control unit 11 may calculate the analysis value related to the lateral inclination of the pelvis by measuring the horizontal inclination of the pelvis with respect to a straight line perpendicular to the ground.

In addition, for example, the control unit 11 calculates a knee joint bending angle (analyzed value) by measuring an angle with the knee formed between the thigh and the lower leg in at least one foot. May be. For example, the control unit 11 may calculate the arm swing angle (analysis value) by measuring the angle of the arm with respect to a straight line perpendicular to the ground.

Further, for example, the control unit 11 may detect the walking aid by performing image analysis such as pattern detection and graphic element detection in the person area based on the shape of each walking aid. When the walking assist device can be detected, the control unit 11 sets an analysis value related to the presence or absence of the walking assist device in the analysis data 40 so as to indicate that the walking assist device is included. On the other hand, when a walking aid is not detected, the control unit 11 sets an analysis value related to the presence or absence of the walking aid in the analysis data 40 so as to indicate that the walking aid is not provided. In addition, the control unit 11 may acquire the analysis values of the respective left-right ratios by calculating each analysis value separately for the right side and the left side and obtaining the ratio of the calculated analysis values.

Note that there are cases where the body part used for the analysis of the walking characteristics is not shown in the captured image 3 and the analysis value relating to the target item cannot be calculated. In this case, the control unit 11 treats the item of the walking characteristic for which the analysis value cannot be calculated as being incapable of analysis, and excludes it from the objects to be compared in the following steps S104 and S106. In step S102, when a plurality of users are extracted, in step S103, the control unit 11 analyzes walking characteristics for each user. And the control part 11 acquires the analysis data 40 for every user.

(Step S104)
Returning to FIG. 7, in the next step S 104, the control unit 11 functions as the user specifying unit 34 and collates the analysis data 40 acquired in step S 103 with the walking characteristic data 121 stored in the storage unit 12. Specifically, the control unit 11 compares the analysis value of the walking characteristic indicated by the analysis data 40 with the current value of the walking characteristic indicated by the walking characteristic data 121 for each item, and the analysis data 40 indicates that any individual's walking It is determined whether or not the characteristic data 121 is met.

In this comparison, when there are a predetermined number or more of items that can be evaluated as matching, the control unit 11 determines that the analysis data 40 acquired in step S103 matches the walking characteristic data 121 stored in the storage unit 12. To do. Then, the control unit 11 specifies that the user extracted in step S102 is the person indicated by the appropriate walking characteristic data 121, and proceeds to the next step S105.

On the other hand, in the comparison of walking characteristics, when there are no more than a predetermined number of items that can be evaluated if they match, the control unit 11 determines that the analysis data 40 does not match the walking characteristics data 121. Then, the control unit 11 treats the user extracted in step S102 as unidentifiable, and proceeds to the next step S105.

Here, regarding the comparison of each item, the method of comparing the analysis value and the current value can be appropriately selected according to the embodiment. For example, when each value is represented by a single numerical value, the control is performed when the difference (absolute value) between the analysis value indicated by the analysis data 40 and the current value indicated by the walking characteristic data 121 is within a threshold value. The unit 11 can evaluate that the analysis value and the current value in the target item match in the analysis data 40 and the walking characteristic data 121.

On the other hand, when the difference between the analysis value indicated by the analysis data 40 and the current value indicated by the walking characteristic data 121 exceeds the threshold, the control unit 11 determines whether the analysis value of the target item in the analysis data 40 and the walking characteristic data 121 is the current value. It can be evaluated that it does not match the value. Even when each value is represented by an array and when each value is represented by two numerical values of a period and a maximum value (maximum value), the control unit 11 represents each value by a single numerical value. It is possible to process in substantially the same manner as in the case of.

Note that the storage unit 12 may store a plurality of walking characteristic data 121. In this case, the control unit 11 collates the plurality of walking characteristic data 121 stored in the storage unit 12 with the analysis data 40 in order, and specifies the walking characteristic data 121 that matches the analysis data 40. When there are a plurality of walking characteristic data 121 that match the analysis data 40, the control unit 11 employs the most suitable walking characteristic data 121 as data that matches the analysis data 40. On the other hand, if all the walking characteristic data 121 do not match the analysis data 40, the control unit 11 treats the user extracted in step S102 as unidentifiable, and proceeds to the next step S105.

In addition, when a plurality of users are extracted in step S102, in step S104, the control unit 11 collates the analysis data 40 with the walking characteristic data 121 for each user. Thereby, the control part 11 specifies a person for every user extracted by step S102.

(Step S105)
In step S 105, the control unit 11 determines whether or not the user can be specified in step S 104, in other words, whether or not the walking characteristic data 121 that matches the analysis data 40 exists. If the user can be specified in step S104, the control unit 11 advances the process to the next step S106. On the other hand, when the user cannot be specified in step S104, the control unit 11 omits the processes in steps S106 and S107 and ends the process according to this operation example.

When a plurality of users are extracted in step S102, the control unit 11 sets the user to be unidentifiable and performs the process according to this operation example only when a person cannot be specified for all the extracted users. finish. On the other hand, when a person can be specified for at least some users, the control unit 11 proceeds to the next step S106 for the user who can specify a person.

(Step S106)
In step S106, the control unit 11 functions as the physical condition specifying unit 33, and compares the analysis data 40 acquired in step S103 with the user's walking characteristic data 121 specified in step S104. Specifically, the control unit 11 compares the analysis value of the walking characteristic indicated by the analysis data 40 with the normal value of the walking characteristic indicated by the walking characteristic data 121 for each item, and changes the user's walking characteristic satisfying a predetermined condition. Whether or not has occurred. Then, the control unit 11 specifies the physical condition of the user shown in the captured image 3 based on the determination result, and proceeds to the next step S107.

Note that the relationship between the change in walking characteristics and the physical condition of the user can be set as appropriate according to the embodiment. For example, changes in the physical condition of the user can be detected as shown in (a) to (f) below.

(A) Detection of physical condition change based on step distance For example, it is known that the step length is increased due to peripheral neuropathy such as diabetes that causes balance disorder or sensory disorder due to cerebellar or inner ear diseases. Correspondingly, a predetermined condition may be set for the increment of the step.

In this case, for example, when the analysis value of the step included in the analysis data 40 is larger than the normal value of the step included in the walking characteristic data 121, the control unit 11 determines that the user's walking characteristic has a predetermined condition. It is determined that there is a change that satisfies And according to this, the control part 11 detects the deterioration of the health state by said each disorder | damage | failure, and specifies that a user's physical condition is a bad state by these factors.

On the other hand, when that is not right, the control part 11 determines with the change which satisfy | fills a predetermined condition not having arisen in the user's walking characteristic. And according to this, the control part 11 detects the favorable (normal) health condition, and specifies that a user's physical condition is a favorable state.

(B) Detection of physical condition change based on stride It is also known that stride is reduced due to, for example, aging, fatigue, pain, disease, and the like. In response to this, for example, a predetermined condition may be set regarding the amount of decrease in the stride.

In this case, for example, when the analysis value of the stride included in the analysis data 40 is smaller than the threshold value than the normal value of the stride included in the walking characteristic data 121, the control unit 11 satisfies a predetermined condition for the walking characteristic of the user. It is determined that a change has occurred. And according to this, control part 11 detects deterioration of health conditions, such as aging, fatigue, pain, and a disease, and specifies that a user's physical condition is in a bad state by these factors.

(C) Detection of physical condition change based on left / right ratio of stride Also, for example, when there is pain in one of the lower limbs, the walking distance is different between painful and painless lower limbs as much as possible to escape from pain It is known that the rhythm of walking changes. Corresponding to this, for example, a predetermined condition may be set regarding the amount of change in the left-right ratio of the stride.

In this case, for example, when the analysis value of the step ratio included in the analysis data 40 is shifted from the normal value of the step ratio included in the walking characteristic data 121 by more than a threshold, It is determined that a change satisfying a predetermined condition has occurred in the walking characteristics. And according to this, the control part 11 detects that the pain has arisen in one leg (lower leg with a smaller stride), and specifies that a user's physical condition is in a bad state by these factors. .

(D) Detection of physical condition change based on knee joint bending angle Further, it is known that knee extension is restricted when, for example, a swaying gait occurs due to a disorder such as a lumbar muscle. In response to this, for example, a predetermined condition may be set regarding the amount of change in the bending angle of the knee joint.

In this case, for example, when the analysis value of the knee joint bending angle included in the analysis data 40 is smaller than the threshold value than the normal value of the knee joint bending angle included in the walking characteristic data 121, the control unit 11 It is determined that a change satisfying a predetermined condition has occurred in the gait characteristics. And according to this, the control part 11 detects the deterioration of the health state by obstacles, such as a pelvic muscle, and specifies that a user's physical condition is a bad state by these factors.

(E) Detection of physical condition change based on how toe rises Further, for example, it is known that the rise of the toes becomes small due to fatigue, a decrease in muscle strength, or the like. Corresponding to this, for example, a predetermined condition may be set regarding the amount of change in how the toes rise.

In this case, for example, when the analysis value 40 of the toe included in the analysis data 40 is smaller than the normal value of the method of rising the toe included in the walking characteristic data 121, the control unit 11 walks the user. It is determined that a change that satisfies a predetermined condition has occurred in the characteristics. And according to this, the control part 11 detects the deterioration of health conditions, such as fatigue and a fall of muscular strength, and specifies that a user's physical condition is a bad state by these factors.

(F) Detection of changes in physical condition based on body tilt etc. For example, if there is a suspicion of multiple system atrophy such as Parkinson's disease, the body is tilted forward, the body trembles in small steps, the arm during walking It is known that symptoms such as less swing, walking with a slack leg, and accelerated walking appear. Corresponding to this, for example, a predetermined condition may be set regarding the amount of change in the tilt of the body.

In this case, for example, when the analysis value of the body inclination included in the analysis data 40 is inclined more than a threshold value than the normal value of the body inclination included in the walking characteristic data 121, the control unit 11 walks the user. It is determined that a change that satisfies a predetermined condition has occurred in the characteristics. And according to this, the control part 11 detects the suspicion of multiple system atrophy such as Parkinson's disease, and specifies that the user's physical condition is in a bad state based on these factors.

On the other hand, when that is not right, the control part 11 determines with the change which satisfy | fills a predetermined condition not having arisen in the user's walking characteristic. And according to this, the control part 11 detects the favorable (normal) health condition, and specifies that a user's physical condition is a favorable state. Other symptoms can be determined by using other items. For example, it can be determined whether or not the body is trembling in small increments based on the lateral fluctuation of the trunk.

(G) Others In the present embodiment, the control unit 11 employs at least one of the methods (a) to (f) described above to determine whether or not a change satisfying a predetermined condition occurs in the user's walking characteristics. The physical condition of the user is specified based on the determination result. However, the method for determining a change in walking characteristics is not limited to the methods described in (a) to (f) above, and can be selected as appropriate according to the embodiment. Further, in the above (a) to (f), conditions for detecting deterioration of physical condition are set, but the conditions that can be set are not limited to such an example, and recovery of physical condition is detected. May be for. Note that the threshold value in the above conditions (a) to (f) can be set as appropriate.

In addition, when the deterioration of the user's physical condition is detected, the control unit 11 functions as a notification unit when the deterioration of the user's physical condition in the captured image 3 is detected. Notification may be performed. Note that the means for the control unit 11 to perform the notification can be appropriately selected according to the embodiment.

For example, when the physical condition detection device 1 is used in a facility such as a hospital, the physical condition detection device 1 can be connected to equipment such as a nurse call system via the external interface 15. In this case, the control unit 11 may perform notification for notifying that the deterioration of the physical condition of the user has been detected in cooperation with equipment such as the nurse call system. That is, the control unit 11 may control the nurse call system via the external interface 15. And the control part 11 may perform the call by the said nurse call system as alert | reporting for notifying deterioration of a user's physical condition. It is possible to appropriately notify a nurse or the like who watches over the user that his / her physical condition has deteriorated.

Further, for example, the control unit 11 may perform notification to notify that the deterioration of the user's physical condition is detected by outputting a predetermined sound from the speaker 14 connected to the physical condition detection device 1. When this speaker 14 is arranged around the camera 2, such notification is given by the speaker 14 to notify the user in the shooting range that the deterioration of his / her physical condition has been detected. Can do.

For example, the control unit 11 may display a screen on the touch panel display 13 for notifying that the deterioration of the user's physical condition has been detected. In addition, for example, the control unit 11 may perform such notification using an e-mail, a short message service, a push notification, or the like. When performing such notification, the e-mail address, telephone number, etc. of the user terminal as the notification destination may be registered in advance in the storage unit 12, and the control unit 11 Notification may be made to notify that the deterioration of the physical condition of the user himself / herself has been detected using an address, a telephone number, or the like.

(Step S107)
In step S 107, the control unit 11 performs a process for updating the current value of the walking characteristic data 121. Specifically, the control unit 11 determines the current value of each item of the walking characteristic included in the user's walking characteristic data 121 specified in step S104, and each item of the walking characteristic included in the analysis data 40 acquired in step S103. Replace with the analysis value. Thereby, the memory | storage part 12 updates the user's walking characteristic data 121 specified by step S104.

When the processing in step S107 is completed, the control unit 11 ends the processing according to this operation example. When the process related to the physical condition detection is repeated, the control unit 11 executes the process from step S101 again. In addition, about the item which cannot be analyzed in step S103, the control part 11 does not update a present value, but maintains the present value contained in the original walking characteristic data 121. FIG.

It should be noted that when the walking characteristic data 121 is updated in this step S107, an item having a change that satisfies a predetermined condition may be recorded in the walking characteristic data 121. When a series of processes in steps S101 to S107 are repeatedly executed to continuously detect the user's physical condition, a similar result is likely to be obtained in the range of adjacent processes. Therefore, it is highly likely that the item determined to have changed in the predetermined condition in step S106 is determined to have changed in the predetermined condition even in the subsequent processing. Therefore, an item that has changed according to the predetermined condition may be recorded in the walking characteristic data 121, and the comparison process in step S106 from the next time onward may be omitted.

(Action / Effect)
As described above, the physical condition detection device 1 according to the present embodiment acquires depth data indicating the depth of each pixel together with the captured image 3 in order to analyze the user's walking characteristics. By using this depth data, the physical condition detection device 1 can analyze the state of the user in the real space regardless of the direction of movement of the user with respect to the camera 2. Therefore, according to the present embodiment, it is possible to analyze the state of the user in the real space without depending on the installation location of the camera 2, so that it can be introduced relatively easily, and the walking characteristics of the user can be improved. The user's physical condition can be detected after sufficiently extracting.

More specifically, it is difficult to specify the depth in the shooting direction in a two-dimensional image. Therefore, when analyzing a user's walking characteristics using a two-dimensional image, for example, when the user is photographed from the side, it is difficult to analyze the user's step. However, according to the present embodiment, depth data indicating the depth of each pixel is acquired together with the captured image 3. Therefore, even if the user is photographed from the side, the depth of the subject can be specified, and the user's step can be analyzed. Therefore, according to the present embodiment, it is possible to stably analyze the walking characteristics of the user in the real space shown in the captured image 3 regardless of the shooting direction of the camera 2, thereby detecting the physical condition of the user. be able to.

Further, in the present embodiment, in addition to detecting the user's physical condition, the user shown in the captured image 3 is specified based on the user's walking characteristics. Therefore, when a plurality of users are subject to physical condition detection, a process for identifying the user can be performed in association with the physical condition detection process of the user. That is, according to the present embodiment, it is possible to identify a user who is subject to physical condition detection without providing a means for identifying the user such as face authentication.

Also, human walking characteristics can change over time. Therefore, when the user identification process and the physical condition detection process are continuously executed using the same data (value), there is a high possibility that a problem will occur in either process.

That is, when the walking characteristic data 121 is adapted to the user identification process, the attribute value of each item included in the walking characteristic data 121 is changed so as to correspond to the change in the walking characteristic of the user. . Therefore, in this case, information serving as a reference for the user's health condition cannot be obtained, and the user's physical condition cannot be detected.

On the other hand, when the walking characteristic data 121 is adapted to the user's physical condition detection process, the attribute value of each item included in the walking characteristic data 121 is maintained without being changed so as to indicate the user's health condition standard. Will continue. Therefore, in this case, it is not possible to cope with a change in the walking characteristics of the user with time, and the user cannot be identified.

On the other hand, in this embodiment, the walking characteristic data 121 includes two types of attribute values, a normal value used for physical condition detection and a current value used for user identification, as attribute values of each item of walking characteristics. Is included. Therefore, according to the present embodiment, the attribute value used for the user identification process is separated from the attribute value used for the user physical condition detection process, and it is avoided that the same value is continuously used in both processes. Therefore, it is possible to prevent the above processing from occurring.

In the present embodiment, the current value included in the walking characteristic data 121 is updated using the analysis data 40 obtained when the user is successfully identified in step S104. Therefore, the current value used for the user identification can be updated so as to correspond to the change of the walking characteristic with the passage of time without separately providing a means for acquiring the current value of each item of the walking characteristic. .

In addition, the physical condition detection apparatus 1 according to the present embodiment detects the physical condition of the user using the captured image 3 obtained by the camera 2. Therefore, any place where the camera 2 can be installed can be a target range for detecting the user's physical condition. For example, by installing the camera 2 at an entrance, a room, a hallway, or the like of a facility, these places can be used as places where the user's physical condition is detected.

That is, the work of introducing the physical condition detection system according to the present embodiment configured by the information processing apparatus (physical condition detection apparatus 1) and the camera 2 is nothing other than the work of installing the camera 2. Here, since the camera 2 includes the depth sensor 21 and can acquire three-dimensional information, the installation location and shooting direction of the camera 2 do not have to be determined strictly. Therefore, the physical condition detection system according to the present embodiment can be introduced very easily.

§4 Modifications Embodiments of the present invention have been described in detail above, but the above description is merely an illustration of the present invention in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention.

(1) Detection of sign of physical condition change In the above embodiment, in step S106, the control unit 11 determines a predetermined value between the analysis value of the walking characteristic indicated by the analysis data 40 and the normal value of the walking characteristic indicated by the walking characteristic data 121. When there is a difference that satisfies the condition, a change in the physical condition of the target user is detected. Therefore, when there is no difference that satisfies the predetermined condition between the analysis value and the normal value, the control unit 11 does not detect a change in the physical condition of the target user, and the physical condition of the target user is good. Identify.

However, changes in the user's physical condition may occur in stages. In order to cope with this, the control unit 11 may determine whether or not there is a sign of a change in the subject, together with the physical condition change detection process in step S106. This can be realized by repeating a series of processes in steps S101 to S107.

That is, the control unit 11 continuously and repeatedly executes the process of detecting the user's physical condition by repeating a series of processes of steps S101 to S107. Then, the control unit 11 compares the analysis data 40 with the walking characteristic data 121 in accordance with the repeated acquisition of the analysis data 40 in step S103, thereby causing a change that satisfies a predetermined condition in the user's walking characteristics. That is, the process of step S106 for determining whether or not it is repeated is executed.

Here, as described above, when it is determined that a change that satisfies the predetermined condition has occurred in the walking characteristics of the user, a change in the physical condition of the user can be detected. On the other hand, while it is determined that the change satisfying the predetermined condition has not occurred in the walking characteristic of the user, the control unit 11 determines that the analysis data 40 repeatedly acquired satisfies the predetermined condition of the walking characteristic of the user. It is further determined whether or not it has changed in the direction in which it is determined that it has occurred. When the control unit 11 determines that the analysis data 40 that is repeatedly acquired has changed in a direction in which it is determined that a change that satisfies the predetermined condition has occurred in the walking characteristics of the user, the user Can detect signs of physical condition changes.

For example, in the example of (a), the control unit 11 changes the physical condition of the user because the analysis value of the stride included in the analysis data 40 is smaller than the normal value of the stride included in the walking characteristic data 121 by a threshold value or more. Is detected. Therefore, in this case, when the analysis value of the stride included in each analysis data 40 repeatedly acquired is not smaller than the normal value of the stride included in the walking characteristic data 121, the control unit 11 It is determined whether or not the analysis value is gradually reduced. Then, when the analysis value of the stride included in each analysis data 40 repeatedly acquired is gradually reduced, in other words, the control unit 11 analyzes the analysis value of the stride so that it is smaller than the normal value of the stride by a threshold value or more. Is detected, a sign of a change in the physical condition of the user is detected. On the other hand, when the analysis value of the acquired stride does not have such a tendency, the control unit 11 determines that the user has no sign of physical condition change.

According to the present modification, the control unit 11 grasps the change tendency of the analysis data in the repeated user physical condition detection process, and detects a sign of the user physical condition change. Thereby, it is possible to grasp the sign of the change in the physical condition of the user in advance. That is, according to the present modification, changes in the user's physical condition can be grasped step by step.

(2) Use of different items In the above-described embodiment, the item of walking characteristics used for user identification processing and the item of walking characteristics used for physical condition detection are common. However, the item of the walking characteristic used for the user identification process and the item of the walking characteristic used for the physical condition detection may not be common, and some or all of them may be different. For example, in the above-described embodiment, items such as the stride and the step are used for physical condition detection among a plurality of items included in the walking characteristics. Therefore, users may be identified based on items that are not used for physical condition detection, except for items that are used for physical condition detection.

In other words, in step S104, the control unit 11 compares the analysis data 40 and the walking characteristic data 121 with respect to at least one of the plurality of items included in the walking characteristic data 121, and is captured in the captured image 3. User identification may be performed. Then, in step S106, the control unit 11 compares the analysis data 40 and the walking characteristic data 121 with respect to one or more items different from the items used in step S104, so that the physical condition of the user shown in the captured image 3 is obtained. May be detected.

In this case, since the common item is not used, the walking characteristic data 121 may not include two types of attribute values, that is, a normal value and a current value. Moreover, the control part 11 may abbreviate | omit the update process of step S107, and may limit the object of the update process of step S107 to the item utilized for a user's identification. However, when the update process of step S107 is omitted, it is preferable to adopt an item that is relatively difficult to change as an item used for user identification.

As mentioned above, human walking characteristics can change over time. Therefore, if the same value for the same item is continuously used in both processes, there is a high possibility that a problem will occur in either the user identification process or the physical condition detection process. On the other hand, in this modification, the item of the walk characteristic used for a user's identification process and the item of the walk characteristic used for a user's physical condition detection process are separated. Therefore, it is possible to avoid problems caused by continuing to use the same value for both processes in the same item.

(3) Use of Foreground Area In the above embodiment, the control unit 11 (walking characteristic analysis unit 32) extracts a person area in which the user is photographed from the photographed image 3 by image processing such as pattern detection in step S102. Yes. However, the method of extracting the person region from the captured image 3 is not limited to such an example, and can be appropriately selected according to the embodiment. For example, the control unit 11 may use the foreground area to narrow the range for searching for the person area.

In this case, in step S102, the control unit 11 extracts the foreground region of the captured image 3 from the difference between the background image set as the background of the captured image 3 acquired in step S101 and the captured image 3. Here, the background image is data used for extracting the foreground region, and is set including the depth of the target as the background. The method for creating the background image may be set as appropriate according to the embodiment. For example, the control unit 11 may create the background image by calculating the average of the captured images for several frames obtained when the camera 2 starts shooting. At this time, a background image including the depth data can be created by calculating the average of the captured images including the depth data.

The foreground area extracted in this way indicates an area where a change has occurred from the background image. Therefore, when a person (user) who is performing a walking motion is shown in the captured image 3, an area where the user is shown is extracted as a foreground area. Therefore, in step S102, the control unit 11 may extract an area in which the user is captured by executing image analysis such as pattern detection and graphic element detection in the foreground area.

Thereby, it is possible to limit the area in the captured image 3 where the user is detected. That is, the area to be subjected to image analysis such as pattern matching can be limited to the foreground area instead of the entire area within the captured image 3. Here, the process for extracting the foreground region is merely a process for calculating the difference between the captured image 3 and the background image. Therefore, according to this embodiment, the control part 11 (physical condition detection apparatus 1) can narrow down the range which detects a user, without utilizing an advanced image process. Therefore, according to the modification, the processing load in step S102 can be reduced.

Note that the method by which the control unit 11 extracts the foreground region is not limited to the above-described method. For example, the background and the foreground may be separated using a background subtraction method. As the background difference method, for example, a method of separating the background and the foreground from the difference between the background image and the input image (captured image 3) as described above, and a method of separating the background and the foreground using three different images And a method of separating the background and the foreground by applying a statistical model. The method for extracting the foreground region is not particularly limited, and may be appropriately selected according to the embodiment.

1 ... Physical condition detection device,
2 ... Camera, 21 ... Depth sensor,
3 ... shot image, 5 ... program, 6 ... storage medium,
11 ... Control unit, 12 ... Storage unit, 13 ... Touch panel display,
14 ... Speaker, 15 ... External interface, 16 ... Communication interface,
17 ... drive,
31 ... Image acquisition unit, 32 ... Walking characteristic analysis unit, 33 ... Physical condition specifying unit, 34 ... User specifying unit,
40: Analysis data, 121: Walking characteristic data

Claims

A storage unit for storing walking characteristic data indicating walking characteristics in the real space of the user;
An image acquisition unit that acquires a captured image that is captured by a capturing device installed to capture the user's walking motion and includes depth data indicating the depth of each pixel in the captured image;
By extracting a region in which the user is captured in the captured image, and analyzing a walking motion in the real space of the user that is captured in the region based on the depth of each pixel in the captured image indicated by the depth data. A walking characteristic analysis unit that acquires analysis data indicating the walking characteristic in the real space of the user that is reflected in the captured image;
By comparing the walking characteristic indicated by the walking characteristic data and the walking characteristic indicated by the acquired analysis data, it is determined whether or not a change satisfying a predetermined condition has occurred in the walking characteristic of the user. Based on the physical condition specifying unit for specifying the physical condition of the user in the captured image,
Comprising
Physical condition detection device.
The physical condition specifying unit is
As the analysis data is repeatedly acquired, by comparing the acquired analysis data with the walking characteristic data, it is repeatedly determined whether or not a change satisfying a predetermined condition occurs in the user's walking characteristic. Judgment,
While it is determined that a change satisfying a predetermined condition has not occurred in the user's walking characteristics, the analysis data repeatedly acquired determines that a change satisfying the predetermined condition has occurred in the user's walking characteristics Whether or not the direction is changed, and the analysis data repeatedly acquired is changed to a direction in which it is determined that a change satisfying a predetermined condition occurs in the walking characteristics of the user. When it is determined that there is a sign of a change in the physical condition of the user,
The physical condition detection apparatus according to claim 1.
The storage unit stores the walking characteristic data for each individual,
By comparing the walking characteristics indicated by the walking characteristics data of each individual and the walking characteristics indicated by the acquired analysis data, it is determined whether or not the acquired analysis data matches the walking characteristics data of any individual. If the analysis data matches the walking characteristic data of any one of the individuals, a user specifying unit that specifies that the user shown in the captured image is a person indicated by the adapted walking characteristic data Prepare
The physical condition detection device according to claim 1 or 2.
The walking characteristic data stored in the storage unit includes a normal value and a current value of the user's walking specification,
The analysis data acquired by the walking characteristic analysis unit includes an analysis value of the walking characteristic,
The user specifying unit specifies a user who appears in the captured image by comparing a current value included in the walking characteristic data and an analysis value included in the analysis data,
The physical condition specifying unit specifies the physical condition of the user by comparing a normal value included in the walking characteristic data and an analysis value included in the analysis data.
The physical condition detection apparatus of Claim 3.
The storage unit replaces the current value of the walking characteristic included in the user's walking characteristic data specified by the user specifying unit with the analysis value of the analysis data, thereby including the walking characteristic included in the specified user's walking characteristic data. Update the current value of,
The physical condition detection device according to claim 4.
The walking characteristic is composed of a plurality of items,
The user specifying unit specifies a user in the captured image by comparing the acquired analysis data with the walking characteristic data in at least one of a plurality of items included in the walking characteristic. And
The physical condition specifying unit specifies the physical condition of the user by comparing the analysis data and the walking characteristic data in one or more items different from the items used by the user specifying unit.
The physical condition detection apparatus of Claim 3.
The walking characteristic analysis unit extracts a foreground region of the photographed image as a region in which the user is captured from a difference between a background image set as a background of the photographed image and the photographed image, and refers to the depth data. Identifying the depth of each pixel in the foreground area, and analyzing the walking motion of the user in the foreground area in the foreground area based on the depth of each pixel in the specified foreground area,
The physical condition detection apparatus of any one of Claim 1 to 6.
A computer including a storage unit that stores walking characteristic data indicating walking characteristics in a user's real space,
A captured image captured by a capturing device installed to capture the user's walking motion, the captured image including depth data indicating the depth of each pixel in the captured image;
Extracting an area in which the user appears in the captured image;
Based on the depth of each pixel in the captured image indicated by the depth data, the walking characteristics of the user in the real space captured in the captured image are analyzed by analyzing the walking motion of the user in the real space captured in the region. Obtaining analysis data indicating
Determining whether or not a change satisfying a predetermined condition occurs in the walking characteristics of the user by comparing the walking characteristics indicated by the walking characteristics data and the walking characteristics indicated by the acquired analysis data;
Identifying the physical condition of the user in the captured image based on the result of the determination;
The physical condition detection method to execute.
A computer having a storage unit that stores walking characteristic data indicating walking characteristics in the user's real space,
A captured image captured by a capturing device installed to capture the user's walking motion, the captured image including depth data indicating the depth of each pixel in the captured image;
Extracting an area in which the user appears in the captured image;
Based on the depth of each pixel in the captured image indicated by the depth data, the walking characteristics of the user in the real space captured in the captured image are analyzed by analyzing the walking motion of the user in the real space captured in the region. Obtaining analysis data indicating
Determining whether or not a change satisfying a predetermined condition occurs in the walking characteristics of the user by comparing the walking characteristics indicated by the walking characteristics data and the walking characteristics indicated by the acquired analysis data;
Identifying the physical condition of the user in the captured image based on the result of the determination;
Physical condition detection program for running.