CN113367658A - Biological information processing device and biological information processing method - Google Patents

Biological information processing device and biological information processing method Download PDF

Info

Publication number
CN113367658A
CN113367658A CN202110195726.5A CN202110195726A CN113367658A CN 113367658 A CN113367658 A CN 113367658A CN 202110195726 A CN202110195726 A CN 202110195726A CN 113367658 A CN113367658 A CN 113367658A
Authority
CN
China
Prior art keywords
unit
information
biological information
display
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110195726.5A
Other languages
Chinese (zh)
Inventor
足立佳久
岩井敬文
昌谷洋
江户勇树
小川莉绘子
富泽亮太
奥村哲也
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sharp Corp
Original Assignee
Sharp Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Corp filed Critical Sharp Corp
Publication of CN113367658A publication Critical patent/CN113367658A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface

Abstract

The biological information processing device includes: a display instruction unit that generates an instruction signal for causing a display unit to display predetermined information in a predetermined display format; a biological information measuring unit that measures biological information of the user based on sensor information indicating a change in state of a biological body of the user acquired by the sensor unit after the display instruction unit generates the instruction signal; a relative velocity calculation unit that calculates a relative velocity between the display unit and the user during a period in which the biological information measurement unit measures the biological information; and a determination unit that determines whether or not the relative velocity calculated by the relative velocity calculation unit is within a predetermined range.

Description

Biological information processing device and biological information processing method
Technical Field
One aspect of the present disclosure relates to a biological information processing apparatus and a biological information processing method.
Background
Japanese patent application laid-open No. 2017-93760 discloses a measurement device for measuring a pulse rate of a subject from a camera image obtained by photographing the subject with a camera.
Documents of the prior art
Patent document
Patent document 1: japanese laid-open patent publication No. 2017-93760
Disclosure of Invention
According to the measuring apparatus disclosed in japanese patent application laid-open No. 2017-93760, there is a case where a subject who moves a face or the like quickly moves so that the relative speed of the subject with respect to a camera that photographs the subject becomes faster. In this case, the accuracy of the pulse rate measured from the camera image obtained by imaging the subject with the camera becomes low. An object of an aspect of the present disclosure is to obtain biometric information of a user with high accuracy by suppressing the speed at which the user moves.
A biological information processing apparatus according to an aspect of the present disclosure includes: a display instruction unit that generates an instruction signal for causing a display unit to display predetermined information in a predetermined display format; a biological information measuring unit that measures biological information of the user based on sensor information indicating a change in state of a biological body of the user acquired by the sensor unit after the display instruction unit generates the instruction signal; a relative velocity calculation unit that calculates a relative velocity between the display unit and the user during a period in which the biological information measurement unit measures the biological information; and a determination unit that determines whether or not the relative velocity calculated by the relative velocity calculation unit is within a predetermined range.
A biological information processing method according to an aspect of the present disclosure includes: generating an instruction signal for causing a display unit to display predetermined information in a predetermined display format; measuring biometric information of the user based on sensor information of a change in state of a biometric object of the user acquired by a sensor unit after the instruction signal is generated; calculating a relative speed between the sensor unit and the user during a period in which the biometric information is measured; and a step of determining whether or not the relative speed is within a predetermined range.
Drawings
Fig. 1 is a front view showing a mobile terminal according to a first embodiment.
Fig. 2 is a functional block diagram of the biological information processing system according to the first embodiment.
Fig. 3 is a diagram showing a display image of the health management application displayed on the display unit by the display instruction unit of the portable terminal according to the first embodiment.
Fig. 4 is a diagram showing an example of a moving image immediately before a user moves among moving images captured by the imaging unit according to the first embodiment.
Fig. 5 is a diagram showing an example of a moving image after a user has moved among moving images captured by the imaging unit according to the first embodiment.
Fig. 6 is a diagram showing an example of a waveform of a pulse wave measured by the biological information measurement unit based on a moving image captured by the imaging unit according to the first embodiment.
Fig. 7 is a diagram showing a change in biological information monitored by the change amount monitoring unit according to the first embodiment.
Fig. 8 is a flowchart showing an example of the operation of the control unit of the mobile terminal according to the first embodiment.
Fig. 9 is a functional block diagram of a biological information processing system including a portable terminal according to a second embodiment.
Fig. 10 is a flowchart showing an example of the operation of the control unit of the mobile terminal according to the second embodiment.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description thereof is omitted.
[ first embodiment ]
Fig. 1 is a front view of a mobile terminal 1 according to a first embodiment. That is, fig. 1 shows a plan view of the mobile terminal 1. The portable terminal 1 is an example of a biological information processing device that contactlessly measures biological information of a person.
The biological information is various information related to a living body that can be measured from a body surface of a person, and examples thereof include a pulse wave, a blood pressure value, a pulse rate, a pressure level, a heart rate, a body temperature, a respiration rate, a blood oxygen saturation level, and information indicating a health state of a blood vessel. The mobile terminal 1 obtains information including at least one of a blood pressure value, a pulse rate, a heart rate, a respiration rate, a body temperature, and a blood oxygen saturation level as biological information based on information (sensor information) indicating a state change of a body surface such as a moving image reflecting the body surface of a user (person).
An example of the information indicating the state change of the body surface is a moving image reflecting the body surface. The dynamic image reflecting the body surface shows, for each frame, a change in the state of the body surface, such as a change in the color of the body surface, a change in the body surface temperature, and a change in the movement of the body surface due to respiration, based on the flow of blood of the user. In the following, as an example, information showing a change in the state of a body surface (sensor information) is described as information mainly reflecting a moving image of the body surface, but the information showing a change in the state of a body surface (sensor information) may be information showing body motion based on the heartbeat of the user obtained by a doppler sensor or the like, for example, and is not limited to the moving image reflecting the body surface.
In the present embodiment, for example, when the user receives an instruction to activate a specific application (software), the mobile terminal 1 activates the specific application to display an image such as a moving image or a still image including predetermined information in a predetermined display format, which will be described later. For example, when the specific application is started, the mobile terminal 1 starts measurement of the biometric information of the user at the same time.
The specific application refers to, for example, an application containing information of a content of interest to the user. The information of the content in which the user is interested is, for example, information for managing a change in daily biometric information of the user, information related to daily news, and the like.
In this way, when measuring the biological information of the user, the mobile terminal 1 displays an image such as a moving image or a still image including predetermined information in a predetermined display format, thereby suppressing the relative speed between the mobile terminal 1 and the user. This can improve the accuracy of measurement of the biometric information of the user. Further, the detailed description will be described later.
The mobile terminal 1 will be described with a smartphone as an example in the present embodiment. The portable terminal 1 has a size and a weight to such an extent that it can be carried by a person. The portable terminal 1 is, for example, of such a size as to be able to be held with one hand. The portable terminal 1 is not limited to a smartphone, and may be a tablet computer, a notebook computer, or the like.
The portable terminal 1 includes, for example, a housing 2, a display unit 3, an input unit 4, an imaging unit (sensor unit) 5, a speaker (sound output unit) 6, and a microphone 7. In the present embodiment, the display unit 3, the input unit 4, the imaging unit 5, the speaker 6, and the microphone 7 are provided in the same housing 2, for example.
The display unit 3 is a display for displaying images such as moving images and still images. The display unit 3 has a display area as an area for displaying images such as moving images and still images. The display unit 3 may be any display device capable of displaying images such as moving images and still images, such as a liquid crystal display and an organic EL (electro-luminescence) display.
The input unit 4 is an input device that receives an input, which is an instruction from a user, and generates an input signal based on the received input. The input unit 4 is, for example, a transparent touch sensor, and is provided so as to overlap with the display area of the display unit 3. That is, for example, the display unit 3 and the input unit 4 are integrated to constitute a touch panel. Then, for example, a contact of a finger of the user, a pen, or the like is detected, and an input signal corresponding to the detected position is generated. The input unit 4 is not limited to a touch sensor, and may be, for example, a mechanical switch that receives an input from a user when pressed by the user, a sensor that receives an input in a non-contact manner by sensing a distance or movement of a finger, a pen, or the like of the user, or the like.
The imaging unit 5 is an example of a sensor unit that acquires information (sensor information) indicating a change in state of a living body of a user in a non-contact manner. The imaging unit 5 is a camera (imaging device) for imaging images such as moving images and still images. The imaging unit 5 captures a moving image at a predetermined frame rate (for example, 10 to 300fps (frame per second)), thereby obtaining sensor information. For example, the imaging unit 5 images a moving image of the body surface of the user in order to measure the biological information of the user. For example, the imaging unit 5 also images a moving image of the body surface of the user in order to measure the relative speed between the mobile terminal 1 and the user.
The sensor unit according to the present disclosure is not limited to the imaging unit 5 that images a moving image, and may be various sensors that can acquire information (sensor information) indicating a change in state of a living body of a user in a non-contact manner, such as a doppler sensor such as a microwave radar or a millimeter wave radar. For example, even if a doppler sensor is used, sensor information indicating a change in the state of a living body of a user, such as sensor information indicating a change in the body motion of the user based on the heartbeat of the user, can be acquired.
The sensor unit according to the present disclosure may be configured by combining a plurality of sensors such as the imaging unit 5 and the doppler sensor, or may be configured by combining a plurality of sensors to obtain sensor information.
The sensor unit according to the present disclosure obtains sensor information for measuring biometric information of a user. Further, for example, the sensor section according to the present disclosure also obtains sensor information in order to measure the relative speed between the portable terminal 1 and the user.
The speaker 6 is a sound output device that generates sound based on a sound signal. The microphone 7 is, for example, a voice input device that receives, as an input, a voice outside the mobile terminal 1 such as a user's voice and converts the received voice into a voice signal.
Fig. 2 is a functional block diagram of a biological information processing system 100 including the mobile terminal 1 according to the first embodiment. As shown in fig. 2, the biological information processing system 100 includes, for example, a mobile terminal 1 and a server 200. For example, the mobile terminal 1 is configured to be able to communicate with the server 200 by wire or wirelessly. In the present embodiment, the mobile terminal 1 is configured to be able to communicate with the server 200 wirelessly via the internet 300. The portable terminal 1 further includes a control unit 10 as a computer, a storage unit 30, and a communication unit 40 in the housing 2 (see fig. 1).
Server 200 includes communication unit 210, control unit 220, and storage unit 230. The communication unit 210 in the server 200 and the communication unit 40 in the mobile terminal 1 are configured to be able to communicate via the internet 300, for example. The control Unit 220 is configured by a processor such as a CPU (Central Processing Unit), and collectively controls various functions of the server 200. The storage unit 230 can be implemented by, for example, a hard disk, an SSD (Solid State Drive), a semiconductor memory, or the like.
The control unit 10 includes a display instruction unit 11, a relative speed calculation unit 12, a biological information measurement unit 13, a time measurement unit 14, a determination unit 15, and a fluctuation amount monitoring unit 16. The control Unit 10 is constituted by a processor such as a CPU (Central Processing Unit), and collectively controls various functions of the mobile terminal 1.
The storage unit 30 is a storage medium that can store arbitrary information. The storage unit 30 can be realized by, for example, a hard disk, an SSD (Solid State Drive), a semiconductor memory, or the like. The storage unit 30 may be a storage medium fixed inside the mobile terminal 1 as a part of the mobile terminal 1, or may be a storage medium that can be taken out from the mobile terminal 1, such as a Sim card. The storage unit 30 stores a biological information processing program installed in advance or from a server (for example, the server 200) capable of communicating with the mobile terminal 1. The biological information processing program causes the control unit 10, which is a computer, to function as a display instruction unit 11, a relative speed calculation unit 12, a biological information measurement unit 13, a determination unit 15, a time measurement unit 14, and a fluctuation amount monitoring unit 16.
Further, for example, the storage unit 30 stores a specific program for starting a specific application which is installed in advance or from a server (for example, the server 200) capable of communicating with the mobile terminal 1 and which includes, as information, images such as moving images and still images of predetermined information in a predetermined display format. Then, for example, when an input signal indicating that a specific application is activated is received as an instruction from the user from the input unit 4, the control unit 10 refers to the storage unit 30, and executes the biological information processing program together with the specific program.
In the present embodiment, the control unit 10 is provided in the mobile terminal 1. However, the control unit 10 may be provided not in the mobile terminal 1 but in a server (for example, the server 200) communicably connected to the mobile terminal 1 by wire or wireless.
When an input signal is acquired from the input unit 4 as an instruction from the user, the display instruction unit 11 causes the display unit 3 to display various images such as a moving image and a still image based on the input signal. For example, the display instructing unit 11 generates an instruction signal for causing the display unit 3 to display predetermined information in a predetermined display format based on an input signal from the input unit 4. Then, the display instruction unit 11 outputs the generated instruction signal to the display unit 3. Thereby, the display instruction unit 11 causes the display unit 3 to display an image including predetermined information in a predetermined display format.
The predetermined display format is a format that includes small-sized characters, increases the number of characters, and the like, and is difficult to recognize when the user does not keep viewing the image displayed on the display unit 3 to some extent (described later with reference to fig. 3). In this way, the display instruction unit 11 can suppress the relative speed between the user and the mobile terminal 1 by displaying the image including the predetermined information in the predetermined display format on the display unit 3.
When generating an instruction signal for causing the display unit 3 to display predetermined information in a predetermined display format, the display instruction unit 11 outputs an imaging instruction signal for starting imaging of a moving image (i.e., a sensor information acquisition instruction signal for starting acquisition of sensor information) to the imaging unit 5. Thereby, the imaging unit 5 starts imaging a moving image including the body surface of the user at a predetermined frame rate set in advance (starts acquiring sensor information). Then, the imaging unit 5 outputs the captured moving image as a moving image signal to the relative speed calculation unit 12 and the biological information measurement unit 13.
When the display instructing unit 11 generates an instruction signal for causing the display unit 3 to display predetermined information in a predetermined display format, the relative speed calculating unit 12 calculates the relative speed between the imaging unit 5 and the user based on the moving image (sensor information) acquired from the imaging unit 5. For example, the relative speed calculation unit 12 calculates the relative speed between the imaging unit 5 and the user by performing image processing on the moving image acquired from the imaging unit 5. Specifically, for example, the relative velocity calculation unit 12 specifies a feature point (for example, an eye, a nose, or a mouth) of the face of the user included in the moving image acquired from the imaging unit 5, and calculates the direction and the amount of movement of the specified feature point for each frame in the moving image, thereby calculating the relative velocity between the imaging unit 5 and the user (described later with reference to fig. 4 and 5).
Then, the relative speed calculation unit 12 outputs information indicating the calculated relative speed between the imaging unit 5 and the user to the determination unit 15.
When the display instructing unit 11 generates an instruction signal for causing the display unit 3 to display predetermined information in a predetermined display format, the biological information measuring unit 13 measures the biological information of the user based on the moving image (sensor information) acquired from the imaging unit 5. For example, the biological information measurement unit 13 measures the biological information of the user by performing image processing on the moving image acquired from the imaging unit 5. Specifically, for example, the biological information measurement unit 13 specifies a local region of the face of the user (for example, a local region of the forehead, cheek, or chin) included in the moving image acquired from the imaging unit 5, and measures the pulse wave based on a change in state of each frame in the moving image in the specified local region (described later with reference to fig. 4 to 6).
Then, the biological information measurement unit 13 measures, for example, a blood pressure value, a pulse rate, a pressure level, a heart rate, blood oxygen saturation, information indicating a health state of a blood vessel, and the like based on the measured pulse wave, thereby obtaining biological information. Then, the biological information measurement unit 13 outputs information indicating the biological information of the user to the determination unit 15.
Here, as described above, the biological information measurement unit 13 measures the biological information of the user by performing image processing on the moving image acquired from the imaging unit 5, for example. Therefore, when the user moves the face of the user at a high speed with respect to the imaging unit 5 that is imaging the user, the accuracy of the biometric information of the user measured based on the moving image captured by the imaging unit 5 may be low due to image blur of the face of the user captured by the imaging unit 5.
Therefore, the biological information measurement unit 13 measures, for example, the biological information of the user while the relative speed calculation unit 12 calculates the relative speed between the imaging unit 5 and the user based on the moving image acquired from the imaging unit 5. In other words, the relative speed calculation unit 12 measures, for example, the relative speed between the user and the imaging unit 5 in a period in which the biological information measurement unit 13 calculates the biological information of the user based on the moving image acquired from the imaging unit 5.
Thus, it is possible to determine whether the accuracy of the biological information measured by the biological information measuring unit 13 is high or low based on the relative velocity calculated by the relative velocity calculating unit 12. That is, when the relative speed calculated by the relative speed calculation unit 12 is slow based on the moving image acquired from the imaging unit 5, it can be determined that the accuracy of the biological information measured by the biological information measurement unit 13 is high based on the moving image at the same time (in other words, the frame at the same time). In addition, when the relative speed calculated by the relative speed calculation unit 12 is high based on the moving image acquired from the imaging unit 5, it can be determined that the accuracy of the biological information measured by the biological information measurement unit 13 is low based on the moving image at the same time (in other words, the frame at the same time).
The timing at which the relative speed calculation unit 12 calculates the relative speed may be calculated every time interval of the input of the moving image signal from the imaging unit 5, may be calculated every time interval of the input of the moving image signal a plurality of times, or may be calculated only once while the biological information measurement unit 13 measures the biological information.
The timing at which the relative velocity calculation unit 12 starts calculating the relative velocity and the timing at which the biological information measurement unit 13 starts measuring the biological information may be performed first.
In the case where the sensor unit according to the present invention is not the imaging unit 5 that captures the moving image but various sensors such as a doppler sensor that can acquire sensor information indicating a change in the state of the living body of the user in a non-contact manner, the relative velocity calculation unit 12 calculates the relative velocity based on the sensor information, and the living body information measurement unit 13 measures the living body information.
In the case where the sensor unit according to the present invention is configured by a combination of a plurality of sensors such as the imaging unit 5 and the doppler sensor, the relative velocity calculation unit 12 may calculate the relative velocity based on the sensor information obtained by the combination of the plurality of sensors, and the biological information measurement unit 13 may measure the biological information.
The timer unit 14 generates time information indicating the current time, and sequentially outputs the time information to the determination unit 15. The determination unit 15 sequentially acquires time information indicating the current time from the time counting unit 14.
The determination unit 15 determines whether or not the relative velocity calculated by the relative velocity calculation unit 12 is within a predetermined range or less. Further, if the relative velocity calculated by the relative velocity calculation unit 12 is not more than the predetermined range, it can be determined that the measurement accuracy of the biological information at that time is high, and therefore, the determination unit 15 associates the time information at the time when the biological information is acquired from the biological information measurement unit 13 with the biological information acquired from the biological information measurement unit 13 as the time information indicating the time when the biological information measurement unit 13 measures the biological information. Then, the determination unit 15 stores the biological information associated with the time information in the storage unit 230 of the server 200 via the communication unit 40. The determination unit 15 outputs the biological information associated with the time information to the fluctuation amount monitoring unit 16.
Further, if the relative velocity calculated by the relative velocity calculation unit 12 exceeds the predetermined range, it can be determined that the measurement accuracy of the biometric information at that time is low, and therefore, for example, the determination unit 15 deletes the biometric information acquired from the biometric information measurement unit 13 without storing it in the storage unit 230 of the server 200.
The determination unit 15 may output the biological information associated with the time information to the storage unit 30 of the mobile terminal 1 instead of the storage unit 230 of the server 200, and store the biological information in the storage unit 30.
The fluctuation amount monitoring unit 16 acquires the biological information associated with the time information via the determination unit 15 from the storage unit 230 of the server 200 via the communication unit 40, and monitors whether or not the fluctuation amount of the acquired biological information is within a predetermined range. For example, if the variation amount per time of the biological information sequentially associated with the time information exceeds a predetermined range, the variation amount monitoring unit 16 performs, for example, a notification process.
That is, the fluctuation amount monitoring unit 16 outputs notification information for notifying the user to the display instruction unit 11. Thus, the display instruction unit 11 causes the display unit 3 to display an image indicating that the fluctuation amount of the biological information exceeds a predetermined range. This makes it possible to visually notify the user that the variation amount of the biometric information exceeds the predetermined range. Alternatively, the fluctuation amount monitoring unit 16 may generate notification information including sound information, and notify the user of the fact that the fluctuation amount of the biological information exceeds a predetermined range as a sound via the speaker 6.
For example, the fluctuation amount monitoring unit 16 may monitor the fluctuation amount of the biological information at each time point on the same day, or the fluctuation amount of the biological information at the same time point on another day. If the fluctuation amount of the biometric information is small, it can be considered that the health state of the user is not changed, and if the fluctuation amount is large beyond a predetermined range, it is considered that the health state of the user is changed a little. In this way, when the fluctuation amount exceeds the predetermined range, the fluctuation amount monitoring unit 16 performs the notification process, thereby enabling the user to start the investigation of the cause.
The fluctuation amount monitoring unit 16 may acquire the biological information associated with the time information from the storage unit 30, or may acquire the biological information from the determination unit 15 in sequence, instead of the storage unit 230 of the server 200. Further, the monitoring may be performed by reproducing the moving image captured by the imaging unit 5 and stored in the storage unit 230 in the past and retrieving the fluctuation amount of the biological information in the past time zone. In this case, when the content of the health management application, which will be described later, is updated at a time after the time slot in which the moving image is captured, the biological information can be calculated, the relative velocity can be calculated, and the fluctuation amount of the biological information can be monitored based on the updated reference.
An example of a display image of a specific application, which is displayed on the display unit 3 by the display instruction unit 11, will be described with reference to fig. 3. Fig. 3 is a diagram showing a display image of the health management application displayed on the display unit 3 by the display instruction unit 11 of the mobile terminal 1. The health management application is an example of a specific application including predetermined information in a predetermined display format for causing the display instruction unit 11 of the control unit 10 to display on the display unit 3.
As shown in fig. 3, a plurality of fields 61 are provided as predetermined information in a predetermined display format in the display image 60 of the biological information management application, for example, and the plurality of fields 61 are displayed in a format that is difficult for a user to recognize without viewing the display image 60 to some extent. In each column 61, content of interest to the user, such as daily news, health-related information, etc., is shown in order for the user to launch the health management application every day.
Here, in browsers of smartphones and PCs (personal computers), a font size of 16px (12pt) is used as an easy-to-read font size. In addition, in this browser, 1px (0.75pt) is used as the minimum font size.
Therefore, as the predetermined information in the predetermined display format, it is preferable that each column 61 displayed on the display image 60 displays characters having a font size of 1px to 16 px. In other words, as the predetermined information of the predetermined display format, it is preferable that characters having a font size of 0.75pt to 12pt are displayed in each column 61 displayed on the display image 60.
Accordingly, when the user reads the characters included in each column 61, it is difficult to recognize the characters without looking at the display image 60, and therefore the relative speed between the user and the image pickup unit 5 can be suppressed.
When character strings of a plurality of lines are displayed as images in each column 61 of the display image 60, it is preferable that the line pitch between the characters of the plurality of lines is 1 to 1.5 cm as predetermined information in a predetermined display format. Thus, since the intervals between the characters in the plurality of lines are relatively narrow, it is difficult for the user to recognize the characters in the respective lines included in the respective columns 61 without viewing the display image 60. Therefore, the relative speed between the user and the imaging unit 5 can be suppressed.
An example of a case where the relative velocity calculation unit 12 calculates the relative velocity between the user and the imaging unit 5 will be described with reference to fig. 4 and 5. Fig. 4 is a diagram showing an example of a moving image immediately before a user moves in the moving image captured by the imaging unit 5. Fig. 5 is a diagram showing an example of a moving image after a user moves in the moving image captured by the imaging unit 5.
As shown in fig. 4, the relative velocity calculation unit 12 specifies a preset feature point in the user's face U reflected in the moving image 63 acquired from the imaging unit 5, and defines an area R1 in the moving image 63 including the specified feature point. In fig. 4, as an example, the relative velocity calculation unit 12 specifies a nose as a feature point set in advance in the face U of the user included in the moving image 63, and defines a region R1 in the moving image 63 including the specified nose.
Then, as shown in fig. 5, in the moving image 63, when the user's face U moves, the prescribed region R1 also moves as indicated by an arrow X1 following the movement of the user's face U.
Here, the imaging unit 5 captures a moving image 63 at a predetermined frame rate (10 to 300 fps). Therefore, the relative speed calculation unit 12 calculates the moving speed of the region R1 in the moving image 63, that is, the relative speed between the imaging unit 5 and the user, based on the predetermined frame rate at which the imaging unit 5 captures the moving image 63 and the moving direction and the moving amount of the region R1 between frames. In addition, when the moving speed of the face U of the user is too high and the region R1 cannot follow, the relative speed calculation unit 12 determines that the relative speed exceeds the predetermined range.
When the display unit 3 displays an image (for example, a display image 60 shown in fig. 3) including predetermined information in a predetermined display format in the display area, the imaging unit 5 captures a moving image 63 used by the relative speed calculation unit 12 to calculate the relative speed.
The display instruction unit 11 may display a display image 60 (see fig. 3) including predetermined information in a predetermined display format on the display unit 3 and display a moving image 63 (see fig. 4 and 5) captured by the imaging unit 5.
Alternatively, the display instruction unit 11 may cause the display unit 3 to display the display image 60 of the display image 60 and the moving image 63 including the predetermined information in the predetermined display format only in the display area, and not to display the moving image 63 in the display area.
An example of a case where the biological information measuring unit 13 measures the biological information of the user will be described with reference to fig. 4 to 6.
As shown in fig. 4 and 5, the biological information measurement unit 13 specifies a local region R2 in the face U of the user shown in the moving image 63 acquired from the imaging unit 5. In the example shown in fig. 4 and 5, the biological information measurement unit 13 designates the forehead of the user's face U shown in the moving image 63 as the region R2.
Then, the biological information measurement unit 13 measures the pulse wave based on a change in the state of the region R2 (for example, a change in the color of the body surface of each frame) of the moving image 63 captured by the imaging unit 5 for each frame.
Fig. 6 is a diagram showing an example of a pulse wave waveform measured by the biological information measurement unit 13 based on the moving image 63 captured by the imaging unit 5. When the biological information measurement section 13 measures the pulse wave shown in fig. 6, the biological information is also obtained by measuring, for example, a blood pressure value, a pulse rate, a pressure level, a heart rate, a blood oxygen saturation level, information indicating a health state of a blood vessel, and the like from the pulse wave shown in fig. 6.
The change of the biological information monitored by the change amount monitoring unit 16 will be described with reference to fig. 7. Fig. 7 is a diagram showing a change in biological information monitored by the change amount monitoring unit 16. The fluctuation amount monitoring unit 16 monitors, for example, from the storage unit 230 of the server 200 via the communication unit 40, whether or not the fluctuation amount of the biological information corresponding to the time information is within a predetermined range.
Further, the fluctuation amount monitoring unit 16 may output information showing the daily or time-based fluctuation of the biological information (for example, blood pressure) corresponding to the time information from the storage unit 230 of the server 200 to the display instruction unit 11, and thereby the display instruction unit 11 may display an image of a graph showing the daily or time-based change of the biological information (for example, blood pressure) on the display unit 3. In this case, the display instruction unit 11 may display an image of a graph showing changes in biological information (for example, blood pressure) every day or every time as a partial area of the display image 60 (see fig. 3) on the display unit 3 together with predetermined information in a predetermined display format (for example, characters in each column 61 shown in fig. 3).
Fig. 8 is a flowchart showing an example of the operation of the control unit 10 of the mobile terminal 1 according to the first embodiment. As shown in fig. 8, in step S11, the mobile terminal 1 causes the display unit 3 to display predetermined information in a predetermined display format in response to an input signal from the input unit 4 based on a user operation. Next, in step S12, the relative speed calculation unit 12 calculates the relative speed between the imaging unit 5 and the user based on the moving image acquired from the imaging unit 5. Then, in step S13, the biological information measurement unit 13 measures the biological information of the user based on the moving image acquired from the imaging unit 5.
Then, in step S14, the determination unit 15 determines whether or not the relative velocity calculated by the relative velocity calculation unit 12 is within a predetermined range.
In step S14, when determining that the relative velocity calculated by the relative velocity calculation unit 12 is equal to or less than the predetermined range (yes in step S14), the determination unit 15 sequentially associates the biological information measured by the biological information measurement unit 13 with the time information when the biological information is measured by the biological information measurement unit 13. Then, the determination unit 15 sequentially stores the biological information associated with the time information in the storage unit 230 of the server 200 via the communication unit 40, for example (step S15).
In step S14, when determining that the relative speed calculated by the relative speed calculation unit 12 exceeds the predetermined range (in the case of no in step S14), the determination unit 15 returns to the process of step S12.
Next, in step S16, the fluctuation amount monitoring unit 16 monitors whether or not the fluctuation amount of the biological information sequentially corresponding to the time information stored in the storage unit 230 of the server 200 is within a predetermined range or less via the communication unit 40. If it is determined in step S16 that the fluctuation amount monitoring unit 16 has reached the predetermined range or less of the fluctuation amount of the biological information sequentially associated with the time information ("yes" in step S16), the process proceeds to the next step S18. In step S16, when determining that the fluctuation amount of the biological information sequentially associated with the time information exceeds the predetermined range (in the case of no at step S16), the fluctuation amount monitoring unit 16 performs a notification process (step S17), and proceeds to the process of the next step S18.
In step S18, the control unit 10 acquires an instruction signal for ending the measurement of the biological information from the input unit 4 (in the case of yes in step S18), and ends the measurement of the biological information. In step S18, if the instruction signal to end the measurement of biological information is not acquired from the input unit 4 (no in step S18), the control unit 10 returns to the process of step S12.
In this way, the control unit 10 in the mobile terminal 1 includes the display instruction unit 11, the biological information measurement unit 13, the relative speed calculation unit 12, and the determination unit 15. The display instructing unit 11 generates an instruction signal for causing the display unit 3 to display predetermined information in a predetermined display format, and outputs the instruction signal to the display unit 3. Thereby, the display unit 3 displays predetermined information in a predetermined display format in the display area. This enables the user to see the predetermined information in the predetermined display format displayed in the display area with a certain degree of detail. This makes it possible to reduce the relative speed between the user and the display unit 3 by making the user hardly move the face or the like.
When the display instruction unit 11 generates the instruction signal, the biological information measurement unit 13 measures the biological information of the user. The relative velocity calculation unit 12 calculates the relative velocity between the display unit 3 and the user while the biological information measurement unit 13 is measuring the biological information. Then, the determination unit 15 determines whether or not the relative velocity calculated by the relative velocity calculation unit 12 is within a predetermined range. Thus, the control unit 10 in the mobile terminal 1 can obtain the biological information when the relative velocity is within the predetermined range. That is, the control unit 10 in the mobile terminal 1 can obtain the biometric information of the user measured with high accuracy.
The predetermined information displayed on the display unit 3 in the predetermined display format preferably includes information of a content in which the user is interested, for example, information for managing a change in daily biometric information of the user, information related to daily news, and the like. Thus, the portable terminal 1 can measure the biological information of the user while the user visually recognizes the information of the content that the user is interested in displayed on the display unit 3. This reduces the burden of measuring biometric information on the user, and therefore, the user can easily obtain a large amount of biometric information. As a result, the mobile terminal 1 can grasp the health status of the user more accurately.
In the mobile terminal 1 according to the present embodiment, the display unit 3 for displaying predetermined information in a predetermined display format and the imaging unit 5 for imaging a user are provided in the same housing 2 and configured as the same device (mobile terminal 1), but the display unit 3 and the imaging unit 5 may be configured as different devices (devices separated from each other) that are communicably connected by wire or wirelessly.
[ second embodiment ]
A second embodiment of the present disclosure will be explained below. For convenience of explanation, members having the same functions as those described in the first embodiment are given the same reference numerals, and description thereof will not be repeated.
Fig. 9 is a functional block diagram of a biological information processing system 100 including the mobile terminal 1 according to the second embodiment. As shown in fig. 9, in embodiment 2, the control unit 10 includes a value information generation unit 17, and stores user specification information 231, point information 234, and health management information 233 in the storage unit 230 of the server 200. The other functional blocks of the biological information processing system 100 are the same as those of the biological information processing system 100 (see fig. 2) according to the first embodiment. The user specification information 231, the point information 234, and the health management information 233 may be stored in the storage unit 30 of the mobile terminal 1, and not in the storage unit 230 of the server 200.
In the present embodiment, when the fluctuation amount of the biological information acquired from the storage unit 230 of the server 200 or the like exceeds a predetermined range, the fluctuation amount monitoring unit 16 outputs information indicating whether the fluctuation amount is equal to or less than the predetermined range or exceeds the predetermined range to the value information generating unit 17 instead of performing the notification process.
The value information generating unit 17 generates value information 232 that is information corresponding to the fluctuation amount of the biological information monitored by the fluctuation amount monitoring unit 16.
The user identification information 231 is identification information assigned to each user for identifying the user.
The value information 232 is information containing at least one of the health management information 233 and the point information 234.
The health management information 233 is, for example, visual information and/or audio information corresponding to biological information (for example, pulse wave, blood pressure value, pulse rate, pressure level, heart rate, respiration rate, body temperature, blood oxygen saturation, information indicating the health state of blood vessels, and the like). Specifically, the health management information 233 is, for example, news, advice, or the like corresponding to the biological information, which is output as an image, and/or output as sound.
When the health management information 233 is visual information, the fluctuation amount monitoring unit 16 outputs the generated health management information 233 to the display instructing unit 11, and displays the health management information 233 as an image on the display unit 3 via the display instructing unit 11. In addition, when the health management information 233 is audio information, the value information generating unit 17 outputs the generated value information to the speaker 6, and outputs the value information to the speaker 6 as audio.
Thereby, the mobile terminal 1 can present useful information about the health status of the user to the user.
When the health management information 233 is visual information, the value information generating unit 17 may display the health management information 233 as predetermined information in a predetermined display format on a part of a display image of a specific application such as a health management application. This can reduce the relative speed between the display unit 3 and the user, and as a result, the biological information measuring unit 13 can measure the biological information with high accuracy.
The point information 234 is visual information indicating a point corresponding to the user specification information 231.
The value information generating unit 17 changes the integral indicated by the integral information 234 based on the result of the fluctuation amount monitoring unit 16 determining whether the fluctuation amount of the biological information is equal to or less than the predetermined range or whether the fluctuation amount of the biological information exceeds the predetermined range. For example, if the fluctuation amount monitoring unit 16 determines that the fluctuation amount of the biological information is equal to or less than the predetermined range, the value information generating unit 17 increases and updates the integral of the integral information 234 stored in the storage unit 230. For example, if the fluctuation amount monitoring unit 16 determines that the fluctuation amount of the biological information exceeds the predetermined range, the value information generating unit 17 updates the integral of the integral information 234 stored in the storage unit 230 without changing the integral or with decreasing the integral.
The value information generating unit 17 causes the storage unit 230 to display the point based on the point information 234 on a part of the display image of a specific application such as the health management application. The value information generating unit 17 may display the integral based on the integral information 234 as predetermined information in a predetermined display format on a part of the display image of the specific application.
Then, for example, the user can obtain various offers such as prizes from the points indicated by the point information 234. This makes it possible for the user to feel the advantage of measuring the biological information, and increases the opportunity of measuring the biological information. As a result, the mobile terminal 1 can grasp the health status of the user more accurately.
Fig. 10 is a flowchart showing an example of the operation of the control unit 10 of the mobile terminal 1 according to the second embodiment. The processing up to steps S11 to S16 is the same as the operation of the control unit 10 of the mobile terminal 1 according to the first embodiment described with reference to fig. 8.
As shown in fig. 10, the controller 10 performs the processing of step S11 to step S16.
When the fluctuation amount monitoring unit 16 determines in step S16 that the fluctuation amount of the biological information sequentially corresponding to the time information stored in the storage unit 230 of the server 200 is equal to or less than the predetermined range (in the case of yes in step S16), the value information generating unit 17 generates and updates the value information 232 with reference to the storage unit 230 in step S21. For example, in step S21, the value information generating unit 17 refers to the storage unit 230, and generates and updates the health management information 233 of the content in which the user is interested. For example, in step S21, the value information generating unit 17 refers to the storage unit 230, and generates and updates the point information 234 in which the point indicated by the point information 234 is increased.
When the fluctuation amount monitoring unit 16 determines in step S16 that the fluctuation amount of the biological information sequentially corresponding to the time information stored in the storage unit 230 of the server 200 exceeds the predetermined range (in the case of no in step S16), the value information generating unit 17 refers to the storage unit 230 and generates and updates the value information 232 in step S22. For example, in step S22, the value information generating unit 17 refers to the storage unit 230, and generates and updates the health management information 233 having the content for maintaining health, which corresponds to the content of the biological information (for example, pulse wave, blood pressure value, pulse rate, pressure level, heart rate, respiration rate, body temperature, blood oxygen saturation level, information indicating the health state of blood vessels, and the like). For example, in step S22, the value information generating unit 17 refers to the storage unit 230, and generates and updates the point information 234 in which the point indicated by the point information 234 is maintained or reduced.
Next, in step S22, the value information generator 17 outputs the content of the value information 232 generated in step S21 or step S22 to the display 3 and/or the speaker 6. Then, the process proceeds to step S18.
As described above, according to the mobile terminal 1 of the present embodiment, it is possible to present the content of the useful value information 232 to the user in accordance with whether or not the fluctuation amount of the biometric information is within the predetermined range or less. This makes it possible for the user to feel the advantage of measuring the biological information, and increases the opportunity of measuring the biological information. As a result, the mobile terminal 1 can grasp the health status of the user more accurately.
[ modified example ]
The main body of the portable terminal 1 or the biological information processing method according to the present disclosure includes a control unit 10 as a computer. The functions of the main body of the portable terminal 1 and the biological information processing method in the present disclosure are realized by the computer executing the biological information processing program. The computer includes a processor operating according to a biological information processing program as a main hardware configuration. The processor is not limited to a type of the biometric information processing program, as long as the processor can realize a function by executing the biometric information processing program. The processor includes a semiconductor Integrated Circuit (IC) or one or more electronic circuits including an LSI (Large Scale Integration). The plurality of electronic circuits may be integrated in one chip or may be provided in a plurality of chips. The plurality of chips may be integrated in one device or may be provided in a plurality of devices. The biometric information processing program is recorded on a non-transitory storage medium such as a computer-readable ROM, an optical disk, and a hard disk drive. The biological information processing program may be stored in advance in a storage medium, or may be supplied to the storage medium via an electric communication line including the internet or the like.
The present invention is not limited to the above-described embodiments, and may be replaced with a configuration having substantially the same configuration, a configuration achieving the same operational effects, or a configuration achieving the same object as those described in the above-described embodiments.

Claims (14)

1. A biological information processing apparatus, comprising:
a display instruction unit that generates an instruction signal for causing a display unit to display predetermined information in a predetermined display format;
a biological information measuring unit that measures biological information of the user based on sensor information indicating a change in state of a biological body of the user acquired by the sensor unit after the display instruction unit generates the instruction signal;
a relative velocity calculation unit that calculates a relative velocity between the sensor unit and the user during a period in which the biological information measurement unit measures the biological information; and
and a determination unit that determines whether or not the relative velocity calculated by the relative velocity calculation unit is within a predetermined range.
2. The biological information processing apparatus according to claim 1,
comprises a time counting part for generating time information,
the determination unit sequentially associates the biological information when the relative velocity is within the predetermined range with the time information when the biological information is measured.
3. The biological information processing apparatus according to claim 2,
the method comprises the following steps: and a fluctuation amount monitoring unit that monitors whether or not the fluctuation amount of the biological information associated with the time information is within a predetermined range.
4. The biological information processing apparatus according to any one of claims 1 to 3,
comprises the sensor part and a control part, wherein the sensor part is arranged on the base plate,
the sensor unit acquires the sensor information by capturing a moving image of the user.
5. The biological information processing apparatus according to claim 4,
the relative speed calculation unit calculates the relative speed based on the dynamic image.
6. The biological information processing apparatus according to any one of claims 1 to 5,
the biological information includes information indicating at least one of a blood pressure value, a pulse rate, a heart rate, a respiration rate, a body temperature, and a blood oxygen saturation level.
7. The biological information processing apparatus according to any one of claims 1 to 6,
the predetermined information is an image including characters having a font size of 1px to 16px, or 0.75pt to 12 pt.
8. The biological information processing apparatus according to any one of claims 1 to 7,
the predetermined information is an image in which the line pitch between a plurality of lines of characters is 1-1.5 cm.
9. The biological information processing apparatus according to claim 3,
the system includes a value information generating unit that generates value information corresponding to the fluctuation amount monitored by the fluctuation amount monitoring unit.
10. The biological information processing apparatus according to claim 9,
comprises the display part and a display part, wherein the display part is arranged on the display part,
the value information includes visual information corresponding to the content of the biological information,
the value information generation unit causes the display unit to display an image based on the visual information.
11. The biological information processing apparatus according to claim 9 or 10,
comprises a sound output part for outputting sound,
the value information includes sound information corresponding to the content of the biological information,
the value information generation unit causes the sound output unit to output a sound based on the sound information.
12. The biological information processing apparatus according to any one of claims 9 to 11,
the value information includes point information representing points associated with user determination information determining the user,
the value information generating unit increases the integral when the fluctuation amount of the biological information is equal to or smaller than the predetermined range, and decreases or does not change the integral when the fluctuation amount of the biological information exceeds the predetermined range.
13. A biometric information processing method, comprising:
generating an instruction signal for causing a display unit to display predetermined information in a predetermined display format;
measuring biometric information of the user based on sensor information indicating a change in state of a biometric object of the user acquired by a sensor unit after the instruction signal is generated;
calculating a relative speed between the sensor unit and the user during a period in which the biometric information is measured; and
and judging whether the relative speed is within a predetermined range.
14. The biometric information processing method according to claim 13, comprising:
sequentially associating the biological information when the relative velocity is within the predetermined range with time information when the biological information is measured;
monitoring whether or not the fluctuation amount of the biological information associated with the time information is within a predetermined range; and
and generating value information corresponding to the fluctuation amount.
CN202110195726.5A 2020-02-25 2021-02-19 Biological information processing device and biological information processing method Pending CN113367658A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-029186 2020-02-25
JP2020029186A JP2021132726A (en) 2020-02-25 2020-02-25 Living body information processor and living body information processing method

Publications (1)

Publication Number Publication Date
CN113367658A true CN113367658A (en) 2021-09-10

Family

ID=77570591

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110195726.5A Pending CN113367658A (en) 2020-02-25 2021-02-19 Biological information processing device and biological information processing method

Country Status (2)

Country Link
JP (1) JP2021132726A (en)
CN (1) CN113367658A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002259572A (en) * 2001-03-06 2002-09-13 Matsushita Electric Ind Co Ltd Advice apparatus for health support service
JP2015205050A (en) * 2014-04-21 2015-11-19 富士通株式会社 Pulse wave detection device, pulse wave detection method, and pulse wave detection program
CN107550474A (en) * 2016-06-30 2018-01-09 松下知识产权经营株式会社 Bioinformation detecting device
CN109922714A (en) * 2016-11-07 2019-06-21 夏普株式会社 Apparatus for measuring biological data, the control method of apparatus for measuring biological data, control device and control program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002259572A (en) * 2001-03-06 2002-09-13 Matsushita Electric Ind Co Ltd Advice apparatus for health support service
JP2015205050A (en) * 2014-04-21 2015-11-19 富士通株式会社 Pulse wave detection device, pulse wave detection method, and pulse wave detection program
CN107550474A (en) * 2016-06-30 2018-01-09 松下知识产权经营株式会社 Bioinformation detecting device
CN109922714A (en) * 2016-11-07 2019-06-21 夏普株式会社 Apparatus for measuring biological data, the control method of apparatus for measuring biological data, control device and control program

Also Published As

Publication number Publication date
JP2021132726A (en) 2021-09-13

Similar Documents

Publication Publication Date Title
US8781791B2 (en) Touchscreen with dynamically-defined areas having different scanning modes
US8768648B2 (en) Selection of display power mode based on sensor data
US20220413692A1 (en) System and method for controlling the display of an image stream
US20170273634A1 (en) Meal estimation method, meal estimation apparatus, and recording medium
US20100145157A1 (en) Compact monitor and a method for monitoring a user
US11651842B2 (en) Server, portable terminal device, electronic device, and control method therefor
US10004411B2 (en) Living body determination devices and methods
CN114449945A (en) Information processing apparatus, information processing system, and information processing method
CN108135511B (en) Diagnosis support device, diagnosis support method, and recording medium
JP2018005512A (en) Program, electronic device, information processing device and system
KR20210060246A (en) The arraprus for obtaining biometiric data and method thereof
US20190365243A1 (en) Blood-pressure-related information display apparatus and method
CN113367658A (en) Biological information processing device and biological information processing method
JP7148624B2 (en) Image proposal device, image proposal method, and image proposal program
JP7325576B2 (en) Terminal device, output method and computer program
US20210012130A1 (en) Method and device for measuring biometric information in electronic device
US20180098729A1 (en) Body composition measuring device and mobile terminal wirelessly connected to the same
EP4111984A1 (en) Information processing method, computer program, information processing device, and information processing system
CN113052032A (en) Human body posture detection method, electronic device and computer readable storage medium
JP7360315B2 (en) Mobile terminal, biometric information acquisition method, and biometric information acquisition program
JP2018149081A (en) Information processing apparatus, information processing method, and program
CN108475462B (en) Operation method and system for camera operation device and device thereof
JP2021096537A (en) Biological information acquisition device, terminal device, biological information acquisition method, biological information acquisition program, and computer readable recording medium
US20240081749A1 (en) Health assessment system, health assessment method, and storage medium storing health assessment program
WO2007083315A2 (en) Compact monitor and a method for monitoring a user

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination