JP7233631B1 - posture improvement system - Google Patents

Abstract

A posture improvement system that determines the distance from the display to the face and the posture using a camera installed in a device such as a personal computer, tablet, or smartphone, and proposes a posture improvement to the user. Kind Code: A1 In a posture improvement system, an image capturing unit 10 captures an image of a user by a camera arranged near a display. The feature point distance measuring unit 11 detects a plurality of feature points of the user's face or upper body from the image of the user acquired by the image capturing unit 10, and measures the distance between each feature point. The posture determination unit 12 determines the inclination of each feature point based on the change in the distance between each feature point, and determines whether the posture is good or bad based on the inclination of each feature point. The visual distance measuring unit 13 determines the visual distance to the display based on the increase or decrease in the distance between each feature point measured by the inter-feature point distance measuring unit. The posture improvement presentation unit 14 presents a posture improvement proposal to the user based on the determination results of the posture determination unit and the visual distance measurement unit. [Selection drawing] Fig. 1

Description

The present invention relates to a posture improvement system that is used to improve the posture of users of personal computers, laptops, tablets, smartphones, and other devices based on image information obtained from cameras installed in these devices.

Conventionally, as shown in the following patent documents, when using a device such as a personal computer, a system that contributes to the improvement of posture by determining whether the user is using the device with a proper posture has been known. It is

Japanese Patent No. 5820366 JP 2012-120647 A JP 2016-123586 A

However, these conventional technologies use a camera provided separately from devices such as personal computers and smartphones, or use devices such as personal computers and smartphones to perform desk work, such as determining the posture of the entire body. It was not suitable for In particular, in classes for students, home study, companies for working adults, telework, etc., it is necessary to determine the posture of the user only from the upper body and face of the device placed on the desk. It has been difficult with conventional techniques to determine whether the posture is good or bad.

Further, in the conventional technology, it is common to determine the user's posture by using a camera provided at a position different from that of the personal computer, for example, on the side of the user, so that the change in posture can be easily determined. there were. However, it is troublesome to prepare a separate camera as in the prior art, and it cannot be said that anyone can easily use it in any place. In particular, in recent years, in-cameras integrated into personal computers and smartphones and external web cameras are widely used. It is expected to determine the posture of the user.

The present invention has been proposed to solve the problems of the prior art as described above. The purpose of the present invention is to determine the distance from the display to the face and posture using a camera installed in a device such as a personal computer, tablet, or smartphone, and to propose a posture to the user to improve the posture. To provide an improvement system.

The posture improvement system of the present invention has the following configuration.
(1) An image capturing unit that captures an image of the user by means of a camera arranged near the display facing the user.
(2) A feature point distance measuring unit that detects a plurality of feature points of the user's face or upper body from the image of the user acquired by the image capturing unit and measures the distance between the feature points.
(3) Determining the inclination of each feature point based on the change in the distance between the feature points measured by the inter-feature point distance measuring unit, and determining whether the posture is good or bad based on the inclination of each feature point. Posture determination unit.
(4) A visual distance measuring unit that determines a visual distance to the display based on an increase or decrease in the distance between the feature points measured by the inter-feature point distance measuring unit.
(5) A posture improvement presentation unit that presents a posture improvement proposal to the user based on the determination results of the posture determination unit and the visual distance measurement unit.

Embodiments of the present invention can have the following configurations.
(1) A storage unit in which the information obtained by the image capturing unit and/or the feature point distance measuring unit is stored in chronological order.
(2) the storage unit stores a plurality of posture improvement proposals presented by the posture improvement presentation unit;
The time series change of the information obtained by the posture determination unit and/or the visual distance measurement unit stored in the storage unit, and the information changed by the plurality of posture improvement proposals presented by the posture improvement presentation unit a proposal selection unit that selects a predetermined posture improvement proposal from among the plurality of posture improvement proposals based on the plurality of posture improvement proposals.

1 is a block diagram of a posture improvement system according to a first embodiment; FIG. 4A and 4B are diagrams showing a method of detecting a tilt in the first embodiment; FIG. The figure which shows the method of detecting visual distance in 1st Embodiment.

[1. First Embodiment]
[1-1. composition]
The posture improvement system of the present embodiment is installed as a computer program on a computer-using device (hereinafter referred to as a device) such as a personal computer, a smartphone, or a tablet. The device has a display unit 1 such as a display, a camera 2 incorporated in the device main body, an operation input unit 3 such as a touch panel that also serves as a keyboard and a display, an audio output unit 4, and stores programs and various data of the present embodiment. It has a storage unit 5 . In addition to the display unit 1 and the audio output unit 4 of the device, it has a communication unit 6 that transmits data to other devices via the Internet, such as a management server, a personal computer owned by a boss or a family member, a smartphone, or the like. .

The posture improvement system incorporated in the device has an image capturing unit 10, a feature point distance measurement unit 11, a posture determination unit 12, a visual distance measurement unit 13, and a posture improvement presentation unit . In particular, in this embodiment, in addition to the above units, information obtained by the image capturing unit 10 and/or the feature point distance measuring unit 11 and a plurality of posture improvement proposals are stored in the storage unit 5, It has a proposal selection unit 15 that selects an appropriate proposal from a plurality of posture improvement proposals.

The image capturing unit 10 captures an image of the user using a camera arranged near the display facing the user, and stores the captured image in the storage unit 5 . The captured image can be the face of the user or the upper half of the body including the face. Since the camera is installed in the display part of the device, while the user is using the device, the entire image captured by the camera of the device may be used, or only the face part of the image may be used. You may extract by analysis.

The image capturing unit 10 can capture and store an image in which the user is using the device in a correct posture as a reference image in advance. In addition, when the device is in use, it may be possible to automatically take pictures at fixed time intervals starting from the start of use, and save the pictures taken at each time with time. In general, users are less tired when they start using the device, so they tend to take the correct posture. , a reference image may be generated and stored in the storage unit 5 .

A feature point distance measuring unit 11 detects a plurality of feature points on the user's face or upper body from the image of the user acquired by the image capturing unit 10, and measures the distance between these feature points. . Various conventionally known methods can be appropriately used as a method for detecting feature points. As feature points, each part of the face, such as eyes, eyebrows, nose, ears, chin, and the left and right or upper and lower ends of each part, can be used. Contours of the jaw, neck, shoulders, etc. may also be used as feature points. 2 and 3 are diagrams showing an example of facial feature points. It is desirable to select a plurality of feature points positioned to the left and right of the image and a plurality of feature points positioned to the top and bottom of the image as the plurality of feature points to be used for distance measurement. For example, it is desirable to measure the distances in the horizontal and vertical directions, such as the distance between the eyes and the distance between the eyes and the mouth. Further, distances may be measured for each of a plurality of feature points arranged in a triangle, such as between the eyes and the mouth, and between the eyes and the tip of the chin.

The posture determination unit 12 determines the inclination of each feature point based on the change in the distance between each feature point measured by the inter-feature point distance measurement unit 11, and determines whether the posture is good or bad based on the inclination of each feature point. do. That is, the inter-feature point distance measuring unit 11 selects a plurality of feature points positioned vertically in the image and measures the distance between them, or measures the respective distances of a plurality of feature points arranged in a triangle. . In general, when the posture becomes worse, the angle between the display of the device and the face or upper body changes. ), the vertical distance of each feature point is shortened. Therefore, by detecting how much the vertical distance of each feature point in the captured image changes with respect to the vertical distance of the same feature point in the reference image shown in FIG. Judge whether the product is good or bad.

As a criterion for determination, images of the user's correct posture and bad posture are taken in advance using the camera of the device. is measured and stored in the storage unit 5. For example, there are 5 types: upright with the pelvis in the middle position, slump with the pelvis posteriorly tilted and weakened, warp with the pelvis in the forward tilted position, bending the head to the left (left), and bending the head to the right (right). , a photographed image is acquired in advance, and the distance between each feature point in the image is measured. The posture determination unit 12 compares which of the five reference images the distance of each feature point in the image taken at the time of determination is closer to, and determines whether the posture of the user is good or bad at the time of determination.

The visual distance measuring unit 13 determines the visual distance to the display based on the increase or decrease in the distance between each feature point measured by the distance between feature points measuring unit 11 . For example, when the user's face approaches the display, the distance between each feature point in the photographed image increases as shown in FIG. It is determined that the viewing distance of the user has become closer. In this case, the distance between the feature points determined by the posture determination unit 12 is tilted because there is a difference between the amount of change in the distance between the feature points in the horizontal direction and the amount of change in the distance in the vertical direction. 13 measures the viewing distance by changing the distance in both the vertical and horizontal directions at substantially the same rate. Moreover, since the object of measurement is the visual distance, the visual distance may be measured based only on the increase or decrease in the distance of the feature points near the eyes such as the eyes and eyebrows. Furthermore, by measuring the visual distance in advance, it is possible to measure the visual distance as an absolute amount at the time of measurement, and to measure the relative visual distance between the visual distance in the correct posture and the visual distance at the time of measurement. This can lead to proposals for improving posture.

The posture improvement presentation unit 14 presents a posture improvement proposal to the user based on the determination results of the posture determination unit 12 and the visual distance measurement unit 13 . The posture improvement presentation unit 14, for example, captures an image of the user at regular intervals, and determines the quality of the posture obtained by the posture determination unit 12 and the change in the visual distance obtained by the visual distance measurement unit 13. When the prescribed standard is exceeded, the user is presented with an improvement proposal such as bad posture or close viewing distance. The criteria to be presented can be appropriately selected, and an improvement proposal can be presented when the posture or visual distance is bad for a certain period of time, or when the degree of bad is greater than a preset standard. As a form of improvement proposal, it is possible to display a warning icon on the display, display a message prompting correct posture or take a break, or output a message or warning sound from the voice output unit 4 . In addition, using the communication unit 6, it is possible to transmit warning contents and improvement proposals to other devices. As a result, it becomes possible for other devices, which are not intended for posture improvement, to incorporate posture improvement secondarily, and the user can secondarily improve posture while using other devices.

Information obtained by the image capturing unit 10 and/or the feature point distance measuring unit 11 is stored in the storage unit 5 in chronological order. This information includes information about users and devices, captured images at the time of setting and judgment, information on distances between feature points in preset normal and bad postures, information on visual distance, information at the time of judgment and photography. time data, contents of improvement proposals, methods of presenting improvement proposals, etc.

In this embodiment, the storage unit 5 stores a plurality of posture improvement proposals presented by the posture improvement presentation unit 14 . Various suggestions for improvement can be prepared according to the degree of poor posture and the degree of inadequacy of the viewing distance. For example, it is not limited to a mere warning, but rather suggests changing the installation state of the device or taking a break according to the user's age, gender, bad posture, degree of bad posture or visual distance, and duration. Data can be presented in graphs, scoring tables, and the like.

The proposal selection unit 15 selects a predetermined posture improvement proposal from among the plurality of posture improvement proposals stored in the storage unit 5 . That is, in the present embodiment, the time-series change of information regarding the distance between each feature point obtained by the posture determination unit 12 and/or the visual distance measurement unit 13 at the time of determination, the state of the determined posture, and the visual distance is stored. Stored in Part 5. At the same time, the storage unit 5 also stores information indicating what kind of posture improvement proposal the posture improvement presentation unit 14 has made at the time of each determination. The proposal selection unit 15 compares the improvement proposal at the time of each determination with changes in posture and/or visual distance before and after the improvement proposal, and determines how much the improvement proposal contributes to the improvement of posture and/or visual distance. determine whether it did.

In this way, the proposal selection unit 15 determines what kind of posture and/or visual distance state, in other words, what kind of posture and/or visual distance state, in other words, normal posture and visual distance is analyzed to determine what kind of improvement proposal is effective for the change pattern. After that, when making a new determination and proposal, the storage unit 5 compares the information on the posture and/or the visual distance accumulated over time with the newly obtained information on the posture and/or the visual distance. Select and implement the best of several improvement suggestions stored in .

[1-3. Effect of Embodiment]
The effects of the first embodiment are as follows.
(1) Incorporating the system into the equipment used for work and study, and using the camera installed in the equipment as it is to determine the posture and viewing distance and propose improvements, so a separate camera etc. This does not require any equipment, does not burden the user, and can be easily performed at the place where the equipment is used, regardless of time and place.

(2) Since improvement proposals are made based on two pieces of information, posture and visual distance, even with information on a relatively narrow range such as the face and upper body, the state of posture and visual distance can be determined with relatively high accuracy. It is possible to make appropriate improvement proposals according to the judgment status.

(3) By using an algorithm that determines the inclination of the posture and the viewing distance based on the change in the distance between a plurality of feature points, the arithmetic processing required to determine the inclination and the viewing distance is easy, and the user can work. It is possible to judge the posture and visual distance while using the equipment for studying or studying, and to make appropriate improvement proposals while using the equipment.

(4) Since the storage unit 5 stores a plurality of posture improvement proposals presented by the posture improvement presentation unit 14, the improvement proposal method can also be In addition to the display of warnings and icons, it is also possible to issue various messages such as taking a break, doing exercises, and promoting a change of pace, etc. Appropriate proposals can be made according to the equipment and software used.

(5) The proposal selection unit 15 automatically selects an effective improvement proposal for the newly determined posture and/or viewing distance based on the information accumulated over time in the storage unit 5. Even if the user, the device, or the software to be implemented is different, the optimum improvement proposal can be presented.

(6) By classifying the correct posture and the bad posture of the user into a plurality of types in advance and photographing them, the posture determination unit 12 can determine the distance of each feature point in the image photographed at the time of determination. Since it is determined whether the posture of the user is good or bad at the time of determination by comparing which of the types of reference images it is closest to, the degree of deviation of the posture and viewing distance from the normal values can be easily determined for each rank.

[2. Other embodiments]
The present invention is not limited to the above-described embodiments, and can be embodied by modifying constituent elements without departing from the scope of the present invention at the implementation stage. Further, various inventions can be formed by appropriate combinations of the plurality of constituent elements disclosed in the above embodiments. For example, some components may be omitted from all components shown in the embodiments. Furthermore, components across different embodiments may be combined as appropriate. Specifically, the following other embodiments are also included.

(1) Judgment of normal posture and viewing distance can be preset for each user at the start of this system, and AI etc. can be used to The correct posture and viewing distance may be measured, and the state of tilt and viewing distance within a certain range may be stored in advance in the storage unit 5 as normal values. In addition, normal values can be set by averaging the data of posture and visual distance for a certain period of time from the start of use of each user, that is, the data of the user's posture and visual distance in a state where fatigue has not accumulated. A normal value can also be set based on the inclination and viewing distance immediately after the improvement proposal is presented.

(2) Since determination of posture and visual distance and improvement proposals are performed in parallel with other work performed by the user, monitoring the implementation status of other work and improving the user's work It is also possible to present an improvement proposal at a non-obtrusive timing, such as when input work to the device is interrupted, when software is switched, when files are waiting to be downloaded or printed, and so on.

(3) The inter-feature point distance measurement unit 11 determines the number of yawns from information on the degree of opening of the mouth, determines the degree of opening of the pupil, blinks and The fatigue level of the user may be determined from the number of times, and a proposal for improving the fatigue level may be presented together with a proposal for improving the posture and/or the viewing distance.

(4) In addition to the inter-feature point distance measuring unit 11, by measuring the angle between feature points and adding that information, it is possible to more accurately determine the inclination of the posture and changes in the visual distance. The viewing distance may be determined by combining information from an autofocus mechanism provided in the camera, other than by increasing or decreasing the distance between feature points.

(5) A “camera placed facing the user near the display” in the claims means a camera built into the device, an external camera, or a camera fixed to a bookshelf or desk around the device. It may be a camera with In addition, "facing" does not necessarily mean only facing the user's face or upper body in front of the user. They may be displaced, and their installation positions can be selected as appropriate.

DESCRIPTION OF SYMBOLS 1... Display part 2... Camera 3... Operation input part 4... Sound output part 5... Storage part 6... Communication part 10... Image photographing part 11... Distance between feature points measuring part 12... Posture determining part 13... Visual distance measuring part 14 ... Posture improvement presentation unit 15 ... Proposal selection unit

Claims (2)

an image capturing unit that captures an image of the user with a camera arranged facing the user near the display;
a feature point distance measuring unit that detects a plurality of feature points of the user's face or upper body from the image of the user acquired by the image capturing unit and measures the distance between each of the feature points;
Determining the inclination of the surface on which the feature points are placed with respect to the display based on the change in the distance between the feature points measured by the inter-feature point distance measuring unit, and determining the posture based on the inclination of the feature points a posture determination unit that determines the quality of the
a posture improvement presentation unit that presents a posture improvement proposal to the user based on the determination result of the posture determination unit;
a storage unit that stores information obtained by the image capturing unit and/or the feature point distance measuring unit in time series;
The storage unit stores a plurality of posture improvement proposals presented by the posture improvement presentation unit,
The plurality of postures are determined based on time-series changes in the information obtained by the posture determination unit stored in the storage unit and the information changed by the plurality of posture improvement proposals presented by the posture improvement presentation unit. A posture improvement system comprising a proposal selection unit that selects a predetermined posture improvement proposal from improvement proposals . a visual distance measuring unit that determines a visual distance to the display based on an increase or decrease in the distance between the feature points measured by the inter-feature point distance measuring unit;
2. The posture improvement system according to claim 1, wherein the posture improvement presentation unit presents a posture improvement proposal to the user based on determination results by the posture determination unit and the visual distance measurement unit.

Images