WO2024176398A1

WO2024176398A1 - Information processing device, information processing system, information processing method, information processing program, and recording medium

Info

Publication number: WO2024176398A1
Application number: PCT/JP2023/006512
Authority: WO
Inventors: 祥史大西; 旭美梅松
Original assignee: 日本電気株式会社
Priority date: 2023-02-22
Filing date: 2023-02-22
Publication date: 2024-08-29

Abstract

In order to enable a user to confirm whether they are performing diaphragm breathing, an information processing device (1) comprises: an image acquisition means (11) for acquiring image data of the user; a calculation means (12) for calculating the user's chest respiratory-cycle and abdomen respiratory-cycle by referring to information contained in the image data; and an output means (14) for outputting the degree of synchronization between the chest respiratory-cycle and the abdomen respiratory-cycle.

Description

Information processing device, information processing system, information processing method, information processing program, and recording medium

The present invention relates to an information processing device, an information processing system, an information processing method, an information processing program, and a recording medium.

　Devices that measure human breathing from images are known. For example, Patent Document 1 discloses a respiratory function measuring device that measures respiratory function to diagnose obstructive pulmonary disease, restrictive pulmonary disease, and the like. This device measures chest and abdominal movement, and calculates the respiratory time difference between the time when the abdominal volume reduction rate of exhaled air is at its maximum and the time when the thoracic volume reduction rate of exhaled air is at its maximum, to detect cases of decreased lung function, etc.

Patent document 2 also discloses an apparatus that determines and evaluates multiple different AC signals corresponding to vital sign information of a subject from multiple different regions of an imaging field based on a motion pattern of image data, and determines multiple different respiratory signals from the subject based on the different AC signals determined from the different regions of the imaging field. With this apparatus, respiratory signals from different parts of the subject, such as the thorax and abdomen, can be determined corresponding to different respiratory techniques to increase the accuracy of respiratory detection and to determine additional information from the subject's respiration. In this way, additional diagnosis can be performed, and more reliable and accurate respiratory detection can be achieved.

Japanese Patent Application Publication No. 2008-154655 Japan Special Publication No. 2016-518191

The device described in Patent Document 1 is intended to detect diseases such as decreased lung function. The device described in Patent Document 2 is intended to perform reliable and accurate breathing detection.

In recent years, teleworking has become widespread, and people are now working long hours in environments where working from home or other places where working is not considered, making it necessary to take time to refresh themselves appropriately. In particular, abdominal breathing is said to be able to improve ventilation efficiency by correcting thoracic movement and reducing the work of breathing and the degree of dyspnea. It has also been reported that during conscious abdominal breathing, the parasympathetic nervous system becomes dominant, allowing a relaxed state to be maintained. However, it is difficult to check for yourself whether you are performing abdominal breathing properly. The devices disclosed in Patent Documents 1 and 2 do not detect whether abdominal breathing is being performed properly.

One aspect of the present invention was made in consideration of the above problems, and one example of the purpose of the present invention is to provide a technology that allows a user to check whether or not they are performing abdominal breathing.

An information processing device according to one aspect of the present invention includes an image acquisition means for acquiring image data of a user, a calculation means for calculating the chest respiratory cycle and the abdominal respiratory cycle of the user by referring to information contained in the image data, and an output means for outputting the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

An information processing system according to one aspect of the present invention includes an imaging device that captures an image of a user, image acquisition means that acquires image data of the image, calculation means that calculates the chest respiratory cycle and abdominal respiratory cycle of the user by referring to information contained in the image data, and output means that outputs the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

An information processing method according to one aspect of the present invention includes acquiring image data of a user, calculating the user's chest respiratory cycle and abdominal respiratory cycle by referring to information contained in the image data, and outputting the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

An information processing program according to one aspect of the present invention causes a computer to execute an acquisition process for acquiring image data of a user, a calculation process for calculating the chest respiratory cycle and abdominal respiratory cycle of the user by referencing information contained in the image data, and an output process for outputting the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

According to one aspect of the present invention, a user can check whether or not they are performing abdominal breathing.

1 is a block diagram showing a configuration of an information processing device 1 according to a first exemplary embodiment of the present invention. 1 is a flow diagram showing the flow of an information processing method S1 according to the first exemplary embodiment. 1 is a block diagram showing a configuration of an information processing system 2 according to a first exemplary embodiment. FIG. 11 is a block diagram showing a configuration of an information processing device 1A according to an exemplary embodiment 2 of the present invention. 1 is an example showing regions that the image analyzer has defined as the chest and abdomen from an image of a user. 11 is a graph showing a chest respiratory cycle and an abdominal respiratory cycle of a user calculated by a calculation unit. 11 is an example of advice selected by a determination unit. FIG. 2 is a block diagram showing a configuration of an activation estimation unit. This is an example of the change in vital signs that occurs when relaxing through abdominal breathing. FIG. 2 is a block diagram showing a configuration of an information processing system 2A. FIG. 1 is a configuration diagram for implementing an information processing device using software. FIG. 11 is a block diagram showing a configuration of an information processing device 1B according to an exemplary embodiment 4 of the present invention. FIG. 11 is a flow chart showing the flow of an information processing method S2 according to the fourth exemplary embodiment. FIG. 11 is a block diagram showing a configuration of an information processing system 2B according to an exemplary embodiment 4.

[Example embodiment 1]
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

(Configuration of information processing device 1)
The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 is a device that allows a user to check for himself/herself whether or not he/she is performing abdominal breathing. The information processing device 1 may be configured as, for example, a personal computer, or may be configured as a mobile device such as a smartphone. Alternatively, the information processing device 1 may be configured as a dedicated device that allows a user to check for himself/herself whether or not he/she is performing abdominal breathing. As shown in FIG. 1, the information processing device 1 includes an image acquisition unit 11, a calculation unit 12, a derivation unit 13, and an output unit 14.

The image acquisition unit 11 acquires image data of the user. The image data may be image data captured by an imaging device (not shown). The imaging device may be a digital camera capable of capturing RGB (Red, Green, Blue) moving images. Alternatively, the imaging device may be a device incorporating a distance measuring device capable of acquiring distance information. The imaging device may be incorporated in the information processing device 1. For example, it may be a camera incorporated in a personal computer (hereinafter referred to as a "personal computer"), a mobile device, or a dedicated device. Alternatively, the imaging device may be a camera disposed at a position separated from the information processing device 1. In this case, the image data may be transmitted from the imaging device to the information processing device 1 by wireless communication or the like. The image acquisition unit 11 is one form of image acquisition means described in the claims.

The calculation unit 12 refers to information contained in the image data acquired by the image acquisition unit 11 to calculate the user's chest respiratory cycle and abdominal respiratory cycle. The information contained in the image data is, for example, luminance information or distance information. The luminance information can be acquired for each pixel. The distance information is distance information to each point if the imaging device is capable of acquiring point cloud data as distance information. The calculation unit 12 is one form of the calculation means described in the claims.

The calculation unit 12 calculates the respiratory cycle of the user's chest using brightness information, distance information, etc. of the user's chest. The calculation unit 12 also calculates the respiratory cycle of the user's abdomen using brightness information, distance information, etc. of the user's abdomen. The chest refers to the area around the ribs, and the abdomen refers to the area below the ribs or solar plexus. The regions of the chest and abdomen do not need to be defined precisely. The calculation unit 12 can identify the approximate chest region and abdominal region using, for example, a known skeletal estimation application that estimates a person's skeleton from an image. The chest region and abdominal region may each be multiple regions. An example of defining multiple regions will be described later.

In the following, it is assumed that the user is working using a computer equipped with a camera. In this case, there is often lighting above the user. When there is lighting above, the upper body bends backward when breathing in, so the brightness of the chest and abdomen increases, and when breathing out, the upper body returns and the brightness decreases. Therefore, the calculation unit 12 can calculate the breathing cycle of breathing in and out by, for example, determining the change period of the average brightness of the user's chest region. Also, the calculation unit 12 can calculate the breathing cycle of breathing in and out by determining the change period of the average brightness of the user's abdominal region. Also, the calculation unit 12 may calculate the optical flow of the chest and abdomen, respectively. In this exemplary embodiment, the optical flow is the movement of the brightness of pixels. For example, the calculation unit 12 may obtain the direction in which the brightness of the pixels in the chest region and abdominal region changes on average. The optical flow can be obtained using a known application. The calculation unit 12 can obtain the breathing cycle by obtaining the change period of the optical flow.

It is preferable for the user to adjust the position of the camera or themselves so that the chest and abdomen are within the field of view of the onboard camera. This allows images of the chest and abdomen to be acquired with one camera. If image data of the chest and abdomen are acquired with two cameras, respectively, it is necessary to acquire two sets of image data with aligned time axes.

Alternatively, the calculation unit 12 may use distance information between the user's chest and abdomen to calculate the breathing cycle of the user's chest and abdomen, respectively. Specifically, for example, when distance information between the chest and abdomen is acquired from the front of the user, it can be determined that the user has exhaled when the distance to the chest or abdomen becomes relatively large. Also, it can be determined that the user has inhaled when the distance to the chest or abdomen becomes relatively small. The calculation unit 12 can calculate the breathing cycle of the chest and abdomen, respectively, from the change period of such distance information.

The derivation unit 13 derives the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle. It is known that abdominal breathing synchronizes with the chest respiratory cycle. In other words, the stronger the synchronization, that is, when the chest expands, the abdomen expands, and when the chest contracts, the abdomen contracts, the more effective the abdominal breathing is. Therefore, the degree of synchronization is an index for determining whether abdominal breathing is being performed properly. The effect of abdominal breathing is that it allows relaxation, or that the parasympathetic nervous system becomes more dominant than the sympathetic nervous system. There are no particular limitations on the method of deriving the degree of synchronization of the respiratory cycles, but as an example, when the chest respiratory cycle and the abdominal respiratory cycle are each represented by a graph, the squared error between the two may be calculated. In that case, it is determined that the smaller the squared error, the greater the degree of synchronization. The derivation unit 13 is one form of derivation means described in the claims.

The output unit 14 outputs the degree of synchronization derived. Specifically, the output unit 14 generates output data of the degree of synchronization derived by the derivation unit 13 and outputs it to the outside. The outside is outside the information processing device 1, and is, for example, a display of a personal computer including the information processing device 1, a printing device, another personal computer, another mobile device, etc. The information processing device 1 may be provided with an input/output interface that communicates information with the outside via a wired or wireless connection. The output unit 14 is one form of the output means described in the claims.

The image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 may be partially or entirely located in different housings. The image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 may be partially or entirely located on the cloud. In such cases, each unit is connected to each other so that information can be communicated. This also applies to the information processing device according to the following exemplary embodiment.

As described above, the information processing device 1 according to this exemplary embodiment is configured to include an image acquisition means for acquiring image data of a user, a calculation means for calculating the user's chest respiratory cycle and abdominal respiratory cycle by referring to information contained in the image data, a derivation means for deriving the degree of synchronization between the chest respiratory cycle and abdominal respiratory cycle, and an output means for outputting the derived degree of synchronization. A user can start up the information processing device 1 as needed and capture an image of themselves to find out the degree to which the chest and abdominal respiratory cycles are synchronized. Therefore, the information processing device 1 according to this exemplary embodiment has the effect of allowing the user to check whether or not they are performing abdominal breathing.

(Flow of information processing method)
The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1.

As shown in the figure, information processing method S1 includes processes S11 to S14. Process S11 is a process in which at least one processor (image acquisition unit 11) acquires image data of a user. The meaning of image data is as explained for information processing device 1.

Process S12 is a process in which at least one processor (calculation unit 12) refers to information contained in the image data and calculates the user's chest respiratory cycle and abdominal respiratory cycle. The meanings of the chest and abdominal respiratory cycles are as explained in the information processing device 1.

Process S13 is a process in which at least one processor (derivation unit 13) derives the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle. The meaning of synchronization of the respiratory cycle is as explained in the information processing device 1.

Process S14 is a process in which at least one processor (output unit 14) outputs the derived degree of synchronization. The meaning of the output is as explained in the information processing device 1.

As described above, the information processing method S1 according to this exemplary embodiment employs a configuration that includes acquiring image data of a user, calculating the user's chest respiratory cycle and abdominal respiratory cycle by referring to information contained in the image data, deriving the degree of synchronization between the chest respiratory cycle and abdominal respiratory cycle, and outputting the derived degree of synchronization. Therefore, the information processing method S1 according to this exemplary embodiment has the effect of allowing the user to check whether or not they are performing abdominal breathing.

(Configuration of Information Processing System 2)
The configuration of the information processing system 2 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing system 2. As shown in the figure, the information processing system 2 includes an imaging device 30 and a control unit 10.

The control unit 10 includes an image acquisition unit 11, a calculation unit 12, a derivation unit 13, an output unit 14, at least one processor 21, and a memory 22. The processor 21 can be configured using at least one general-purpose processor such as an MPU (Micro Processing Unit) or a CPU (Central Processing Unit). The processor 21 may also include a dedicated processor configured using an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a PLD (Programmable Logic Device), or the like.

The memory 22 may include multiple types of memory, such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The memory 22 may also include an internal or external memory, such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive). As an example, the processor 21 realizes the functions of the image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 by expanding various control programs recorded in the ROM of the memory 22 into the RAM and executing them.

The imaging device 30 captures an image of the user. The imaging device 30 transmits image data of the captured image to the control unit 10. The transmitted image data is stored in the memory 22. The image acquisition unit 11 acquires image data of the image captured by the imaging device 30. For example, the image acquisition unit 11 acquires image data of the user stored in the memory 22. The calculation unit 12 calculates the user's chest respiratory cycle and abdominal respiratory cycle by referring to information included in the image data. The derivation unit 13 derives the degree of synchronization between the chest respiratory cycle and abdominal respiratory cycle. The output unit 14 outputs the derived degree of synchronization. The detailed functions of each unit are the same as those described for each unit of the information processing device 1, so a description thereof will be omitted here.

The image acquisition unit 11, calculation unit 12, derivation unit 13, and output unit 14 may be partially or entirely located on the cloud. In this case, each unit is connected to each other so that information can be communicated. This also applies to the information processing system according to the following exemplary embodiment.

As described above, the information processing system 2 according to this exemplary embodiment employs a configuration including an imaging device that captures an image of the user, an image acquisition means that acquires image data of the image, a calculation means that calculates the user's chest respiratory cycle and abdominal respiratory cycle by referring to information contained in the image data, a derivation means that derives the degree of synchronization between the chest respiratory cycle and abdominal respiratory cycle, and an output means that outputs the derived degree of synchronization. Therefore, the information processing system 2 according to this exemplary embodiment has the effect of allowing the user to check whether or not they are performing abdominal breathing.

Exemplary embodiment 2
A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

(Configuration of information processing device 1A)
The configuration of the information processing device 1A according to this exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the information processing device 1A. As shown in the figure, the information processing device 1A includes an image acquisition unit 11, a calculation unit 12, a derivation unit 13, and an output unit 14. The basic functions of the image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 are similar to the functions of the image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 described in the exemplary embodiment 1, but may further include new functions.

The information processing device 1A may include an image analysis unit 15. The image analysis unit 15 may estimate the user's skeletal structure from the image data acquired by the image acquisition unit 11, and define the chest region and abdominal region. FIG. 5 is a schematic diagram showing the user's chest region and abdominal region defined by the image analysis unit 15. Image 50 is an image of the user acquired by the imaging device 30, and shows the determined chest and abdominal regions (regions of interest) within the image. Region information of the chest region and abdominal region may be transmitted to the calculation unit.

In the example shown in FIG. 5, the image analysis unit 15 divides the chest area into two areas, chest A on the left side and chest B on the right side. The image analysis unit 15 also divides the abdominal area into two areas, abdominal A on the left side and abdominal B on the right side. Basically, the pattern of change in brightness, optical flow, distance, etc. of the chest and abdomen due to breathing (hereinafter, "brightness, optical flow, distance, etc." will be referred to as "brightness, etc.") is similar on the right and left sides. Therefore, it is sufficient to divide the two areas, the chest area and the abdominal area. However, by dividing the areas into the right and left sides, it is possible to determine whether the user is in a stationary state facing forward. In other words, if the pattern of change in brightness, etc. is different on the right and left sides, it can be determined that the user is not in a stationary state, such as when the user is looking to the side. The image analysis unit 15 may transmit the pattern of change in the right and left sides of at least either the chest or the abdomen to the calculation unit 12. The calculation unit 12 may calculate the degree of agreement between the patterns of changes in the right and left sides of the chest or abdomen, and if the degree of agreement is equal to or less than a predetermined value, may determine that the posture is inappropriate for evaluating abdominal breathing and may stop calculating the respiratory cycle. Also, the calculation unit 12 may notify the user that the posture is inappropriate for evaluating abdominal breathing.

The image analysis unit 15 may also analyze information such as luminance contained in the image data to derive changes in luminance over time. The derived changes in luminance over time are transmitted to the calculation unit 12. The calculation unit 12 may calculate the user's chest respiratory cycle and abdominal respiratory cycle from the analysis information such as luminance of the chest and abdomen acquired from the image analysis unit 15.

FIG. 6 is a graph showing the user's chest respiratory cycle and abdominal respiratory cycle calculated by the calculation unit 12. In the figure, the solid line represents the abdominal region respiratory cycle, and the dotted line represents the chest region respiratory cycle. The derivation unit 13, for example, calculates the squared error between the solid line graph and the dotted line graph, and derives the degree of synchronization between the chest respiratory cycle and abdominal respiratory cycle as an evaluation value from this. The output unit 14 may also output the respiratory cycle graph to the user's display. Specifically, the output unit 14 may output the user's chest respiratory cycle and abdominal respiratory cycle in a manner that allows them to compare them. This allows the user to visually recognize whether the chest respiratory cycle and abdominal respiratory cycle are in agreement.

The graph in FIG. 6 may, for example, display only frequency components close to the respiratory frequency through a band-pass filter. The horizontal axis may represent time, and the vertical axis may represent standardized values, for example, with an average of 0 and a standard deviation of 1 within the display range of the graph. The display range of the graph is arbitrary, but taking into account the standard respiratory cycle, it may be possible to display a moving time range, for example, from 10 to 20 seconds. Furthermore, marks may be displayed on the convex parts of the graph to make it easier to visually compare cycles.

The information processing device 1A may include a judgment unit 16. The judgment unit 16 judges the degree of synchronization derived by the derivation unit 13 by comparing it with a threshold value. The judgment result may be an evaluation value indicating the degree of synchronization. The evaluation value may be, for example, a numerical value of squared error, or an evaluation value between 0% and 100%. Alternatively, the evaluation value may be a qualitative expression such as good, average, or poor. The output unit 14 may output the judgment result judged by the judgment unit 16. The judgment unit 16 is one form of the judgment means described in the claims.

The determination unit 16 may determine that abdominal breathing is being performed appropriately, for example, if the degree of synchronization is greater than a preset threshold. Conversely, the determination unit 16 may determine that abdominal breathing is not being performed appropriately, if the degree of synchronization is equal to or less than a preset threshold. There may be multiple thresholds. That is, the determination unit 16 may determine the degree of synchronization by dividing it into three or more regions using multiple thresholds. For example, the determination unit 16 may set a first threshold and a second threshold that is greater than the first threshold. In that case, the degree of synchronization is divided into three ranges: a range equal to or less than the first threshold, a range greater than the first threshold and equal to or less than the second threshold, and a range greater than the second threshold. Then, for each range, abdominal breathing may be determined to be good, normal, poor, etc.

The judgment unit 16 may select advice for the user's abdominal breathing as the judgment result based on the level of the evaluation value. In other words, the judgment result may include at least one of the evaluation value indicating the level of synchronization and advice for the user's abdominal breathing. FIG. 7 is an example of advice selected by the judgment unit 16. FIG. 7 shows an example in which the judgment unit 16 selects advice in four levels. For example, when the level of the evaluation value is "bad", advice such as "It seems that you are not breathing abdominally" or "Please take a short break" is selected as the advice. When the level of the evaluation value is "slightly bad", advice such as "It seems that you are slightly tense" is selected. When the level of the evaluation value is "slightly good", advice such as "Relaxed" is selected. When the level of the evaluation value is "good", advice such as "Ideal abdominal breathing is possible" is selected. The output unit 14 may output advice instead of or in addition to the evaluation value of the judgment unit 16.

The information processing device 1A may include a change unit 17. The change unit 17 changes the threshold value depending on the situation when the user's image data is acquired. Specifically, the user's work environment may be estimated from the user's image data, and the threshold value may be changed according to the estimated work environment. For example, the threshold value may be changed depending on whether the user is working at home or working in an office with many workers. The change unit 17 may also change the threshold value depending on the number of times the degree of respiratory synchronization is evaluated. For example, the threshold value may be increased as the number of times increases, as the user becomes accustomed to abdominal breathing. The change unit 17 may also change the threshold value to be increased as the measurement time increases. The information processing device 1A may be configured to allow the user to change the threshold value. Alternatively, the information processing device 1A may be configured to allow the user to select the situation or environment in which the measurement is performed. The change unit 17 is one form of the change means described in the claims.

The information processing device 1A may include an activation estimation unit 18. The activation estimation unit 18 estimates activation of the user's parasympathetic nerves. FIG. 8 is a block diagram showing the configuration of the activation estimation unit 18. As shown in the figure, the activation estimation unit 18 includes a vital sign estimation unit 181, a vital sign change calculation unit 182, and a vital sign change determination unit 183. The activation estimation unit 18 is one form of the activation estimation means described in the claims.

In this exemplary embodiment, the vital values are, for example, the RR interval, CVRR, blood pressure, SpO2, etc. SpO2 is the percutaneous arterial oxygen saturation. The RR interval is the time from one ventricular excitation to the next, and the number of times the ventricle contracts per minute is calculated by dividing 60 seconds by the RR interval (seconds). The CVRR is calculated by the following formula (1) by calculating the average value and standard deviation of the RR intervals of consecutive heartbeats.
CVRR = (standard deviation / average value) x 100 (%) ... formula (1)

Such vital values can be estimated using image data, or estimated (measured) using a vital value measuring device. Thus, the vital value estimation unit 181 may estimate the user's vital value using image data or vital value measurement information. A publicly known application that determines the vital value from image data, or a measuring device that is attached to the human body to measure the vital value, can be used.

The vital sign change calculation unit 182 calculates the change in the vital sign value estimated by the vital sign estimation unit 181. The vital sign change determination unit 183 estimates the change in the activity level of the user's parasympathetic nerves by referring to the change in the estimated vital sign. FIG. 9 shows an example of the change in the vital sign value that occurs when relaxing by abdominal breathing. For example, when the parasympathetic nerves are activated (become dominant), there is a tendency for the RR interval to increase, the CVRR to increase, the blood pressure to decrease, and the SpO2 to increase. Conversely, when the parasympathetic nerves are deactivated (become inferior), the RR interval, CVRR, blood pressure, and SpO2 change in the opposite manner. The vital sign change determination unit 183 estimates that the user's parasympathetic nerves have been activated or deactivated as described above by referring to the change in the vital sign value calculated by the vital sign change calculation unit 182. The output unit 14 may output the change in the estimated activity level of the parasympathetic nerves. Furthermore, the determination unit 16 may determine the effectiveness of abdominal breathing based on whether or not abdominal breathing is effective and to what extent it is effective, by referring to the change in the estimated vital values.

The information processing device 1A may be equipped with a breathing rhythm presentation unit 19. When the user wants to check whether or not they are performing appropriate abdominal breathing, the breathing rhythm presentation unit 19 displays the periodic rhythm of inhalation and exhalation times on a user display (not shown). The display form is not limited, but may be, for example, a curve that changes like a sine curve, or a figure whose height goes up and down like a bar graph. Alternatively, the user may be notified by music, voice, etc. through a speaker or headphones (not shown). The period is a rhythm that is considered to be suitable for abdominal breathing. By displaying such a rhythm, the user can be assisted in performing appropriate abdominal breathing.

As shown in FIG. 4, a trained artificial intelligence (AI) 40 may be used in the information processing device 1A. For example, the calculation unit 12 may use the AI 40 to analyze an image and calculate the user's chest breathing cycle and abdominal breathing cycle, respectively. Furthermore, the derivation unit 13 may use the AI 40 to derive the degree of synchronization between the chest breathing cycle and abdominal breathing cycle. Furthermore, the processing performed by the calculation unit 12 and the derivation unit 13 may be performed by the AI 40. The AI 40 may be trained using images of the chest movement and abdominal movement of a person skilled in abdominal breathing, and the chest movement and abdominal movement of a person not skilled in abdominal breathing.

Also, the AI 40 may be used to perform the process of defining the user's chest and abdomen from an image (at least a part of the process performed by the image analysis unit 15), or the process of determining the effectiveness of abdominal breathing by referring to changes in vital signs (at least a part of the process performed by the activation estimation unit 18).

(Effects of information processing device 1A)
As described above, the information processing device 1A according to this exemplary embodiment employs a configuration including the determination unit 16 in addition to the information processing device 1 according to the exemplary embodiment. Therefore, according to the information processing device 1A according to this exemplary embodiment, in addition to the effects of the information processing device 1 according to the exemplary embodiment 1, the effect of being able to determine whether the user is performing appropriate abdominal breathing can be obtained. Furthermore, by providing the change unit 17 that changes the threshold for determination, the threshold can be changed according to the situation, and abdominal breathing can be determined in detail. Furthermore, by further providing the activation estimation unit 18, abdominal breathing can be determined by referring to the specific result of parasympathetic nerve activation. Furthermore, by providing the breathing rhythm presentation unit 19, assistance can be provided to the user to perform appropriate abdominal breathing.

Exemplary embodiment 3
A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first or second exemplary embodiment are denoted by the same reference numerals, and the description thereof will not be repeated.

(Configuration of Information Processing System 2A)
The configuration of an information processing system 2A according to this exemplary embodiment will be described with reference to the drawings. Fig. 10 is a block diagram showing the configuration of the information processing system 2A. As shown in the figure, the information processing system 2A includes a control unit 10A, an imaging device 30, and an AI 40.

The control unit 10A includes an image acquisition unit 11, a calculation unit 12, a derivation unit 13, an output unit 14, at least one processor 21, and a memory 22. The basic functions of the image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 are similar to the functions of the image acquisition unit 11, the calculation unit 12, the derivation unit 13, and the output unit 14 described in exemplary embodiment 1, but may further include new functions. The processor 21 and the memory 22 may have the same configuration as the processor 21 and the memory 22 described in exemplary embodiment 1.

The information processing system 2A may include an image analysis unit 15, a determination unit 16, a change unit 17, an activation estimation unit 18, or a respiratory rhythm presentation unit 19. The functions of these units are similar to those of the units described in the information processing device 1A of the exemplary embodiment 2, and therefore will not be described here.

(Effects of Information Processing System 2A)
As described above, the information processing system 2A according to this exemplary embodiment employs a configuration including the image analysis unit 15, the determination unit 16, the change unit 17, the activation estimation unit 18, or the respiratory rhythm presentation unit 19 in addition to the configuration of the information processing system 2 according to the exemplary embodiment 1. Therefore, according to the information processing system 2A according to this exemplary embodiment, in addition to the effects of the information processing system 2 according to the exemplary embodiment 1, it is possible to obtain an effect of determining whether or not the user is performing appropriate abdominal breathing. Furthermore, by providing the change unit 17 that changes the threshold for determination, it is possible to change the threshold according to the situation, and it is possible to perform a detailed determination of abdominal breathing. Furthermore, by further providing the activation estimation unit 18, it is possible to determine abdominal breathing by referring to the specific result of activation of the parasympathetic nerves. Furthermore, by providing the respiratory rhythm presentation unit 19, it is possible to assist the user in performing appropriate abdominal breathing.

Exemplary embodiment 4
A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

(Configuration of information processing device 1B)
The configuration of the information processing device 1B according to this exemplary embodiment will be described with reference to FIG. 12. FIG. 12 is a block diagram showing the configuration of the information processing device 1B according to the exemplary embodiment 4. The information processing device 1B includes an image acquisition unit 11, a calculation unit 12, and an output unit 14. The image acquisition unit 11 and the calculation unit 12 have the same functions as the image acquisition unit 11 and the calculation unit 12 described in the exemplary embodiment 1, and therefore their description will be omitted. The output unit 14 outputs the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle. In this way, the output unit 14 may derive the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle and output it. In addition, one or both of the process of calculating the chest respiratory cycle and the abdominal respiratory cycle of the user performed by the calculation unit 12 and the process of deriving the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle performed by the output unit 14 may be performed by AI.

The information processing device 1B having the above configuration has the effect of allowing the user to check whether or not they are performing abdominal breathing.

(Flow of information processing method S2)
The flow of the information processing method S2 according to this exemplary embodiment will be described with reference to Fig. 13. Fig. 13 is a flow diagram showing the flow of the information processing method S2.

As shown in the figure, information processing method S1 includes processes S21 to S23. Process S21 is a process in which at least one processor (image acquisition unit 11) acquires image data of a user. The meaning of image data is as explained in the information processing device 1.

Process S22 is a process in which at least one processor (calculation unit 12) refers to information contained in the image data and calculates the user's chest respiratory cycle and abdominal respiratory cycle. The meanings of the chest and abdominal respiratory cycles are as explained in the information processing device 1.

Process S23 is a process in which at least one processor (output unit 14) outputs the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle. The meaning of the synchronization of the respiratory cycle and the meaning of the output are as explained in the information processing device 1.

The information processing method S2 configured as described above has the effect of allowing the user to check whether or not they are performing abdominal breathing.

(Configuration of Information Processing System 2B)
The configuration of an information processing system 2B according to this exemplary embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing the configuration of the information processing system 2B. As shown in the figure, the information processing system 2B includes an imaging device 30 and a control unit 10. The imaging device 30 is as described in the exemplary embodiment 1, and therefore description thereof will be omitted.

The control unit 10 includes an image acquisition unit 11, a calculation unit 12, an output unit 14, at least one processor 21, and a memory 22. The image acquisition unit 11 and the calculation unit 12 have the same functions as the image acquisition unit 11 and the calculation unit 12 described in the exemplary embodiment 1, and therefore their description will be omitted. The output unit 14 outputs the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle. In this manner, the output unit 14 may derive the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle and output it. In addition, one or both of the process of calculating the chest respiratory cycle and the abdominal respiratory cycle of the user performed by the calculation unit 12 and the process of deriving the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle performed by the output unit 14 may be performed by AI. The configuration of the processor 21 and the memory 22 is as described in the exemplary embodiment 1.

The information processing system 2B configured as described above has the effect of allowing the user to check whether or not they are performing abdominal breathing.

[Software implementation example]
Some or all of the functions of the information processing devices 1, 1A, 1B and the information processing systems 2, 2A, 2B (hereinafter referred to as "information processing devices 1, etc.") may be realized by hardware such as an integrated circuit (IC chip), or by software.

In the latter case, the information processing device 1 etc. is realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 11. Computer C has at least one processor C1 and at least one memory C2. Memory C2 stores program P for operating computer C as information processing device 1 etc. In computer C, processor C1 reads and executes program P from memory C2, thereby realizing each function of information processing device 1 etc.

The processor C1 may be, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination of these.

Computer C may further include a RAM (Random Access Memory) for expanding program P during execution and for temporarily storing various data. Computer C may further include a communications interface for sending and receiving data to and from other devices. Computer C may further include an input/output interface for connecting input/output devices such as a keyboard, mouse, display, and printer.

The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[Additional Note 1]
The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[Additional Note 2]
Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the aspects described below.

(Appendix 1)
An information processing device comprising: an image acquisition means for acquiring image data of a user; a calculation means for calculating a chest respiratory cycle and an abdominal respiratory cycle of the user by referring to information contained in the image data; and an output means for outputting a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

The above configuration allows the user to check whether or not they are performing abdominal breathing.

(Appendix 2)
2. The information processing device according to claim 1, further comprising a determination means for determining the derived degree of synchronization by comparing it with a threshold value.

The above configuration makes it possible to determine whether the user is performing proper abdominal breathing.

(Appendix 3)
The information processing device according to claim 2, further comprising a change unit that changes the threshold value depending on a situation when the image data of the user is acquired.

The above configuration allows the threshold to be changed depending on the situation, enabling more precise determination of abdominal breathing.

(Appendix 4)
The information processing device according to claim 2 or 3, wherein the output means outputs a determination result determined by the determination means.

The above configuration allows the user to check the judgment result.

(Appendix 5)
5. The information processing device according to claim 2, wherein the determination result includes at least one of an evaluation value indicating a degree of synchronization and advice on abdominal breathing for the user.

The above configuration makes it easier for users to understand how well they are performing abdominal breathing.

(Appendix 6)
6. The information processing device according to any one of claims 2 to 5, further comprising an activation estimation means for estimating activation of the parasympathetic nerves of the user.

The above configuration makes it possible to determine abdominal breathing by referring to the specific result of parasympathetic nerve activation.

(Appendix 7)
The information processing device according to claim 6, wherein the activation estimation means estimates the user's vital value using the image data or vital value measurement information.

The above configuration allows parasympathetic nerve activation to be determined using specific vital signs, making it possible to make a scientifically proven determination.

(Appendix 8)
The information processing device according to claim 7, wherein the determination means determines an effectiveness of abdominal breathing of the user by referring to a change in the estimated vital value.

The above configuration allows parasympathetic nerve activation to be determined by specific changes in vital signs, making it possible to make a scientifically proven determination.

(Appendix 9)
The information processing device according to any one of claims 1 to 8, wherein the output means outputs the respiratory cycle of the chest and the respiratory cycle of the user in a comparable manner.

The above configuration allows the user to visually understand how well they are performing abdominal breathing.

(Appendix 10)
An information processing system comprising: an imaging device that captures an image of a user; an image acquisition means that acquires image data of the image; a calculation means that calculates the chest respiratory cycle and abdominal respiratory cycle of the user by referring to information contained in the image data; and an output means that outputs the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

(Appendix 11)
11. The information processing system according to claim 10, further comprising a determination means for determining the derived degree of synchronization by comparing it with a threshold value.

(Appendix 12)
12. The information processing system according to claim 11, further comprising a change means for changing the threshold value depending on a situation when the image data of the user is acquired.

(Appendix 13)
13. The information processing system according to claim 11, wherein the output means outputs a determination result determined by the determination means.

The above configuration allows the user to check the judgment result.

(Appendix 14)
The information processing system according to claim 13, wherein the determination result includes at least one of an evaluation value indicating the degree of synchronization and advice on abdominal breathing for the user.

(Appendix 15)
15. The information processing system according to any one of appendices 11 to 14, further comprising an activation estimation means for estimating activation of the user's parasympathetic nerves.

(Appendix 16)
The information processing system according to claim 15, wherein the activation estimation means estimates the user's vital values using the image data or vital value measurement information.

(Appendix 17)
The information processing system according to claim 16, wherein the determination means determines the effectiveness of abdominal breathing of the user by referring to the estimated change in the vital value.

The above configuration allows parasympathetic nerve activation to be determined based on specific changes in vital signs, making it possible to make a scientifically proven determination.

(Appendix 18)
An information processing method including: acquiring image data of a user; calculating a chest respiratory cycle and an abdominal respiratory cycle of the user by referring to information contained in the image data; and outputting a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.

The above method allows users to check whether they are performing abdominal breathing.

(Appendix 19)
An information processing program for causing a computer to function as the information processing device described in appendix 1, the information processing program causing the computer to function as the image acquisition means, the image acquisition means, and the output means.

(Appendix 20)
A non-transitory computer-readable recording medium having recorded thereon the information processing program described in appendix 19.

(Appendix 21)
An information processing device described in any one of Appendix 1 to 9, further comprising a derivation means for deriving a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle, and the output means outputs the degree of synchronization derived by the derivation means.

(Appendix 22)
18. An information processing system according to any one of appendices 10 to 17, further comprising a derivation means for deriving a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle, wherein the output means outputs the degree of synchronization derived by the derivation means.

(Appendix 23)
19. The information processing method of claim 18, further comprising: deriving a degree of synchronization between the thoracic respiratory cycle and the abdominal respiratory cycle after calculating the thoracic respiratory cycle and the abdominal respiratory cycle, respectively.

[Additional Note 3]
A part or all of the above-described embodiments can be further expressed as follows.

An information processing device comprising at least one processor that executes an acquisition process for acquiring image data of a user, a calculation process for calculating the chest respiratory cycle and abdominal respiratory cycle of the user by referring to information contained in the image data, and an output process for outputting the degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle.
The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process, the calculation process, and the output process. The program may be recorded in a computer-readable, non-transitory, tangible recording medium.

Reference Signs List 1, 1A:

Information processing device

10, 10A: Control unit 11: Image acquisition unit 12: Calculation unit 13: Derivation unit 14: Output unit 15: Image analysis unit 16: Determination unit 17: Change unit 18: Activation estimation unit 181: Vital sign estimation unit 182: Vital sign change calculation unit 183: Vital sign change determination unit 19: Respiratory rhythm presentation unit 2, 2A: Information processing system 21: Processor 22: Memory 30: Imaging device 40: AI

Claims

Image acquisition means for acquiring image data of a user;
a calculation means for calculating a chest respiratory cycle and an abdominal respiratory cycle of the user by referring to information included in the image data;
an output means for outputting a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle;
An information processing device comprising:
The information processing device according to claim 1, further comprising a determination means for determining the derived degree of synchronization by comparing it with a threshold value.
The information processing device according to claim 2, further comprising a change means for changing the threshold value depending on the situation when the image data of the user is acquired.
The information processing device according to claim 2 or 3, wherein the output means outputs the determination result determined by the determination means.
The information processing device according to claim 4, wherein the determination result includes at least one of an evaluation value indicating the degree of synchronization and advice on abdominal breathing for the user.
The information processing device according to any one of claims 2 to 5, further comprising an activation estimation means for estimating activation of the user's parasympathetic nerves.
The information processing device according to claim 6, wherein the activation estimation means estimates the user's vital values using the image data or vital value measurement information.
The information processing device according to claim 7, wherein the determining means determines the effectiveness of the user's abdominal breathing by referring to the estimated change in the vital value.
The information processing device according to any one of claims 1 to 8, wherein the output means outputs the user's chest respiratory cycle and abdominal respiratory cycle in a manner that allows comparison.
An imaging device that captures an image of a user;
image acquisition means for acquiring image data of the image;
a calculation means for calculating a chest respiratory cycle and an abdominal respiratory cycle of the user by referring to information included in the image data;
an output means for outputting a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle;
An information processing system comprising:
The information processing system according to claim 10, further comprising a determination means for determining the derived degree of synchronization by comparing it with a threshold value.
The information processing system according to claim 11, further comprising a change means for changing the threshold value depending on the situation when the image data of the user is acquired.
The information processing system according to claim 11 or 12, wherein the output means outputs the determination result determined by the determination means.
The information processing system according to claim 13, wherein the determination result includes at least one of an evaluation value indicating the degree of synchronization and advice on abdominal breathing for the user.
The information processing system according to any one of claims 11 to 14, further comprising an activation estimation means for estimating activation of the user's parasympathetic nerves.
The information processing system according to claim 15, wherein the activation estimation means estimates the vital values of the user using the image data or vital value measurement information.
The information processing system according to claim 16, wherein the determining means determines the effectiveness of the user's abdominal breathing by referring to the estimated change in the vital value.
Obtaining image data of a user;
calculating a chest respiratory cycle and an abdominal respiratory cycle of the user by referring to information included in the image data;
outputting a degree of synchronization between the chest respiratory cycle and the abdominal respiratory cycle;
An information processing method comprising:
An information processing program for causing a computer to function as the information processing device according to claim 1, the information processing program causing the computer to function as the image acquisition means, the image acquisition means, and the output means.
A non-transitory computer-readable recording medium having the information processing program according to claim 19 recorded thereon.
The information processing device according to any one of claims 1 to 9, further comprising a derivation means for deriving a degree of synchronization between the respiratory cycle of the chest and the respiratory cycle of the abdomen, and the output means outputs the degree of synchronization derived by the derivation means.
The information processing system according to any one of claims 10 to 17, further comprising a derivation means for deriving a degree of synchronization between the respiratory cycle of the chest and the respiratory cycle of the abdomen, and the output means outputs the degree of synchronization derived by the derivation means.
The information processing method according to claim 18 , further comprising: deriving a degree of synchronization between the thoracic respiratory cycle and the abdominal respiratory cycle after calculating the thoracic respiratory cycle and the abdominal respiratory cycle, respectively.