KR20220079172A - Method for detecting hand washing behavior and electronic device thereof - Google Patents

Abstract

The electronic device may include at least one motion sensor, a barometric pressure sensor, a display, and the at least one motion sensor, the barometric pressure sensor, and at least one processor. The at least one processor detects a first event occurring to the user of the electronic device based on the motion information obtained through the at least one motion sensor, and the first event based on the barometric pressure data obtained through the barometric pressure sensor It is determined whether the detection of the event is misrecognized, and when the detection of the first event is normal recognition, a user interface related to a score determined based on motion information obtained through the at least one motion sensor is displayed on the display. can In addition to this, various embodiments identified through the specification are possible.

Description

METHOD FOR DETECTING HAND WASHING BEHAVIOR AND ELECTRONIC DEVICE THEREOF

Embodiments disclosed in this document relate to a method of determining a hand washing operation through an electronic device and displaying a user interface related thereto, and an electronic device thereof.

Recently, there is an increasing demand for wearable devices that are not simply portable electronic devices, but can be worn on the body. Consumers demand a wearable device that provides more useful functions, and accordingly, the wearable device provides various functions such as heart rate measurement or step count measurement.

In addition, as environmental problems such as fine dust and infection problems such as viruses have recently become issues, not only the above functions, but also a function to check the cleanliness is required. In particular, functions for recognizing basic actions such as hand washing and checking and informing of cleanliness are being developed.

In order to check the cleanliness, it is necessary to distinguish and judge the user's washing behavior. However, the conventional wearable device recognizes the sound of water and determines that there is movement of the electronic device as a hand washing situation, and thus cannot distinguish between a hand washing situation and a showering situation similar thereto.

In addition, even if the hand washing situation is sensed, the conventional wearable device guides the user to wash their hands more using a timer when the user finishes hand washing sooner than the recommended time. However, this is simply a judgment considering only the total time of hand washing, and it was not easy to determine whether the hands were washed in the correct way because the hand washing area or time for each area was not taken into account. Therefore, it is impossible to provide a correct guide for hand washing based on the user's hand washing behavior.

According to various embodiments of the present disclosure, an electronic device and an electronic device that detect a user's hand washing event through an electronic device, determine whether a user's hand washing event is not a similar situation such as a showering situation, and provide a score and guide for hand washing control method can be provided.

The electronic device according to an embodiment disclosed herein includes at least one motion sensor, a barometric pressure sensor, a display, and at least one operatively connected to the at least one motion sensor, the barometric pressure sensor, and the display. It may include a processor. The at least one processor detects a first event occurring to the user of the electronic device based on the motion information obtained through the at least one motion sensor, and the first event based on the barometric pressure data obtained through the barometric pressure sensor It is determined whether the detection of the event is misrecognized, and when the detection of the first event is normal recognition, a user interface related to a score determined based on motion information obtained through the at least one motion sensor is displayed on the display. can

In addition, the method of operating an electronic device according to an embodiment of the present disclosure includes an operation of detecting a first event occurring to a user of the electronic device based on motion information obtained through at least one motion sensor, and an air pressure sensor. Determining whether or not the detection of the first event is erroneously recognized based on the atmospheric pressure data obtained through the and displaying a user interface related to the determined score on a display.

According to various embodiments disclosed herein, the hand washing situation may be determined by distinguishing it from other situations.

In addition, according to various embodiments, convenience may be provided to the user through a score and guide for hand washing.

In addition, various effects directly or indirectly identified through this document may be provided.

1A illustrates an electronic device according to an embodiment.
1B illustrates an electronic device according to an exemplary embodiment.
2 illustrates a hardware configuration of an electronic device according to an embodiment.
3 is a flowchart illustrating a process of determining a user's hand washing situation and displaying a user interface related thereto in an electronic device according to an exemplary embodiment.
4 is a flowchart illustrating a process of determining a hand washing situation in an electronic device according to an exemplary embodiment.
5 is a diagram illustrating acceleration data when a first event that is a hand washing situation occurs in an electronic device according to an exemplary embodiment.
6 is a flowchart illustrating a process of determining whether detection of a first event is normally recognized in an electronic device according to an exemplary embodiment.
7 is a diagram illustrating barometric pressure data obtained by an electronic device through a barometric pressure sensor, according to an exemplary embodiment.
8 is a diagram illustrating air pressure data obtained through an air pressure sensor in a situation in which an electronic device washes hands according to an exemplary embodiment.
9 is a diagram illustrating air pressure data obtained through an air pressure sensor in a situation in which an electronic device takes a shower, according to an exemplary embodiment.
10 is a flowchart illustrating a process of calculating a score for hand washing in an electronic device according to an exemplary embodiment.
11 is a flowchart illustrating a process of determining a hand washing part in an electronic device according to an exemplary embodiment.
12 is a diagram illustrating data acquired through a gyro sensor in a situation in which an electronic device washes hands, according to an exemplary embodiment.
13 is a diagram of an interface illustrating a portion to be additionally washed in an electronic device according to an exemplary embodiment.
14 is a diagram for displaying a score for hand washing and providing a notification in an electronic device according to an exemplary embodiment.
15 is a diagram for providing information on hand washing activity and a notification in an electronic device according to an exemplary embodiment.
16 is a diagram of an interface for changing a score calculation criterion for hand washing in an electronic device according to an embodiment.
17 is a block diagram of an electronic device in a network environment, according to various embodiments of the present disclosure;

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. However, this document is not intended to limit the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of the present invention.

1A illustrates an electronic device 100 (eg, the electronic device 1701 of FIG. 17 ) according to an embodiment.

Referring to FIG. 1A , the display 110 may be disposed on the front surface of the electronic device 100 according to an embodiment. In an embodiment, the display 110 may occupy most of the front surface of the electronic device 100 . A display 110 and a bezel 120 region surrounding at least some edges of the display 110 may be disposed on a portion of the front surface of the electronic device 100 .

In one embodiment, referring to FIG. 1A , although the display 110 is illustrated as including a flat area, the display 110 includes a flat area and the electronic device 100 in the flat area. ) may include a curved area extending toward the side.

In an embodiment, although the shape of the front surface of the electronic device 100 and the shape of the display 110 are shown in a circle with reference to FIG. 1A , it may be implemented in various shapes such as a rectangle, an ellipse, and a polygon with a pentagon or more. .

In an embodiment, a fingerprint sensor for recognizing a user's fingerprint may be included in an area of the display 110 . Since the fingerprint sensor is disposed on a lower layer of the display 110 , the fingerprint sensor may not be recognized by the user or may be difficult to be recognized. In addition, a sensor for additional user/biometric authentication in addition to the fingerprint sensor may be disposed in a portion of the display 110 . In another embodiment, a sensor for user/biometric authentication may be disposed in one area of the bezel 120 . For example, the IR sensor for iris authentication may be exposed through one area of the display 110 or exposed through one area of the bezel 120 .

In an embodiment, at least one camera and/or at least one sensor may be disposed in one area of the front surface of the electronic device 100 . The camera may include a lens, an aperture, and an image pickup device.

In an embodiment, at least one physical key may be disposed on a side surface of the electronic device 100 . For example, the first function key 151 for turning on/off the display 110 or turning on/off the power of the electronic device 100 may be disposed on the right edge with respect to the front surface of the electronic device 100 . have. In an embodiment, the second function key 152 for controlling the volume or screen brightness of the electronic device 100 may be disposed on the left edge with respect to the front surface of the electronic device 100 . In addition to this, additional buttons or keys may be disposed on the front or rear of the electronic device 100 . For example, a physical button or a touch button mapped to a specific function may be disposed in a lower region of the front bezel 120 .

1B illustrates an electronic device 200 according to an embodiment. The electronic device 200 may include a display having a size smaller than that of the display 110 included in the electronic device 100 . The electronic device 200 may include a processor 210 , an acceleration sensor 220 , a gyro sensor 222 , a barometric pressure sensor 230 , a sound sensor 232 , and a memory 240 . In addition to the components included in the electronic device 100 , the electronic device 200 may include a touch sensor, an inertial sensor, a biometric sensor, a communication module, an interface, an output module, a battery, and a charging module. In the electronic device 200, at least one of the components (eg, the output module 240) may be omitted, or one or more other components (eg, a proximity sensor) may be added.

Hereinafter, various embodiments will be described with reference to the electronic device 100 illustrated in FIG. 1A for convenience of description. However, it is apparent to those skilled in the art that the technical ideas described below may also be applied to the electronic device 200 illustrated in FIG. 1B .

2 illustrates a hardware configuration of an electronic device according to an embodiment.

Referring to FIG. 2 , the electronic device 100 includes a display 110 , an acceleration sensor 220 , a gyro sensor 222 , a barometric pressure sensor 230 , a sound sensor 232 , a memory 240 , and a processor 210 . ) may be included. The components illustrated in FIG. 2 are exemplary, and the electronic device 100 may further include additional components.

In an embodiment, the processor 210 may acquire motion information of the electronic device 100 through at least one motion sensor. For example, the processor 210 may obtain motion information of the electronic device 100 using the acceleration sensor 220 and/or the gyro sensor 222 . The processor 210 may determine the motion of the electronic device 100 based on the obtained motion information. The movement of the electronic device 100 may be a concept including a posture such as an angle and a direction of the electronic device 100 . In an embodiment, the acceleration sensor 220 is a 3-axis sensor, and may acquire (x-axis, y-axis, z-axis) motion data to acquire acceleration information on the x-axis, y-axis, and z-axis. In an embodiment, the gyro sensor 222 may detect angular velocities in a first direction (eg, roll) and a second direction (eg, pitch) of the electronic device 100 .

In an embodiment, the processor 210 may acquire barometric pressure data through the barometric pressure sensor 230 . The processor 210 may check a change in atmospheric pressure in the ambient atmosphere of the electronic device 100 through the atmospheric pressure sensor 230 . In an embodiment, the processor 210 may detect a sound through the sound sensor 232 . The sound sensor 232 may be a concept including audio. The processor 210 may determine what kind of sound the sound sensed through the sound sensor 232 is. For example, the processor 210 may determine whether the sensed sound is a wind sound, a human voice, or a water sound.

In an embodiment, the electronic device 100 may include a memory 240 that stores data obtained from at least one sensor. The memory 240 may store sensor data obtained from the acceleration sensor 220 , the gyro sensor 222 , the barometric pressure sensor 230 , and the sound sensor 232 . The memory 240 stores at least one instruction, and when the at least one instruction is executed, the processor 210 may perform a function corresponding to the instruction.

In an embodiment, the processor 210 may display at least one user interface through the display 110 . When an instruction stored in the memory is executed, the processor 210 may output a function corresponding to the instruction through the display 110 .

3 is a flowchart illustrating a process of determining a user's hand washing situation and displaying a user interface related thereto in the electronic device 100 according to an exemplary embodiment.

In operation 310 according to an embodiment, the processor 210 may detect a first event occurring to the user of the electronic device 100 based on motion information acquired through at least one motion sensor. The processor 210 may detect a first event occurring to the user of the electronic device 100 based on motion information obtained through the acceleration sensor 220 and/or the gyro sensor 222 . The first event may mean a hand washing situation. Hereinafter, the first event may be referred to as a hand washing situation.

In operation 320 according to an embodiment, the processor 210 may determine whether the detection of the first event is misrecognized based on the atmospheric pressure data acquired through the atmospheric pressure sensor 230 . In other words, when the processor 210 detects the first event based on the motion information of the electronic device 100 , the processor 210 determines whether the detection of the first event based on the obtained barometric pressure data satisfies a specified condition. can For example, after the processor 210 first determines the hand washing situation through the movement of the electronic device 100 , the hand washing situation or the submerged situation is a situation in which the movement is similar to the hand washing situation. It can be judged secondarily whether or not it has been misrecognized.

In an embodiment, the processor 210 may detect a change in atmospheric pressure for a predetermined time through the atmospheric pressure sensor 230 . When the sensed change range of atmospheric pressure is the first change range, the processor 210 may determine that the detection of the first event is normally recognized. Conversely, when the sensed change range of atmospheric pressure is a second change range that is wider than the first change range, the processor 210 may determine that the detection of the first event is misrecognized. For example, when the atmospheric pressure P2P (peak to peak) value for about 10 seconds is about 0.02 hPa or less, the processor 210 may determine that the detection of the first event is normally recognized. Conversely, when the atmospheric pressure P2P value for about 10 seconds is about 0.06 hPa or more, the processor 210 may determine that the detection of the first event is misrecognized. However, the value of the atmospheric pressure P2P presented above is only an example, and is not limited thereto.

In an embodiment, the processor 210 is configured to determine the first event based on barometric pressure data obtained through the barometric pressure sensor 230 and height change data of the electronic device 100 obtained through wireless communication with an external device. Whether or not the detection is misrecognized can be determined. In an embodiment, the processor 210 may determine whether the misrecognition is based on height change data of the electronic device 100 acquired through wireless communication with an external device in a state in which the barometric pressure sensor 230 is turned off. . The processor 210 may perform wireless communication (eg, Bluetooth) with a wireless communication device (eg, a beacon). The processor 210 is based on a displacement (eg, a height value) of the electronic device 100 obtained through radio wave transmission and/or radio wave reception between the electronic device 100 and a wireless communication device (eg, a beacon). It can be determined whether the misrecognition is. For example, when the obtained displacement (eg, a height value) of the electronic device 100 is greater than or equal to a first distance (eg, about 20 cm or greater), the processor 210 performs a first event (eg, hand washing). situation) can be judged as misrecognition. Here, the misrecognition is a detection of a second event (eg, a showering situation) and/or a third event (eg, a submersion situation of the electronic device 100 ) rather than the detection of the first event (eg, hand washing situation). may include

In operation 330 according to an embodiment, when the detection of the first event is normal recognition, the processor 210 displays a user interface related to a score determined based on motion information obtained through the at least one motion sensor. can Details of this may be described with reference to FIGS. 10 and 11 below.

4 is a flowchart illustrating a process of determining a hand washing situation in the electronic device 100 according to an exemplary embodiment. In other words, the operations of FIG. 4 are described in detail by dividing the process of “detecting the first event” of operation 310 of FIG. 3 into at least two operations.

In operation 410 according to an embodiment, the processor 210 may acquire acceleration data through the acceleration sensor 220 . The processor 210 may obtain 3-axis acceleration data through the acceleration sensor 220 . For example, the processor 210 may obtain acceleration data for the x-axis, y-axis, and z-axis of the electronic device 100 .

In an embodiment, the processor 210 coordinates the obtained acceleration data from a body frame coordinate system (X-axis, Y-axis, Z-axis) to a navigation frame coordinate system (EN-axis, U-axis) can be converted The processor 210 may obtain an acceleration component perpendicular to the ground and an acceleration component horizontal to the ground based on the acceleration data converted into the navigation frame coordinate system. Hereinafter, an acceleration component perpendicular to the ground may be referred to as an acceleration vertical component, and an acceleration component horizontal to the ground may be referred to as an acceleration horizontal component.

In operation 420 according to an embodiment, the processor 210 may calculate a magnitude of the 3-axis acceleration based on the acquired acceleration data. The processor 210 may calculate magnitudes for the x-axis, y-axis, and z-axis acceleration through the following equation.

In operation 430 according to an embodiment, the processor 210 may calculate a sliding window summing (SWS) difference value based on the calculated magnitude of the acceleration. The processor 210 may calculate the SWS difference value through the following equation.

In operation 440 according to an embodiment, the processor 210 may detect a zero-crossing (ZC) point. The processor 210 may detect a ZC point for the calculated SWS difference signal. The ZC point may mean a point at which a sign of a function changes. The ZC point can be regarded as one periodic hand motion. A more detailed description is described with reference to FIG. 5 .

In operation 450 according to an embodiment, the processor 210 may check the number of ZC points for the first time. For example, the processor 210 may check the number of ZC points for 5 seconds. In an embodiment, the processor 210 may check the number of ZC points within the range of the first number of samples for data acquired through the acceleration sensor 220 . For example, the processor 210 may check the number of ZC points in 500 samples.

In operation 460 according to an embodiment, when the number of ZC points is greater than the first threshold number, the processor 210 may determine that the first event has occurred to the user. When the number of ZC points is greater than the first threshold number, the processor 210 may determine that there is a periodic movement of the electronic device 100 . For example, when the number of ZC points is greater than 10 for 5 seconds, the processor 210 may determine that there is a periodic movement of the electronic device 100 . When the periodic movement is confirmed, the processor 210 may determine that the first event has occurred to the user.

In an embodiment, the processor 210 may determine that the first event has occurred to the user based on the vertical acceleration component and the acceleration horizontal component in addition to considering the number of ZC points. In an embodiment, the processor 210 may determine that the first event has occurred to the user based on the vertical acceleration component and the acceleration horizontal component without calculating the number of ZC points. For example, when the processor 210 detects a value for a vertical component of acceleration and a value for a horizontal component of acceleration as in the graph of FIG. 8 below through the acceleration sensor 220, it is determined that the first event has occurred to the user. can judge

In an embodiment, the processor 210 may determine that the first event has occurred to the user based on the sound acquired through the sound sensor 232 in addition to considering the number of ZC points. The processor 210 may recognize whether the acquired sound is the sound of water, and may determine that the first event has occurred to the user based on the recognized sound of water. In an embodiment, the processor 210 may determine that the first event has occurred to the user based on the recognized sound of water without motion data acquired from at least one motion sensor. In an embodiment, the processor 210 may recognize the sound of water by always turning on the sound sensor 232 or turning it on when a movement of the electronic device 100 occurs.

The first event may mean a hand washing situation. Hereinafter, the first event may be referred to as a hand washing situation.

5 is a diagram illustrating acceleration data when a first event, which is a hand washing situation, occurs in the electronic device 100 according to an exemplary embodiment.

Referring to FIG. 5 , a graph 501 is a graph of the x-axis acceleration, the y-axis acceleration, and the z-axis acceleration in a hand washing situation, and the graph 502 is the x-axis acceleration, the y-axis acceleration, and the z-axis It is a graph regarding the sliding window summing (SWS) and zero-crossing (ZC) points calculated based on the acceleration. For example, the x-axis may include a front direction of the electronic device 100 , the y-axis may include a side direction of the electronic device 100 , and the z-axis may be perpendicular to the x-axis and the y-axis can The unit of the horizontal axis of the graph 501 and the graph 502 is the number of samples, and the unit of the vertical axis is m/s² as the unit of acceleration. The number of 100 samples may correspond to 1 second.

In an embodiment, referring to the graph 502 , a graph for SWS difference values periodically having positive and negative values based on 0 is displayed. A zero-crossing (ZC) point is a point at which a sign of a function with respect to an SWS difference value changes, and may mean a point at which a positive value changes to a negative value. For example, when the movement of the electronic device 100 periodically changes, as shown in the graph 502 , the SWS difference value may change while repeating (+) and (-) based on 0. Also, when the movement of the electronic device 100 changes periodically, the ZC point may also occur repeatedly. The processor 210 may determine that the first event has occurred when the ZC point occurs more than a predetermined number for a predetermined time.

6 is a flowchart illustrating a process of determining whether detection of a first event is normally recognized in the electronic device 100 according to an exemplary embodiment. Hereinafter, each operation in the embodiment of FIG. 6 may be sequentially performed, but is not necessarily performed sequentially. The order of each operation of FIG. 6 may be changed, and at least two operations may be performed in parallel. For example, the order of operations 620 and 630 may be reversed, and operations 620 and 630 may be performed in parallel.

In operation 610 according to an embodiment, the processor 210 may acquire sensor data through at least one sensor. The processor 210 may acquire motion data (eg, acceleration data) through a motion sensor (eg, the acceleration sensor 220 and the gyro sensor 222 ). The processor 210 may acquire barometric pressure data through the barometric pressure sensor 230 . The processor 210 may acquire sound data through the sound sensor 232 . The processor 210 may recognize the sound of water based on the acquired sound data.

In operation 620 according to an embodiment, the processor 210 may calculate a pressure P2P (peak to peak) value for the second time period. The processor 210 may calculate a P2P (peak to peak) value for the second time to distinguish a hand washing situation from a showering situation through the barometric pressure sensor 230 . Hereinafter, referring to FIG. 7 , since the difference in the P2P value significantly occurs in a hand washing situation and other situations, the processor 210 may use the P2P value to distinguish a hand washing situation from a showering situation.

In operation 630 according to an embodiment, the processor 210 may detect a characteristic of the acceleration data obtained through the acceleration sensor 220 . The processor 210 may detect a characteristic of a horizontal component of acceleration and a characteristic of a vertical component of acceleration in order to distinguish a hand washing situation and a showering situation through the acceleration sensor 220 . The processor 210 coordinates the data acquired through the acceleration sensor 220 from the body frame coordinate system (X-axis, Y-axis, Z-axis) to the navigation frame coordinate system (EN-axis, U-axis). can be converted The processor 210 may extract features of the horizontal component of the acceleration and the vertical component of the acceleration based on the acceleration data coordinate-converted into the navigation frame coordinate system. The characteristic of the horizontal component of the acceleration is a value obtained by dividing the integral value of the (+) signal with respect to the EN axis for a predetermined time by the integral value of the (-) signal. The vertical component of the acceleration is characterized by dividing the integral value of the (+) signal with respect to the U-axis for a predetermined time by the integral value of the (-) signal. The EN axis may mean an east-west, north-south (EWSN) axis in a direction horizontal to the ground, and the U-axis may mean an up-down axis in a direction perpendicular to the ground.

In operation 640 according to an embodiment, the processor 210 may determine whether the atmospheric pressure P2P value satisfies the first condition. The processor 210 may determine that the first condition is satisfied when the atmospheric pressure P2P value is less than the first threshold value or greater than the second threshold value. The second threshold value may be greater than the first threshold value. For example, a case in which the P2P value is smaller than the first threshold value may correspond to a case in which the change in atmospheric pressure is small. The processor 210 may determine that when there is little change in atmospheric pressure, it is a case of hand washing. As another example, a case in which the P2P value is greater than the second threshold value may correspond to a case in which a change in atmospheric pressure is large. A case in which the atmospheric pressure change is large may correspond to a case in which water is submerged in a vent hole of the atmospheric pressure sensor 230 . Since water submerged in the vent hole of the barometric pressure sensor 230 may occur not only when the electronic device 100 is submerged, but also occurs during hand washing, the processor 210 determines that the P2P value is greater than the second threshold value. In this case, operation 650 may be performed. The processor 210 may perform operation 650 if the atmospheric pressure P2P value satisfies the first condition, and may perform operation 670 if not satisfied.

In operation 650 according to an embodiment, the processor 210 may determine whether the characteristic of the detected acceleration data satisfies the second condition. The processor 210 may determine whether a horizontal component of the detected acceleration and a vertical component of the acceleration satisfy the second condition. The processor 210 may determine that the second condition is satisfied when the values of the horizontal and vertical components of the acceleration regularly change with a predetermined period for a predetermined time. For example, referring to FIG. 8 below, the value of the acceleration horizontal component periodically changes from about -5 m/s² to about +5 m/s², and the value of the acceleration vertical component varies from about 0 m/s² to about -4 m/s² When it periodically changes between , it may be determined that the second condition is satisfied. The processor 210 may perform operation 660 if the characteristic of the detected acceleration data satisfies the second condition, and may perform operation 670 if not satisfied.

In operation 660 according to an embodiment, the processor 210 may determine that the detection of the first event is normal recognition. In other words, the processor 210 may determine that the sense of hand washing is normally recognized.

In operation 670 according to an embodiment, the processor 210 may determine that the detection of the first event is not normal recognition. In other words, the processor 210 may determine that the sense of hand washing is not normal recognition.

7 is a diagram illustrating barometric pressure data obtained by the electronic device 100 through the barometric pressure sensor 230 according to an exemplary embodiment.

Referring to FIG. 7 , a graph 701 is a graph for air pressure data in a hand washing situation, a graph 702 is a graph for atmospheric pressure data in a showering situation, and a graph 703 is an electronic device 100 is a graph of the atmospheric pressure data in a flooded situation. The unit of the horizontal axis of the graph 701 , the graph 702 , and the graph 703 is the number of samples, and the unit of the vertical axis is hectopascals (hPa). The number of 100 samples may correspond to 1 second. In an embodiment, the processor 210 may calculate the P2P value in a sliding window (SW) method in units of the first number of samples (eg, units of 1000 units).

In one embodiment, referring to graphs 701 to 703, the atmospheric pressure P2P value in the hand washing situation is about 0.0177 hPa, the atmospheric pressure P2P value in the showering situation is about 0.1017 hPa, and in a submerged situation The barometric P2P value of is about 1.5487 hPa.

Since the difference in the barometric pressure P2P value in each situation is significantly different, the processor 210 determines the first event (eg, the first event in FIG. 3 ) based on the barometric pressure P2P value obtained through the barometric pressure sensor 230 . It can be determined whether the detection is normal recognition or not.

8 is a diagram illustrating barometric pressure data obtained through the barometric pressure sensor 230 in a situation in which the electronic device 100 washes hands according to an exemplary embodiment.

Referring to FIG. 8 , a graph 801 is a graph of a horizontal component of acceleration in a hand washing situation, and a graph 802 is a graph of a vertical component of acceleration in a hand washing situation. The unit of the horizontal axis of the graph 801 and the graph 802 is the number of samples, and the unit of the vertical axis is m/s² as the unit of acceleration. The number of 100 samples may correspond to 1 second.

In an embodiment, referring to the graph 801 , it can be seen that the value of the horizontal acceleration component in the hand washing situation periodically changes between about -5 m/s² and about +5 m/s². Referring to the graph 802, it can be seen that the value of the vertical acceleration component in the hand washing situation periodically changes between about 0 m/s² and about -4 m/s². In other words, in the hand washing situation, it can be seen that the horizontal movement of the electronic device 100 is greater than the vertical movement.

9 is a diagram illustrating barometric pressure data acquired through the barometric pressure sensor 230 in a situation in which the electronic device 100 takes a shower, according to an exemplary embodiment.

Referring to FIG. 9 , a graph 901 is a graph of a horizontal component of acceleration in a showering situation, and a graph 902 is a graph of a vertical component of acceleration in a showering situation. The unit of the horizontal axis of the graph 901 and the graph 902 is the number of samples, and the unit of the vertical axis is m/s² as the unit of acceleration. The number of 100 samples may correspond to 1 second.

In an embodiment, referring to the graph 901 , it can be seen that the horizontal acceleration component in the showering situation varies from about -10 m/s² to about +10 m/s². Referring to the graph 902 , it can be seen that the vertical component of acceleration in the showering situation varies from about +5 m/s² to about -10 m/s². In other words, in a showering situation, it can be seen that the vertical movement and the horizontal movement of the electronic device 100 are at the same level. In addition, it can be seen that the vertical movement in the showering situation is more than the vertical movement in the hand washing situation (see FIG. 8 ).

Since the values of the acceleration data in each situation are significantly different, the processor 210 determines that the detection of the first event (eg, the first event of FIG. 3 ) is normal based on the acceleration data acquired through the acceleration sensor 220 . It can be determined whether it is recognized or not.

10 is a flowchart illustrating a process of calculating a score for hand washing in the electronic device 100 according to an exemplary embodiment. The operations of FIG. 10 may be performed after operation 660 of FIG. 6 . In other words, the processor 210 may detect a first event (eg, hand washing) and, when it is determined that the detection of the first event is normal recognition, may determine a hand washing area. In the following embodiment, each operation may be performed sequentially, but is not necessarily performed sequentially. The order of each operation may be changed, and at least two operations may be performed in parallel. For example, providing a user interface related to the hand washing guide in operation 1060 may be performed even before the first event ends.

In operation 1010 according to an embodiment, the processor 210 may determine a hand washing area based on the movement of the electronic device 100 . The processor 210 may determine a change in the posture of the electronic device 100 based on at least one motion sensor (eg, the acceleration sensor 220 of FIG. 2 and/or the gyro sensor 222 of FIG. 2 ). The processor 210 may estimate the hand washing part of the user of the electronic device 100 based on the change in the posture of the electronic device 100 . Details of this may be described with reference to FIG. 11 below.

In operation 1020 according to an embodiment, the processor 210 may set as one session from the time the first event (eg, hand washing) is detected to the time the first event ends. In other words, the processor 210 may set as one session for hand washing from the point in time when it is determined that hand washing is finished to the time when it is determined that hand washing is finished. For example, the processor 210 may be configured to perform a motion and/or a motion in which the user shakes his/her hand through motion data (eg, acceleration data) acquired through at least one motion sensor (eg, the acceleration sensor 220 of FIG. 2 ). Alternatively, if the sound of water is not detected through the sound sensor 232 for a certain period of time or more or it is determined that the user is walking, it may be determined that the hand washing has been completed. As another example, the processor 210 performs a motion in which the user rubs his or her hand through at least one motion sensor (eg, the acceleration sensor 220 of FIG. 2 ) without detecting the sound of water through the sound sensor 232 for a certain period of time or longer. If it does not detect (eg, soaping motion), it can be judged that hand washing has been completed.

In operation 1030 according to an embodiment, the processor 210 may calculate a hand washing time within the set session. The processor 210 may calculate the total time to wash hands within the set session. The processor 210 may calculate an accumulated time for each hand washing area within the session. The total hand washing time may be calculated as a time excluding a motion in which a motion of the electronic device 100 is not detected or is not determined as a hand washing motion. For example, the total hand washing time may be calculated as a time excluding a motion in which the user of the electronic device 100 does not move his hand and is still within the set session.

In operation 1040 according to an embodiment, the processor 210 may calculate an accumulated time for each part within the set session. The processor 210 accumulates for each first hand motion (eg, first hand motion 1201 in FIG. 12 ) to fifth hand motion (eg, fifth hand motion 1205 in FIG. 12 ) corresponding to the hand washing part. time can be calculated. The processor 210 may set a motion not corresponding to the first hand motion to the fifth hand motion as a sixth motion, and calculate an accumulated time for the sixth motion. The processor 210 may calculate a start time and an end time for each hand motion based on a roll value and a pitch value of the electronic device 100 obtained through the gyro sensor 222 . The processor 210 may calculate the accumulated time for each part based on the start time and the end time calculated for each hand motion.

In operation 1050 according to an embodiment, the processor 210 may calculate a score for hand washing. The processor 210 may calculate a score for hand washing through the following formula based on the total time for hand washing in the set session and the accumulated time for each part. If the total time of hand washing exceeds the recommended time, the score may increase, and the score may increase as the cumulative time for each part is performed evenly within the session.

In an embodiment, the processor 210 may apply a weight to each hand washing area. The processor 210 may calculate a score based on the weight. The processor 210 may prioritize each hand washing part to apply a first weight to a part having the highest priority, and apply a second weight lower than the first weight to a part having the lowest priority. For example, the processor 210 may apply a first weight to a portion that is generally not washed well, and may apply a second weight that is lower than the first weight to a portion that is easy to wash.

In operation 1060 according to an embodiment, the processor 210 may provide a user interface related to the hand washing guide through the display 110 based on the calculated score.

In an embodiment, the processor 210 may provide a hand washing guide through the display 110 based on a score calculated by synthesizing hand washing during the first period. For example, the processor 210 may calculate a score by accumulating hand washing activities during the day, and provide a hand washing guide corresponding to the score. In an embodiment, a score may be calculated based on the first number of hand washing actions, and a hand washing guide may be provided through the display 110 based on the score. For example, a score for one hand washing may be calculated, and a hand washing guide corresponding to the score may be provided.

In an embodiment, the processor 210 may display the hand washing guide through the display 110 and provide the hand washing guide through voice. In an embodiment, the processor 210 may provide the hand washing guide through a voice while the display 110 is off. For example, the processor 210 may say "The handwashing score is 95." or "Why don't you wash your thumb additionally?" Hand washing instructions can be provided through the same voice guidance.

In an embodiment, the processor 210 may provide the first feedback through the display 110 . The first feedback may be reactive feedback. The processor 210 may determine whether a hand washing guide is satisfied at every moment while the user is washing his/her hands, and may immediately provide the guide to the user. For example, the processor 210 may determine whether the time for each hand washing area is complied with or whether various areas are washed, and may provide feedback on the non-observed area.

In an embodiment, the processor 210 may provide the second feedback through the display 110 . The second feedback may be proactive feedback. The processor 210 may provide the guide to the user by utilizing the learned AI (eg, on-device artificial intelligence) based on various situations. For example, the processor 210 may be configured to perform various functions such as the time the user wears the electronic device 100 when washing hands, the frequency of hand washing, the time it takes to wash one hand, the frequency for each hand washing operation, and hand washing information for each situation. Data can be collected and stored. The processor 210 may determine the frequency of providing the guide based on the collected and stored data. For example, the processor 210 may give different weights to indoor hand washing situations and outdoor hand washing situations. In an embodiment, the processor 210 may provide a feedback-related message in a situation in which feedback is required based on the data, and may not provide a feedback-related message in a situation in which feedback is unnecessary. For example, the processor 210 may provide a guide to the user to wash their hands in a specific time period, in a specific space, or in a specific situation (eg, immediately after going out). For another example, when the user's spatial movement is small or the meeting with another person is not detected, the processor 210 may decrease the frequency of providing the hand washing guide or may not provide the hand washing guide.

Examples of the user interface may be referred to through FIGS. 13 to 16 below.

11 is a flowchart illustrating a process of determining a hand washing area in the electronic device 100 according to an exemplary embodiment. In other words, the operations of FIG. 11 are described in detail by dividing the process of "determining the hand washing area based on the movement of the electronic device 100" of operation 1010 of FIG. 10 into at least two operations.

In operation 1110 according to an embodiment, the processor 210 obtains motion data through at least one motion sensor (eg, the acceleration sensor 220 and/or the gyro sensor 222 ) to move the electronic device 100 . can be calculated. The processor 210 configures the first direction (eg, roll direction) and the second direction (eg, pitch) of the electronic device 100 based on data acquired through the acceleration sensor 220 and/or the gyro sensor 222 . direction) can be detected.

In operation 1120 according to an embodiment, the processor 210 may calculate an average roll value and an average pitch value within a predetermined section. For example, the processor 210 may determine an average roll value and an average pitch for a predetermined time (eg, about 5 seconds to about 10 seconds) based on angle information obtained through the gyro sensor 222 . value can be calculated.

In operation 1130 according to an embodiment, the processor 210 may calculate a variance of a roll value and a variance of a pitch value within a predetermined section. For example, the processor 210 distributes the roll value for a predetermined time (eg, about 5 seconds to about 10 seconds) based on the calculated average roll value and the average pitch value. , and the variance of the values of pitch can be calculated.

In operation 1140 according to an embodiment, the processor 210 may determine a hand washing area based on the classification model. The processor 210 may classify and recognize the hand washing area through a classification model such as a decision tree and SVM. The classification model may be designed using various machine learning techniques such as DTs, SVM, KNN, and MLP. The classification model is designed based on the extracted feature information, and the processor 210 may recognize a hand motion based on the extracted feature information. For example, the processor 210 may transmit a roll value, a pitch ( pitch) value, the hand motion corresponding to each roll value and the pitch value may be distinguished and recognized. <Table 1> below is a table showing the characteristics of roll values and pitch values corresponding to each hand motion.

Roll average Pitch average Roll dispersion Pitch dispersion first hand motion -109.2749 0.9589 7.9843 8.3380 second hand motion -35.0936 -12.8989 31.4015 3.1903 3rd hand motion -43.4610 -0.4549 954.9839 114.2436 4th hand motion -16.2991 -14.5129 237.8623 55.8409 5th hand motion -42.7526 -1.1615 4.7321 2.1770

In an embodiment, the processor 210 performs a first hand motion (eg, the first hand motion 1201 of FIG. 12 ) and a second hand motion (eg, a second hand motion 1202 of FIG. 12 ) by moving the hand washing area. ), third hand motion (eg, third hand motion 1203 in FIG. 12 ), fourth hand motion (eg, fourth hand motion 1204 in FIG. 12 ), fifth hand motion (eg, in FIG. 12 ) The fifth hand motion 1205) can be distinguished and detected. The processor 210 may distinguish a motion that does not correspond to the first hand motion to the fifth hand motion by setting it as a sixth motion. The sixth motion may be an unspecified motion. <Table 2> below is a table showing the washing area corresponding to the hand motion.

first motion second motion 3rd motion 4th motion 5th motion 6th motion no movement Palm O X X O X O X back of the hand X O X X X O X between the fingers O O X X X O X a thumb, a big finger X X O X X X X fingertips X X X O O X X

12 is a diagram illustrating data on a roll value and a pitch value obtained through the acceleration sensor 220 and/or the gyro sensor 222 in a situation in which the electronic device 100 washes hands according to an embodiment. In one embodiment, when the first hand motion 1201 is performed, the processor 210 is configured to use the accelerometer 220 and/or the gyro sensor 222 from about -120° to about -100°. A roll value having a value and a pitch value having a value of about -10˚ to about +10˚ may be obtained. In an embodiment, when the second hand motion 1202 is performed, the processor 210 is an acceleration sensor A roll value having a value of about -40˚ to about -20˚ and a pitch value having a value of about -20˚ to about 0˚ may be acquired through 220 and/or the gyro sensor 222 .

In one embodiment, when the third hand motion 1203 is performed, the processor 210 via the acceleration sensor 220 and/or the gyro sensor 222 a roll value having a value of about -100° to about 0° and a pitch value with a value of about -20˚ to about +20˚ can be obtained.

In one embodiment, when the fourth hand motion 1204 is performed, the processor 210 rolls through the acceleration sensor 220 and/or the gyro sensor 222 with a value of about -60° to about +10°. It is possible to obtain a value and a pitch value having a value of about -30˚ to about 0˚.

In one embodiment, when the fifth hand motion 1205 is performed, the processor 210 performs a roll having a value of about -50° to about -40° through the acceleration sensor 220 and/or the gyro sensor 222 . It is possible to obtain a value and a pitch value having a value of about -10˚ to about 0˚.

13 is a diagram of an interface showing a portion to be additionally washed in the electronic device 100 according to an exemplary embodiment.

In an embodiment, the electronic device 100 may display, through the display 110 , the washing-completed part during hand washing. The electronic device 100 may display the completed portion through the display 110 whenever there is a hand-washed portion before the hand-washing situation ends. When washing of the first part is completed during hand washing, the electronic device 100 may highlight and display the first part. When washing of the first part and the second part is completed during hand washing, the electronic device 100 may highlight and display the first part and the second part. For example, when it is determined that hand washing has been completed for the thumb and palm regions, the electronic device 100 may display the phrase “clear” on the corresponding parts or use a different color to emphasize the part where hand washing has been completed. In other words, the electronic device 100 may induce the user to wash a part that has not yet been washed by displaying the part where hand washing has been completed through the display 110 .

In an embodiment, when the hand washing situation is over, the electronic device 100 may analyze a part where washing is completed. The electronic device 100 may determine a portion to be washed additionally by analyzing a portion that has been washed. When there is a portion to be additionally washed, the electronic device 100 may highlight the portion and display it on the display 110 . For example, if the electronic device 100 does not detect a thumb washing motion (eg, the third hand motion 1203 of FIG. 12 ) during the hand washing session, a circle is marked on the thumb part or a different color is displayed. to provide hand-washing guidelines. In an embodiment, the electronic device 100 may provide a vibration notification and/or a sound notification while providing the guide.

14 is a diagram for displaying a score for hand washing and providing a notification in the electronic device 100 according to an exemplary embodiment.

In an embodiment, when the hand washing situation is over, the processor 210 may analyze the total hand washing time and washing time for each part. The processor 210 may calculate a score for hand washing by analyzing the total hand washing time and washing time for each part. The processor 210 may display the calculated score on one area of the display 110 .

In an embodiment, the processor 210 may share the calculated score with an external electronic device registered in the electronic device 100 . For example, the processor 210 may provide the calculated score and/or data related to the score to an external electronic device of a family member and/or acquaintance. The processor 210 may transmit the score and/or the data to a server. The processor 210 may receive feedback on hand washing from an external electronic device of a family member and/or acquaintance, and may induce the user to properly wash hands through this. For example, the processor 210 may transmit data on the calculated score to an external electronic device registered as a guardian of the user of the electronic device 100 through the server. The external electronic device registered as the guardian may monitor the electronic device 100 based on the received data.

15 is a diagram for providing information on hand washing activity and a notification in the electronic device 100 according to an exemplary embodiment.

Referring to FIG. 15 , according to an embodiment, the electronic device 100 may provide an interface that displays the number of times to be washed during the day and the number of times to wash until now. For example, the number of times a person needs to wash in a day may be set to a number set by the WHO (eg 8 times). However, since this is an example, the present invention is not limited thereto.

Referring to FIG. 15 , in an embodiment, the electronic device 100 may provide an interface informing the user of a case in which hand washing has not been performed for a certain period of time in order to induce the user to periodically wash his/her hands. For example, when the electronic device 100 does not detect a user's hand washing for about 3 hours, a phrase recommending hand washing (eg, "3 hours have passed since you have not washed your hands. How about washing your hands?" ?") can be provided. In an embodiment, the electronic device 100 may provide a vibration notification and/or a sound notification at the same time as providing the interface.

In an embodiment, the electronic device 100 may provide a notification related to the interface through an external device operatively connected to the electronic device 100 . For example, the electronic device 100 may provide the notification through an external device connected through short-range communication (eg, Wifi or Bluetooth) in an Internet of things (IoT) environment. The electronic device 100 may provide the notification through an external device connected to the same account in the server.

16 is a diagram of an interface for changing a score calculation criterion for hand washing in the electronic device 100 according to an embodiment.

In an embodiment, the electronic device 100 may change the standard for hand washing and/or the score calculation standard for hand washing based on a public message (eg, a public notification text) provided by a public institution. have. In other words, the electronic device 100 may change the standard for hand washing and/or the score calculation standard for hand washing by using a service linked with a public institution. Criteria for handwashing and/or scoring for handwashing may vary depending on the time, location, and situation of service for the handwashing recognition function. For example, when the WHO recommendation is changed or the level of social distancing for each country is changed, the electronic device 100 may change the standard for hand washing and/or the score calculation standard. In other words, the electronic device 100 may adjust the threshold for factors such as hand washing motion, minimum hand washing time, minimum hand washing area, and hand washing frequency, and change and apply the score calculation criteria for hand washing. have.

Referring to FIG. 16 , the electronic device 100 may provide an interface related to a public message (eg, a public notification text) provided by a public institution. The electronic device 100 may receive the message provided by a certified office through a wireless network. The electronic device 100 may display an interface related to the message on the display 110 based on the received message. For example, the electronic device 100 performs a first step (eg, social distancing step 1), a second step (eg, social distancing step 2), and a third step (eg, social distancing) Step 3) may be provided. In an embodiment, the electronic device 100 may highlight the level of environmental pollution applied to the handwashing standard and display it on the display 110 . The electronic device 100 may set a handwashing score calculation criterion based on the applied environmental pollution level. For example, as the step increases, the electronic device 100 may increase the number of times to wash during the day, and may change the score calculation criterion based on the increased number of times.

17 is a block diagram of an electronic device 1701 in a network environment 1700 according to various embodiments of the present disclosure. Referring to FIG. 17 , in a network environment 1700 , the electronic device 1701 communicates with the electronic device 1702 through a first network 1798 (eg, a short-range wireless communication network) or a second network 1799 . It may communicate with the electronic device 1704 or the server 1708 through (eg, a remote wireless communication network). According to an embodiment, the electronic device 1701 may communicate with the electronic device 1704 through the server 1708 . According to an embodiment, the electronic device 1701 includes a processor 1720 , a memory 1730 , an input module 1750 , a sound output module 1755 , a display module 1760 , an audio module 1770 , and a sensor module ( 1776), interface 1777, connection terminal 1778, haptic module 1779, camera module 1780, power management module 1788, battery 1789, communication module 1790, subscriber identification module 1796 , or an antenna module 1797 . In some embodiments, at least one of these components (eg, the connection terminal 1778 ) may be omitted or one or more other components may be added to the electronic device 1701 . In some embodiments, some of these components (eg, sensor module 1776, camera module 1780, or antenna module 1797) are integrated into one component (eg, display module 1760). can be

The processor 1720, for example, executes software (eg, a program 1740) to execute at least one other component (eg, hardware or software component) of the electronic device 1701 connected to the processor 1720. It can control and perform various data processing or operations. According to an embodiment, as at least part of data processing or operation, the processor 1720 may store a command or data received from another component (eg, the sensor module 1776 or the communication module 1790 ) into the volatile memory 1732 . may be stored in , process commands or data stored in the volatile memory 1732 , and store the result data in the non-volatile memory 1734 . According to an embodiment, the processor 1720 is a main processor 1721 (eg, a central processing unit or an application processor) or a secondary processor 1723 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 1701 includes a main processor 1721 and a sub-processor 1723, the sub-processor 1723 uses less power than the main processor 1721 or is set to be specialized for a specified function. can The co-processor 1723 may be implemented separately from or as part of the main processor 1721 .

The coprocessor 1723 may, for example, act on behalf of the main processor 1721 while the main processor 1721 is in an inactive (eg, sleep) state, or when the main processor 1721 is active (eg, executing an application). ), together with the main processor 1721, at least one of the components of the electronic device 1701 (eg, the display module 1760, the sensor module 1776, or the communication module 1790) It is possible to control at least some of the related functions or states. According to an embodiment, the coprocessor 1723 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 1780 or the communication module 1790). have. According to an embodiment, the auxiliary processor 1723 (eg, a neural network processing device) may include a hardware structure specialized in processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 1701 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 1708). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 1730 may store various data used by at least one component of the electronic device 1701 (eg, the processor 1720 or the sensor module 1776 ). The data may include, for example, input data or output data for software (eg, a program 1740) and instructions related thereto. The memory 1730 may include a volatile memory 1732 or a non-volatile memory 1734 .

The program 1740 may be stored as software in the memory 1730 , and may include, for example, an operating system 1742 , middleware 1744 , or an application 1746 .

The input module 1750 may receive a command or data to be used by a component (eg, the processor 1720 ) of the electronic device 1701 from the outside (eg, a user) of the electronic device 1701 . The input module 1750 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 1755 may output a sound signal to the outside of the electronic device 1701 . The sound output module 1755 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to an embodiment, the receiver may be implemented separately from or as a part of the speaker.

The display module 1760 may visually provide information to the outside (eg, a user) of the electronic device 1701 . The display module 1760 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the corresponding device. According to an embodiment, the display module 1760 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 1770 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 1770 obtains a sound through the input module 1750 or an external electronic device (eg: A sound may be output through the electronic device 1702 (eg, a speaker or headphones).

The sensor module 1776 detects an operating state (eg, power or temperature) of the electronic device 1701 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state can do. According to an embodiment, the sensor module 1776 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 1777 may support one or more specified protocols that may be used for the electronic device 1701 to connect directly or wirelessly with an external electronic device (eg, the electronic device 1702 ). According to an embodiment, the interface 1777 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 1778 may include a connector through which the electronic device 1701 can be physically connected to an external electronic device (eg, the electronic device 1702 ). According to an embodiment, the connection terminal 1778 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 1779 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 1779 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 1780 may capture still images and moving images. According to an embodiment, the camera module 1780 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 1788 may manage power supplied to the electronic device 1701 . According to an embodiment, the power management module 1788 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 1789 may supply power to at least one component of the electronic device 1701 . According to an embodiment, the battery 1789 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 1790 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 1701 and an external electronic device (eg, the electronic device 1702, the electronic device 1704, or the server 1708). It can support establishment and communication performance through the established communication channel. The communication module 1790 may include one or more communication processors that operate independently of the processor 1720 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to an embodiment, the communication module 1790 may include a wireless communication module 1792 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 1794 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 1798 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 1799 (eg, legacy). It may communicate with the external electronic device 1704 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunication network such as a computer network (eg, LAN or WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 1792 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 1796 within a communication network, such as the first network 1798 or the second network 1799 . The electronic device 1701 may be identified or authenticated.

The wireless communication module 1792 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 1792 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 1792 uses various technologies for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 1792 may support various requirements specified in the electronic device 1701 , an external electronic device (eg, the electronic device 1704 ), or a network system (eg, the second network 1799 ). According to an embodiment, the wireless communication module 1792 provides a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

The antenna module 1797 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 1797 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 1797 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 1798 or the second network 1799 is connected from the plurality of antennas by, for example, the communication module 1790 . can be selected. A signal or power may be transmitted or received between the communication module 1790 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 1797 .

According to various embodiments, the antenna module 1797 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a specified high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 1701 and the external electronic device 1704 through the server 1708 connected to the second network 1799 . Each of the external electronic devices 1702 and 1704 may be the same as or different from the electronic device 1701 . According to an embodiment, all or part of the operations performed by the electronic device 1701 may be executed by one or more external electronic devices 1702 , 1704 , or 1708 . For example, when the electronic device 1701 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 1701 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 1701 . The electronic device 1701 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 1701 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. According to another embodiment, the external electronic device 1704 may include an Internet of things (IoT) device. Server 1708 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 1704 or the server 1708 may be included in the second network 1799 . The electronic device 1701 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

In an embodiment, the electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) includes at least one motion sensor 220 , 222 , a barometric pressure sensor 230 , a display ( 110 ), and at least one processor 210 . The processor 210 detects a first event occurring to the user of the electronic device 100 based on the motion information acquired through the at least one motion sensor, and adds Based on the determination, whether the detection of the first event is misrecognized may be determined. When the detection of the first event is normal recognition, the processor 210 may display a user interface related to a score determined based on motion information obtained through the at least one motion sensor on the display 110 .

In an embodiment, the first event generated to the user may be a hand washing event. The at least one motion sensor may include an acceleration sensor 220 . The processor 210 calculates an acceleration magnitude of the electronic device 100 based on the acceleration data obtained through the acceleration sensor 220 , and periodically moves the electronic devices 100 and 200 based on the acceleration magnitude. can be detected. The processor 210 may detect a first event occurring to the user based on the periodic movement.

In an embodiment, the processor 210 detects a change in air pressure for a first time through the barometric pressure sensor 230 , and when the detected change range in air pressure is the first change range, detecting the first event It can be judged that this is normal recognition. The processor 210 may determine that the detection of the first event is a misrecognition when the sensed change range of the atmospheric pressure is a second change range that is larger than the first change range.

In an embodiment, the processor 210 is configured to, when the sensed change range of atmospheric pressure is a third change range that is larger than the second change range, a horizontal acceleration component obtained through the at least one motion sensor, and an acceleration vertical Based on the component, it may be determined whether the detection of the first event is misrecognition.

In an embodiment, the one or more motion sensors may include an acceleration sensor 220 and/or a gyro sensor 222 . The processor 210 may obtain a direction value for a roll direction and a pitch direction of the electronic device 100 and/or 200 through the acceleration sensor 220 and/or the gyro sensor 222 . can The processor 210 may recognize at least one motion corresponding to the obtained direction value based on the obtained direction value. The processor 210 may determine the hand washing area based on the recognized motion information. The at least one motion may be a motion predetermined according to the direction value.

In an embodiment, the processor 210 may set the period from the time when the occurrence of the first event is detected to the time when the first event is terminated as at least one session. The processor 210 may calculate a total hand washing time and an accumulated time for each hand washing part based on the movement of the electronic device 100 within the set at least one session.

In an embodiment, the processor 210 may calculate a high score when the total accumulated time of washing hands exceeds the recommended time. The processor 210 may calculate a higher score as the cumulative time for each hand washing part is uniform within the at least one session.

In an embodiment, the total time for washing hands may be a time excluding a time during which no movement of the electronic device 100 is detected within the session.

In an embodiment, the user interface may include at least one of a user interface for guiding the determined score, a user interface for guiding the score and a guide corresponding to the score, and a user interface for guiding an unwashed area. . The processor 210 may provide at least one of a vibration notification and a sound notification in response to the display of the user interface.

In one embodiment, the processor 210 obtains information on the external environment from the external server, and based on the obtained information on the external environment, when the external environment is the first environment, calculates the score This can be done in the first way. When the external environment is a second environment different from the first environment, the processor 210 may calculate the score using a second method different from the first method.

In an embodiment, the processor 210 may obtain a distance value between the external device and the electronic device 100 through wireless communication with the external device. When the change range of the obtained distance value is within the fourth change range, the processor 210 may determine that the detection of the first event is normal recognition. The processor 210 may determine that the detection of the first event is erroneous when the change range of the obtained distance value is a fifth change range that is larger than the fourth change range.

In an embodiment, the method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) includes the electronic device based on motion information acquired through at least one motion sensor. An operation of detecting a first event that has occurred to the user of the device, an operation of determining whether to erroneously recognize the detection of the first event based on the atmospheric pressure data obtained through the atmospheric pressure sensor 230, and the detection of the first event In the case of normal recognition, the method may include displaying a user interface related to a score determined based on motion information obtained through the at least one motion sensor on a display. In an embodiment, the first event generated to the user may be a hand washing event. The at least one motion sensor may include an acceleration sensor 220 .

In an embodiment, the method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) may include the electronic device based on acceleration data acquired through the acceleration sensor 220 . It may include calculating a magnitude of the acceleration of the device, detecting a periodic motion of the electronic device based on the magnitude of the acceleration, and detecting a first event occurring to the user based on the periodic motion. have.

In an embodiment, the method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) detects a change in air pressure for a first time through the barometric pressure sensor 230 . detecting operation, determining that the detection of the first event is normal recognition when the detected change range of air pressure is within the first change range, and determining that the detected air pressure change range is greater than the first change range In the case of a large second change range, determining that the detection of the first event is an erroneous recognition may be included.

In an embodiment, in the method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ), the sensed barometric pressure change range is larger than the second change range. In the case of the third change range, determining whether the detection of the first event is erroneous based on the acceleration horizontal component and the acceleration vertical component obtained through the at least one motion sensor may be included.

In an embodiment, a method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) may include a roll of the electronic device through an acceleration sensor and/or a gyro sensor. ) direction, an operation of obtaining a direction value for a pitch direction, an operation of recognizing at least one motion corresponding to the obtained direction value based on the obtained direction value, and It may include an operation of determining a hand washing area based on the.

In an embodiment, in the method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ), the first event is detected from a point in time when the occurrence of the first event is detected. It may include an operation of setting the period up to the end time point as at least one session, an operation of calculating a total time for hand washing within the set at least one session, and an operation of calculating an accumulated time for each hand washing part.

In an embodiment, the operating method of the electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) may increase as the accumulated time for each washing part is uniform within the at least one session. , it may include an operation of calculating a high score.

In an embodiment, the method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) includes obtaining information about an external environment from an external server, performing the method of calculating the score in a first manner when the external environment is a first environment based on information on the external environment; and when the external environment is a second environment different from the first environment, and performing a second method different from the first method for calculating the score.

In an embodiment, a method of operating an electronic device (eg, the electronic device 100 of FIG. 1A and/or the electronic device 200 of FIG. 1B ) may include a distance between the external device and the electronic device through wireless communication with the external device. obtaining a value, determining that the detection of the first event is normal recognition when the change range of the obtained distance value is within a fourth change range, and determining that the change range of the obtained distance value is within the fourth change range If the fifth change range is larger than the change range, determining that the detection of the first event is a misrecognition may include.

The electronic device according to various embodiments disclosed in this document may be a device of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of, or all possible combinations of, items listed together in the corresponding one of the phrases. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other such components, and may refer to components in other aspects (eg, importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, for example, and interchangeably with terms such as logic, logic block, component, or circuit. can be used A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present disclosure, one or more instructions stored in a storage medium (eg, internal memory 1736 or external memory 1738) readable by a machine (eg, electronic device 1701) may be implemented as software (eg, a program 1740) including For example, a processor (eg, processor 1720 ) of a device (eg, electronic device 1701 ) may call at least one of one or more instructions stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not include a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment, the method according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly between smartphones (eg: smartphones) and online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. . According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, omitted, or , or one or more other operations may be added.

Claims (20)

In an electronic device,
at least one motion sensor;
barometric pressure sensor;
display; and
at least one processor operatively coupled to the at least one motion sensor, the barometric pressure sensor, and the display;
The at least one processor comprises:
detecting a first event occurring to the user of the electronic device based on motion information acquired through the at least one motion sensor;
Based on the barometric data acquired through the barometric pressure sensor, it is determined whether the detection of the first event is misrecognized,
When the detection of the first event is normal recognition, displaying a user interface related to a score determined based on motion information obtained through the at least one motion sensor on the display. The method according to claim 1,
The first event that occurred to the user is a hand washing event,
the at least one motion sensor comprises an acceleration sensor,
The at least one processor comprises:
calculating the magnitude of the acceleration of the electronic device based on the acceleration data obtained through the acceleration sensor;
detecting a periodic motion of the electronic device based on the magnitude of the acceleration;
and detecting a first event occurring to the user based on the periodic movement. The method according to claim 1,
The at least one processor comprises:
Detecting a change in air pressure for a first time through the barometric pressure sensor,
When the detected change range of the atmospheric pressure is the first change range, it is determined that the detection of the first event is normal recognition,
and determining that the detection of the first event is an erroneous recognition when the sensed change range of the atmospheric pressure is a second change range that is larger than the first change range. 4. The method according to claim 3,
The at least one processor comprises:
When the sensed change range of air pressure is a third change range that is larger than the second change range, based on the acceleration horizontal component and the acceleration vertical component obtained through the at least one motion sensor, An electronic device for determining whether the detection is a misrecognition. The method according to claim 1,
The at least one motion sensor includes an acceleration sensor and/or a gyro sensor,
The at least one processor comprises:
obtaining direction values for a roll direction and a pitch direction of the electronic device through the acceleration sensor and/or the gyro sensor;
Recognizing at least one motion corresponding to the obtained direction value based on the obtained direction value,
and determining a hand washing part based on the recognized motion information, and wherein the at least one motion is a predetermined motion according to the direction value. 6. The method of claim 5,
The at least one processor comprises:
Setting a period from the time when the occurrence of the first event is detected to the time when the first event is terminated is at least one session;
and calculating a total hand washing time and an accumulated time for each hand washing part based on a movement of the electronic device within the set at least one session. 7. The method of claim 6,
The at least one processor calculates a higher score when the total accumulated time of washing hands exceeds a recommended time, and calculates a higher score as the accumulated time for each part of the hand is uniform within the at least one session. . 7. The method of claim 6,
The total time for washing the hands excludes a time during which no movement of the electronic device is detected within the session. The method according to claim 1,
The user interface includes at least one of a user interface guiding the determined score, a user interface guiding the score and a guide corresponding to the score, and a user interface guiding the unwashed portion,
The at least one processor provides at least one of a vibration notification and a sound notification in response to the display of the user interface. The method according to claim 1,
The at least one processor comprises:
Acquire information about the external environment from an external server,
Based on the obtained information on the external environment, when the external environment is a first environment, the method for calculating the score is performed in a first manner;
and when the external environment is a second environment different from the first environment, calculating the score using a second method different from the first method. The method according to claim 1,
The at least one processor comprises:
acquiring a distance value between the external device and the electronic device through wireless communication with the external device;
If the change range of the obtained distance value is within a fourth change range, it is determined that the detection of the first event is normal recognition,
and determining that the detection of the first event is a misrecognition when the change range of the obtained distance value is a fifth change range that is larger than the fourth change range. A method of operating an electronic device, comprising:
detecting a first event occurring to a user of the electronic device based on motion information acquired through at least one motion sensor;
determining whether the detection of the first event is misrecognized based on the atmospheric pressure data acquired through the atmospheric pressure sensor; and
When the detection of the first event is normal recognition, displaying a user interface related to a score determined based on motion information obtained through the at least one motion sensor on a display. 13. The method of claim 12,
The first event that occurred to the user is a hand washing event,
the at least one motion sensor comprises an acceleration sensor,
calculating an acceleration magnitude of the electronic device based on the acceleration data acquired through the acceleration sensor;
detecting a periodic motion of the electronic device based on the magnitude of the acceleration; and
and detecting a first event occurring to the user based on the periodic movement. 13. The method of claim 12,
detecting a change in air pressure for a first time through the pressure sensor;
determining that the detection of the first event is normal recognition when the detected change range of the atmospheric pressure is a first change range; and
and determining that the detection of the first event is an erroneous recognition when the sensed change range of air pressure is a second change range that is larger than the first change range. 15. The method of claim 14,
When the sensed change range of air pressure is a third change range that is larger than the second change range, detection of the first event based on the acceleration horizontal component and the acceleration vertical component obtained through the at least one motion sensor and determining whether this is a misrecognition. 13. The method of claim 12,
obtaining direction values for a roll direction and a pitch direction of the electronic device through an acceleration sensor and/or a gyro sensor;
recognizing, based on the obtained direction value, at least one motion corresponding to the obtained direction value; and
and determining a hand washing area based on the recognized motion information. 17. The method of claim 16,
setting a period from a time point when the occurrence of the first event is detected to a time point at which the first event ends as at least one session;
and calculating a total hand washing time and an accumulated time for each hand washing part within the set at least one session. 18. The method of claim 17,
and calculating a higher score as the cumulative time for each washing part is uniform within the at least one session. 13. The method of claim 12,
obtaining information about an external environment from an external server;
performing, based on the acquired information on the external environment, a method of calculating the score as a first method when the external environment is a first environment; and
and performing, when the external environment is a second environment different from the first environment, calculating the score in a second method different from the first method. 13. The method of claim 12,
acquiring a distance value between the external device and the electronic device through wireless communication with the external device;
determining that the detection of the first event is normal recognition when the change range of the acquired distance value is within a fourth change range; and
and determining that the detection of the first event is an erroneous recognition when the change range of the obtained distance value is a fifth change range that is larger than the fourth change range.

Images