KR20220012061A - Electronic device identifying height of object included in image and method for operating thereof - Google Patents

Abstract

According to various embodiments of the present invention, an electronic device for checking height of an object in an image may check a position of a feature point based on a model regarding the object. According to various embodiments, the electronic device comprises a camera module and at least one processor. The at least one processor may be set to obtain an image including a first object of a first type and the ground through the camera module, check the ground, check a position of a feature point of the first object, check a first plane corresponding to the first object, and check a distance between a line where the first plane and the ground intersect and the feature point. Various other embodiments are possible.

Description

Electronic device for checking the height of an object in an image and its operation method

Various embodiments of the present disclosure relate to an electronic device for checking the height of an object in an image, and an operating method thereof.

In an electronic device including a camera, various additional functions other than simply acquiring an image are being developed. For example, the electronic device may perform face recognition on an image acquired through a camera, and may apply a user-specified effect to the acquired image based on the face recognition result. In another example, the electronic device may recognize a defined type of object, such as a QR code, from an image acquired through a camera. Alternatively, the electronic device including the camera may measure the height of a person included as a subject in the acquired image.

In an existing method of measuring the height of a person included as a subject in an image acquired by an electronic device, there are several factors that limit accuracy. For example, according to the angle of the camera, a key point that is a standard for calculating the height, for example, the position of the crown of a person who is a subject, is calculated differently from the actual key point position, thereby limiting the accuracy of measurement. Alternatively, when a person who is a subject is wearing a hat, a position higher than the actual position of the characteristic point is calculated as the characteristic point because of the hat, and thus the accuracy of measurement may be limited. Alternatively, when a feature point is calculated using a depth image, since there is no depth value for a human hair part, measurement accuracy may be limited.

The electronic device for checking the height of the object in the image according to various embodiments of the present disclosure may check the position of the feature point based on a model regarding the object.

According to various embodiments, the electronic device includes a camera module and at least one processor, wherein the at least one processor acquires, through the camera module, an image including a first object of a first type and a ground, Check the ground, check the location of the feature point of the first object, check the first plane corresponding to the first object, and check the distance between the line where the first plane and the ground intersect and the feature point can be set to

According to various embodiments of the present disclosure, a method performed in an electronic device includes an operation of acquiring an image including a first object of a first type and a ground, an operation of checking the ground, and an operation of checking a position of a feature point of the first object The method may include an operation, an operation of identifying a first plane corresponding to the first object, and an operation of determining a distance between a line where the first plane and the ground intersect and the feature point.

According to various embodiments, an electronic device for checking a height of an object in an image and a method of operating the same are provided. The electronic device according to various embodiments may accurately identify the feature point regardless of the angle of the camera or whether the hat is worn by confirming the position of the feature point based on the object model. In addition, the electronic device according to various embodiments identifies a distance from a feature point to a line where the first plane corresponding to the object and the ground intersect as the height of the object, so that even if the ground is tilted or there is another object covering the ground in front of the object, the object height can be accurately determined.

In addition, various effects directly or indirectly identified through this document may be provided.

1 is a block diagram of an electronic device in a network environment, according to various embodiments of the present disclosure;
2A is a block diagram of an electronic device according to various embodiments of the present disclosure;
2B is a flowchart illustrating an operation of an electronic device according to various embodiments of the present disclosure;
3A illustrates an example of a situation in which an electronic device identifies a location of a feature point according to various embodiments of the present disclosure;
3B illustrates an example of a situation in which an electronic device identifies a location of a feature point according to various embodiments of the present disclosure.
4A illustrates an example of an image acquired by an electronic device according to various embodiments of the present disclosure;
4B illustrates an example of a situation in which an electronic device identifies a region corresponding to an object according to various embodiments of the present disclosure.
4C illustrates an example of a situation in which the electronic device checks the height of the first object according to various embodiments of the present disclosure.
4D illustrates the height of the first object checked according to the comparative example and various embodiments when the ground is inclined.
5 is a flowchart illustrating an operation of an electronic device according to various embodiments of the present disclosure;
6 is a flowchart illustrating an operation of an electronic device according to various embodiments of the present disclosure;
7A illustrates an example of an exoskeleton recognition result of an electronic device, according to various embodiments.
7B illustrates an example of correction based on an exoskeleton recognition result, according to various embodiments.
8A illustrates an example of a screen displayed on an electronic device, according to various embodiments.
8B illustrates an example of a screen displayed on an electronic device, according to various embodiments.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 may be included. In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in the volatile memory 132 , and may process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 may use less power than the main processor 121 or may be set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the co-processor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ). The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used in a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . A sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 includes various technologies for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

2A is a block diagram of an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 2A , the electronic device 200a may include a processor 210a, a communication module 220a, a memory 230a, a sensor module 240a, a camera module 250a, and a display 260a. .

According to various embodiments, the processor 210a may perform an operation of the electronic device 200a according to one or more instructions stored in the memory 230a. A detailed description of the operation of the electronic device 200a performed by the processor 210a will be described later with reference to FIGS. 2B to 8B .

According to an embodiment, the processor 210a may be operatively, functionally, or electrically connected to other components of the electronic device 200a.

According to an embodiment, the processor 210a may detect at least one object from the image obtained from the camera module 250a. For example, the processor 210a may analyze the image obtained from the camera module 250a.

According to various embodiments, the memory 230a may be implemented substantially the same as or similar to the memory 130 of FIG. 1 .

According to an embodiment, the memory 230a may be operatively connected to the processor 210a. The memory 230a may store various data used by at least one component (eg, the processor 210a or the camera module 250a) of the electronic device 200a. The data may include, for example, input data or output data for software (eg, the program 140 of FIG. 1 ) and instructions related thereto.

According to various embodiments, the display 260a may be implemented substantially the same as or similar to the display module 160 of FIG. 1 .

According to an embodiment, the display 260a may display various screens according to the use of the electronic device 200a under the control of the processor 210a. For example, the display 260a may display an image acquired from the camera module 250a by the processor 210a.

According to various embodiments, the camera module 250a may be implemented substantially the same as or similarly to the camera module 180 of FIG. 1 , and includes at least one camera located at the front and at least one camera located at the rear. may include

According to an embodiment, the camera module 250a may include a distance measuring camera. The ranging camera may include a time of flight (TOF) camera and/or a depth camera. According to an embodiment, the electronic device 200a recognizes one of the objects located in the gaze direction (eg, FOV) that the user is looking at by using the camera module 250a, and measures the distance to the object by the depth camera. You can calculate the depth through , or measure the distance to the object through the TOF camera.

According to various embodiments, the sensor module 240a may be implemented substantially the same as or similar to the sensor module 176 of FIG. 1 .

According to an embodiment, the sensor module 240a may include a gravity sensor.

For example, the processor 210a may detect the direction of gravity of the electronic device 200a using raw data received from the sensor module 240a.

According to various embodiments, the communication module 220a may be implemented substantially the same as or similar to the communication module of FIG. 1 .

According to an embodiment, the processor 210a may perform wireless communication with another electronic device through the communication module 220a. For example, the processor 210a may transmit an image obtained from the camera module to an external server through a communication module, and may receive analysis information of the image from the external server.

2B is a flowchart illustrating an operation of an electronic device according to various embodiments of the present disclosure; 3A and 3B illustrate examples of a situation in which an electronic device identifies a location of a feature point according to various embodiments of the present disclosure. 4A illustrates an example of an image acquired by an electronic device according to various embodiments of the present disclosure; 4B illustrates an example of a situation in which an electronic device identifies a region corresponding to an object according to various embodiments of the present disclosure. 4C illustrates an example of a situation in which the electronic device checks the height of the first object according to various embodiments of the present disclosure. 4D illustrates the height of the first object checked according to the comparative example and various embodiments when the ground is inclined.

According to various embodiments, the operations illustrated in FIG. 2B are instructions (eg, processor 120 of FIG. 1 ) that may be performed by at least one processor (eg, processor 120 of FIG. 1 ) of the electronic device (eg, electronic device 101 of FIG. 1 ). commands) can be implemented. The operations illustrated in FIG. 2B are not limited to the illustrated order and may be performed in various orders. Also, according to various embodiments, more operations or fewer operations than those illustrated in FIG. 2B may be performed or at least one operation may be performed. Hereinafter, with reference to FIGS. 3A to 3B and 4A to 4D, FIG. 2B will be described.

According to various embodiments, in operation 210 , at least one processor (eg, the processor 120 ) of the electronic device (eg, the electronic device 101 ) is configured with a camera module (eg, the camera module 180 ). )), an image including the first object of the first type and the ground may be acquired. According to various embodiments, the camera module 180 may acquire depth data corresponding to the acquired image. For example, the camera module 180 may include a depth camera. The distance measuring camera may measure a distance to an object located in front of the electronic device 101 . The processor 120 may acquire depth data corresponding to a distance of an object included in an image obtained by using a distance measuring camera.

According to various embodiments, the type of the object may indicate the type of the subject. For example, the type of object may include a person, a building, or an animal. The image acquired in operation 210 may include a specified first type of object among various types of objects. Hereinafter, an example in which the first type of object is a person will be mainly described. However, a person skilled in the art will understand that the electronic device according to various embodiments may check a distance from the ground with respect to various types of objects.

In operation 220, the at least one processor 120 may check the ground based on the acquired image. For example, the ground can be checked using a function of an AR software development kit (AR SDK) such as augmented reality core (ARCore) or ARKit. According to various embodiments, when the at least one processor 120 acquires depth data corresponding to the image acquired from the camera module 180, the at least one processor 120 may check the ground based on the acquired image and the depth data.

According to various embodiments, the at least one processor 120 may check the ground based on sensor information obtained from a sensor module (eg, the sensor module 176 of FIG. 1 ). For example, the at least one processor 120 may detect the direction of gravity based on raw data obtained from the gravity sensor. The at least one processor 120 may identify a plane substantially perpendicular to the detected direction of gravity as the ground in the image acquired from the camera module 180 .

In operation 230, the at least one processor 120 may identify the position of the feature point of the first object in the acquired image. According to various embodiments, the at least one processor 120 recognizes the first object in the acquired image based on the model of the first type of object (hereinafter referred to as the first object) to locate the feature point. can be checked. According to an embodiment, the model DB according to the type of object may be stored in the memory 130 of the electronic device 101 or implemented by an external electronic device. For example, a model for the first type of object may be stored in the memory 130 . For example, when the first type of the object is a human, the at least one processor 120 may fit the first object in the obtained image to the human head model based on the human head model. According to various embodiments, the at least one processor 120 does not directly perform an operation of fitting the first object to a model related to the object, but provides an external electronic device (eg, a server) through the communication module 190 . The acquired image is transmitted, and information about the fitting result (eg, at least one fitted part fitted to the model related to the object) is transmitted through the communication module 190 by referring to the model related to the object in the external electronic device. may be received from the electronic device.

According to an embodiment, the at least one processor 120 may identify a point farthest from the ground confirmed in operation 220 among parts fitted to the human head model in the first object as a feature point. According to various embodiments, When the camera module 180 (eg, a distance measuring camera) acquires depth data corresponding to the image, the at least one processor 120 fits the first object to the three-dimensional human head model, and from the first object to the three-dimensional person Among the parts fitted to the head model, the point farthest from the ground confirmed in operation 220 may be identified as a feature point. Unlike various embodiments, when the location of the feature point is set based on the field of view (FOV) direction 320a of the camera module 180 as shown in FIG. 3A , the point 325a may be determined as the feature point. On the other hand, at least one processor 120 according to various embodiments identifies a point 315a furthest from the ground using a line 310a substantially parallel to the ground confirmed in operation 220, so the camera module 180 The feature point can be accurately identified regardless of the gaze direction 320a of the . Unlike various embodiments, when the position of the feature point is set without fitting the first object to the human head model, when the person who is the first object wears a hat, as shown in FIG. 3B , the point at the height 320b of the hat is It can be identified as a feature point. On the other hand, the at least one processor 120 according to various embodiments fits the first object to the human head model, and selects a point farthest from the ground identified in operation 220 among parts fitted to the human head model in the first object. Since it is confirmed as a feature point, a point at the height 310b farthest from the ground among parts corresponding to the head according to the human head model may be identified as the feature point. For example, the feature point may mean a reference point for confirming the height of the first object from the ground. Accordingly, the at least one processor 120 according to various embodiments may accurately identify the feature point even if at least a portion of the first object is obscured by another object.

According to various embodiments, the at least one processor 120 transmits the acquired image to an external electronic device (eg, a server) through the communication module 190, and the external electronic device relates to the object of the first type. Information on the feature point of the first object, which is confirmed based on a model (eg, a human head model), may be received from the external electronic device, and thus may be identified as the feature point of the first object.

In operation 240 , the at least one processor 120 may identify a first plane corresponding to the first object. According to various embodiments, when the camera module 180 does not acquire depth data corresponding to the image, the at least one processor 120 performs exoskeleton recognition on the first object, Based on the first plane corresponding to the first object may be identified.

According to various embodiments, when the camera module 180 acquires depth data corresponding to an image, the at least one processor 120 may identify the first plane by using the depth data. According to an embodiment, the at least one processor 120 may identify a region corresponding to the first object in the image acquired from the camera module 180 . For example, when the image obtained by the at least one processor 120 from the camera module 180 is the image 400a of FIG. 4A and the first type of the object is a person, the at least one processor 120 performs the first A region 410b corresponding to the object 410a may be identified. Referring to FIG. 4B , according to various embodiments, the at least one processor 120 may identify a region 410b corresponding to the first object 410a based on a deep learning algorithm. According to various embodiments, the at least one processor 120 transmits an image to an external electronic device (eg, a server) through the communication module 190, and the third identified based on the deep learning algorithm in the external electronic device. By receiving information about the area 410b corresponding to the first object 410a from the external electronic device, the area 410b corresponding to the first object 410a may be identified.

According to an exemplary embodiment, the at least one processor 120 is configured to generate a plurality of 3D points corresponding to the first object 410a based on depth data corresponding to the region 410b corresponding to the first object 410a. The coordinates may be checked, and a plane that minimizes an error with a plurality of three-dimensional points may be identified as the first plane. For example, the error between a plurality of three-dimensional points and an arbitrary plane may include a square root of a sum of the squares of the distances between each three-dimensional point and an arbitrary plane for all the plurality of three-dimensional points.

In operation 250 , the at least one processor 120 may determine the distance between the feature point and the line where the first plane and the ground intersect as the height of the first object. Referring to FIG. 4C , the at least one processor 120 checks the ground 410c, checks the feature point 430c of the first object 420c, and a first plane ( After checking 440c), the distance between the line 450c where the first plane 440c and the ground 410c intersect and the feature point 430c may be checked as the height of the first object 420c. For example, the distance between the line 450c where the first plane 440c and the ground plane 410c intersect and the feature point 430c is the line 450c where the first plane 440c and the ground plane 410c intersect and the feature point. It may mean a substantially vertical straight line distance between the 430c. Referring to FIG. 4D , in a situation where the ground 410d is inclined in a range of an angle with respect to the +x axis, the feature point 430d of the first object ) and the distance 421d between the ground and the height of the first object may be different. On the other hand, the distance 422d between the feature point and the line where the first plane corresponding to the first object and the ground intersect may accurately represent the height of the first object.

In addition, according to various embodiments, since the at least one processor 120 checks the distance between the feature point and the line where the first plane and the ground intersect as the height of the first object, the first object and the ground are different subjects The height of the first object may be checked by extending at least one of the first plane or the ground even when it is covered by the .

5 is a flowchart illustrating an operation of an electronic device according to various embodiments of the present disclosure; According to various embodiments, the operations shown in FIG. 5 are instructions (eg, the processor 120 of FIG. 1 ) that may be performed by at least one processor (eg, the processor 120 of FIG. 1 ) of the electronic device (eg, the electronic device 101 of FIG. 1 ). commands) can be implemented. The operations illustrated in FIG. 5 are not limited to the illustrated order and may be performed in various orders. In addition, according to various embodiments, more operations or fewer operations than those illustrated in FIG. 5 may be performed or at least one operation may be performed.

In operation 510 , at least one processor (eg, the processor 120 ) of the electronic device (eg, the electronic device 101 ) performs a first operation through a camera module (eg, the camera module 180 ). An image including the first object of the type and the ground may be acquired. In operation 520, the at least one processor 120 may identify the ground based on the acquired image. In operation 530, the at least one processor 120 may determine the location of the feature point of the first object in the acquired image. In operation 540, the at least one processor 120 may identify a first plane corresponding to the first object. In operation 550 , the at least one processor 120 may determine a distance between a line where the first plane and the ground intersect and a feature point as the height of the first object. Since the details of operations 210 to 250 may be equally applied to operations 510 to 550, they will not be repeated herein.

In operation 560, the at least one processor 120 may perform exoskeleton recognition on the first object. The exoskeleton recognition may be an operation of fitting the first object to an exoskeleton model related to the first type of object. According to various embodiments, the at least one processor 120 may perform exoskeleton recognition with reference to the exoskeleton model stored in the memory 130 of the electronic device 101 . According to various embodiments, the at least one processor 120 transmits the acquired image to an external electronic device (eg, a server) through the communication module 190 without directly performing exoskeleton recognition, and the external electronic device Information on the exoskeleton recognition result confirmed by referring to the exoskeleton model may be received from the external electronic device through the communication module 190 .

In operation 570 , the at least one processor 120 may correct the distance between the feature point and the line where the first plane and the ground intersect based on the exoskeleton recognition result, and check the corrected value as the height of the first object. According to various embodiments, the at least one processor 120 may correct the distance between the feature point and the line where the first plane and the ground intersect through an operation that will be described later with reference to operations 610 to 640 of FIG. 6 . According to various embodiments, the at least one processor 120 may correct the distance between the feature point and the line where the first plane and the ground intersect by performing operation 630 of FIG. 6 .

6 is a flowchart illustrating an operation of an electronic device according to various embodiments of the present disclosure; 7A illustrates an example of an exoskeleton recognition result of an electronic device, according to various embodiments. 7B illustrates an example of correction based on an exoskeleton recognition result, according to various embodiments. According to various embodiments, the operations shown in FIG. 6 are instructions (eg, processor 120 of FIG. 1 ) that may be performed by at least one processor (eg, processor 120 of FIG. 1 ) of the electronic device (eg, electronic device 101 of FIG. 1 ). commands) can be implemented. According to various embodiments, more operations or fewer operations than those illustrated in FIG. 6 may be performed or at least one operation may be performed. Hereinafter, FIG. 6 will be described with reference to FIGS. 7A to 7B .

In operation 610, at least one processor (eg, processor 120) of the electronic device (eg, electronic device 101) determines a difference between the posture of the first object and the correct posture of the first object. can The posture of the first object may be determined based on the exoskeleton recognition result of operation 560 of FIG. 5 . In the example of FIG. 7A , as a result of exoskeleton recognition, the first object may be simplified in the form of an exoskeleton 710a specified by a plurality of nodes. Nodes 721a, 722a, 723a among all nodes 721a, 722a, 723a, 724a, 725a, 726a, 727a, 728a, 729a, 731a, 732a, 733a, 734a, 735a, 736a constituting the exoskeleton 710a , 724a, 725a, 726a, 727a, 728a, 729a) may be used to determine the posture of the first object. The at least one processor 120 determines that the posture of the first object is the correct posture when the nodes 721a, 722a, and 724a form a straight line and the straight line formed by the nodes 721a, 722a, 724a is in the vertical direction. It can be included as a condition for The at least one processor 120 determines that the posture of the first object is the correct posture when the nodes 723a, 726a, and 728a form a straight line and the straight line formed by the nodes 723a, 726a, 728a is in the vertical direction. It can be included as a condition for The at least one processor 120 determines that the posture of the first object is the correct posture when the nodes 725a, 727a, and 729a form a straight line and the straight line formed by the nodes 725a, 727a, 729a is in the vertical direction. It can be included as a condition for According to various embodiments, the at least one processor 120 defines the normal direction of the ground checked in operation 220 as a vertical direction, or a gravity direction determined according to a sensing value of the gravity sensor included in the sensor module 176. It can be defined in the vertical direction.

)를 제1 오브젝트의 자세와 제1 오브젝트의 바른 자세 사이의 차이로서 확인할 수 있다.In the example of FIG. 7B , the direction of the straight line formed by the nodes 723a and 728a corresponding to the leg may be different from the vertical direction. The at least one processor 120 determines the angle (

) may be identified as a difference between the posture of the first object and the correct posture of the first object.

)에 대해서는 적어도 하나의 프로세서(120)는 약 5° 이상 약 10° 이하의 각도를 결정된 수준으로서 결정할 수 있다. 다양한 실시예에 따라서, 결정된 수준은 노드들(721a, 722a, 723a, 724a, 725a, 726a, 727a, 728a, 729a) 중 어느 노드들이 형성하는 각도인지에 따라서 상이하게 설정될 수 있다.In operation 620 , the at least one processor 120 may determine whether a difference between the posture of the first object and the correct posture of the first object is equal to or less than a determined level. For example, the angle (

), the at least one processor 120 may determine an angle of about 5° or more and about 10° or less as the determined level. According to various embodiments, the determined level may be set differently depending on which of the nodes 721a , 722a , 723a , 724a , 725a , 726a , 727a , 728a , and 729a forms an angle.

When it is determined that the difference between the posture of the first object and the correct posture of the first object is less than or equal to the determined level, the at least one processor 120 determines the distance between the feature point and the line where the first plane and the ground intersect in operation 630 . can be corrected In the example of FIG. 7B , when checking the distance L between the feature point and the line where the first plane and the ground intersect in operation 250, the distance L from the node 724a to the ground 731b along the vertical direction 730b ₁ ) was considered. However, instead of the distance L ₁ from the node 724a to the ground 731b along the vertical direction 730b, the length L ₂ of the straight line formed by the nodes 723a and 728a should be considered. The relationship as in Equation 1 is established between the distance L ₁ from the node 724a to the ground 731b along the vertical direction 730b and the length L ₂ of the straight line formed by the nodes 723a and 728a. can do.

The at least one processor 120 determines the distance L ₁ from the node 724a to the ground 731b along the vertical direction 730b based on the exoskeleton recognition result, and based on the identified distance L ₁ . Thus, as shown in Equation 2, the height L' of the first object can be confirmed by correcting the distance L between the line where the first plane and the ground intersect and the feature point 732b.

If it is determined that the difference between the posture of the first object and the correct posture of the first object exceeds the determined level, the at least one processor 120 requests, in operation 640, to change the posture of the first object to match the correct posture. message can be output. According to various embodiments, the at least one processor 120 may display a message through a display (eg, the display module 160 of FIG. 1 ) or output a voice message through the sound output module 155 .

Although not shown in FIG. 6 , according to various embodiments, even when it is determined that the difference between the posture of the first object and the correct posture of the first object is less than or equal to the determined level, the at least one processor 120 performs operation 630 . If correction of the distance between the feature point and the line where the first plane intersects the ground is attempted but cannot be corrected, the at least one processor 120 may perform operation 640 .

8A illustrates an example of a screen displayed on an electronic device, according to various embodiments. In the screen 800a of FIG. 8A , an image acquired through the camera module 180 may be displayed as a preview image through a display (eg, the display module 160 of FIG. 1 ), and on the screen 800a, the first An object 810a and a shutter button 801a corresponding to a photographing command may be included. In the example of FIG. 8A , the at least one processor 120 of the electronic device 101 determines that the difference between the posture of the first object and the correct posture of the first object exceeds the determined level, or the at least one processor 120 If correction of the distance between the first plane and the ground intersecting line and the feature point is attempted but cannot be corrected, a message 820a requesting to change the posture of the first object 810a to match the correct posture may be displayed. .

Although FIG. 8A illustrates an example in which a message 820a of “Please stand in a correct posture” is displayed, according to various embodiments, the at least one processor 120 of the electronic device 101 determines the posture and the second position of the first object. If the difference between the correct postures of the first object exceeds the determined level, or the at least one processor 120 attempts to correct the distance between the line and the feature point where the first plane and the ground intersect, but cannot correct it, the first object It is possible to identify a problem in the posture of , and display specific action guidelines for changing the posture of the first object to a correct posture based on the identified problem. The action guideline may indicate at least one of a body part to be moved in order for the posture of the first object to become a correct posture and an angle of movement required.

For example, referring to FIG. 7A , the at least one processor 120 determines that the nodes 721a, 722a, and 724a form a straight line, and the straight line formed by the nodes 721a, 722a, 724a is a vertical direction. 1 It may be included as a condition for determining that the posture of the object is the correct posture. If the nodes 721a, 722a, and 724a do not form a straight line, the straight line formed by the nodes 721a and 722a is not in the vertical direction, and the straight line formed by the nodes 722a and 724a is in the vertical direction, at least one The processor 120 may determine that the head direction of the first object is not straight, and display a message to straighten the head direction as a specific action guide. According to various embodiments, the at least one processor 120 may display at least one of a body part to be moved and an angle of movement required for the posture of the first object to become a correct posture. For example, the at least one processor 120 may display a message "Tilt your head 3° to the right".

In another example, the nodes 721a, 722a, 724a form a straight line, and the straight line formed by the nodes 721a, 722a, 724a is in the vertical direction is included as a condition for determining that the posture of the first object is the correct posture In this case, when the nodes 721a, 722a, 724a form a straight line, and the straight line formed by the nodes 721a, 722a, 724a is not in the vertical direction, at least one processor 120 determines that the first object is a waist can be judged to be bowing down. In this case, the at least one processor 120 may display a message "Please straighten your back" or "Raise your back by 3°".

8B illustrates an example of a screen displayed on an electronic device, according to various embodiments. According to various embodiments, the at least one processor 120 may display the exoskeleton recognition result 820b of the first object 810b overlapping the first object 810b on the screen 800b. According to various embodiments, the at least one processor 120 may display the exoskeleton 830b indicating the correct posture for the first object 810b on the screen 800b by overlapping the first object 810b. . According to various embodiments, the at least one processor 120 generates visual signs 840b and 845b indicating the direction of movement of the first object 810b for matching the posture of the first object 810b with the correct posture. One object 810b may be overlapped and displayed on the screen 800b. Although not shown in FIG. 8B , according to various embodiments, the at least one processor 120 indicates a movement angle required for the first object 810b so that the posture of the first object 810b matches the correct posture. The visual mark may be displayed on the screen 800b by overlapping the first object 810b.

According to various embodiments of the present disclosure, the electronic device 101 includes a camera module 180 and at least one processor 120 , and the at least one processor 120, through the camera module 180 , provides a first Acquire an image including a first object of a type and a ground, check the ground, check a position of a feature point of the first object, check a first plane corresponding to the first object, and It may be set to check a distance between a line intersecting a plane and the ground and the feature point.

According to various embodiments, the at least one processor 120 may be configured to confirm the location of the feature point based on the model of the first type of objects.

According to various embodiments, the at least one processor 120 may be configured to check the ground based on depth data corresponding to the image.

According to various embodiments, the at least one processor 120 may be configured to identify the first plane based on depth data corresponding to the image.

According to various embodiments, the at least one processor 120 identifies a region corresponding to the first object in the image, and selects a plurality of 3D points corresponding to the first object based on the depth data. It may be set to confirm the coordinates and to identify a plane that minimizes an error with the plurality of three-dimensional points as the first plane.

According to various embodiments, the at least one processor 120 may be configured to perform exoskeleton recognition on the first object and identify the first plane based on the exoskeleton recognition result.

According to various embodiments, the at least one processor 120 may recognize an exoskeleton corresponding to the first object, and based on a result of the exoskeleton recognition, a line intersecting the first plane and the ground and the feature point can be set to correct the distance between them.

According to various embodiments, the at least one processor 120 checks the difference between the posture of the first object and the correct posture of the first object based on the exoskeleton recognition result, and determines the level at which the difference is determined In the following case, based on the difference between the posture of the first object and the correct posture of the first object, it may be set to correct the distance between a line where the first plane and the ground intersect and the feature point .

According to various embodiments, when the difference exceeds a determined level, the at least one processor 120 may be configured to output a message requesting to change the posture of the first object to match the correct posture. have.

According to various embodiments, when the difference exceeds a determined level, the at least one processor 120 may be configured to display the correct posture on the display 160 .

According to various embodiments of the present disclosure, the method performed by the electronic device 101 includes an operation of acquiring an image including a first type of first object and a ground, an operation of checking the ground, and a location of a feature point of the first object. It may include an operation of confirming , an operation of confirming a first plane corresponding to the first object, and an operation of confirming a distance between a line where the first plane and the ground intersect and the feature point.

According to various embodiments, the operation of confirming the location of the feature point of the first type of object may include the operation of confirming the location of the feature point based on a model of the first type of objects.

According to various embodiments, the method may further include checking the ground based on depth data corresponding to the image.

According to various embodiments, the checking of the first plane corresponding to the first type of object may include checking a region corresponding to the first object in the image based on depth data corresponding to the image. to check the coordinates of a plurality of 3D points corresponding to the first object, and checking a plane that minimizes an error with the plurality of 3D points as the first plane.

According to various embodiments, the identifying of the first plane corresponding to the first type of object may include performing recognition of the exoskeleton corresponding to the first object and the first plane based on a result of recognizing the exoskeleton. It may include an operation to confirm.

According to various embodiments of the present disclosure, the method may include performing exoskeleton recognition on the first object and correcting the distance between the feature point and a line where the first plane intersects the ground based on the exoskeleton recognition result It may further include an operation to

According to various embodiments, the correcting of the distance between the line where the first plane and the ground intersect and the feature point based on the exoskeleton recognition result is based on the exoskeleton recognition result of the first object an operation of confirming a difference between a posture and the correct posture of the first object, and when the difference is less than or equal to a determined level, based on the difference between the posture of the first object and the correct posture of the first object, the The method may include correcting the distance between a line crossing the first plane and the ground and the feature point.

According to various embodiments of the present disclosure, when the difference exceeds the determined level, the method may further include displaying a message requesting to change the posture of the first object to match the correct posture.

According to various embodiments, the method may further include outputting a voice requesting to change the posture of the first object to match the correct posture when the difference exceeds the determined level.

According to various embodiments, the method may further include displaying the correct posture when the difference exceeds the determined level.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the element from other elements in question, and may refer to elements in other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, a processor (eg, processor 120 ) of a device (eg, electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not include a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly between smartphones (eg: smartphones) and online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

Claims (20)

In an electronic device,
camera module; and
at least one processor operatively connected to the camera module;
The at least one processor,
Obtaining an image including the first object and the ground of the first type through the camera module,
check the ground,
Check the position of the feature point of the first object,
Check the first plane corresponding to the first object,
The electronic device is configured to identify a distance between a line intersecting the first plane and the ground and the feature point.
The method of claim 1, wherein the at least one processor comprises:
The electronic device is configured to identify a location of a feature point based on a model regarding the first type of objects.
The method of claim 1, wherein the at least one processor comprises:
The electronic device is configured to identify the ground based on depth data corresponding to the image.
The method of claim 1, wherein the at least one processor comprises:
and identify the first plane based on depth data corresponding to the image.
The method of claim 4, wherein the at least one processor comprises:
Checking an area corresponding to the first object in the image,
Check the coordinates of a plurality of three-dimensional points corresponding to the first object based on the depth data,
The electronic device is configured to identify a plane that minimizes an error with the plurality of three-dimensional points as the first plane.
The method of claim 1, wherein the at least one processor comprises:
performing exoskeleton recognition on the first object,
The electronic device is configured to identify the first plane based on a result of recognizing the exoskeleton.
The method of claim 1, wherein the at least one processor comprises:
performing exoskeleton recognition corresponding to the first object,
and correcting the distance between a line where the first plane and the ground intersect and the feature point based on a result of recognizing the exoskeleton.
The method of claim 7, wherein the at least one processor comprises:
Based on the exoskeleton recognition result, check a difference between the posture of the first object and the correct posture of the first object,
When the difference is equal to or less than the determined level, the distance between the first plane and the ground intersecting line and the feature point is corrected based on the difference between the posture of the first object and the correct posture of the first object An electronic device configured to do so.
9. The method of claim 8,
The at least one processor,
and outputting a message requesting to change the posture of the first object to match the correct posture when the difference exceeds the determined level.
9. The method of claim 8,
The at least one processor is configured to display the correct posture on a display when the difference exceeds a determined level.
A method performed in an electronic device, comprising:
an operation of acquiring an image including a first object of a first type and a ground;
checking the ground,
confirming the position of the feature point of the first object;
checking a first plane corresponding to the first object; and
and determining a distance between a line where the first plane and the ground intersect and the feature point.
12. The method of claim 11,
The operation of confirming the position of the feature point of the first type of object includes:
and determining a location of a feature point based on a model of the first type of objects.
12. The method of claim 11, wherein the method
The method further comprising the operation of identifying the ground based on the depth data corresponding to the image.
12. The method of claim 11,
The operation of identifying the first plane corresponding to the first type of object includes:
checking a region corresponding to the first object in the image;
identifying coordinates of a plurality of three-dimensional points corresponding to the first object based on depth data corresponding to the image; and
and identifying a plane that minimizes an error with the plurality of three-dimensional points as the first plane.
12. The method of claim 11,
The operation of identifying the first plane corresponding to the first type of object includes:
performing exoskeleton recognition corresponding to the first object;
and identifying the first plane based on the exoskeleton recognition result.
12. The method of claim 11, wherein the method comprises:
performing exoskeleton recognition on the first object, and
Based on a result of the recognition of the exoskeleton, the method further comprising correcting the distance between a line where the first plane and the ground intersect and the feature point.
17. The method of claim 16,
The operation of correcting the distance between a line where the first plane and the ground intersect and the feature point based on the exoskeleton recognition result comprises:
checking a difference between the posture of the first object and the correct posture of the first object based on the exoskeleton recognition result; and
When the difference is equal to or less than the determined level, the distance between the first plane and the ground intersecting line and the feature point is corrected based on the difference between the posture of the first object and the correct posture of the first object A method comprising the action of:
18. The method of claim 17, wherein the method comprises:
If the difference exceeds the determined level, displaying a message requesting to change the posture of the first object to match the correct posture.
18. The method of claim 17, wherein the method comprises:
and outputting a voice requesting to change the posture of the first object to match the correct posture when the difference exceeds the determined level.
18. The method of claim 17, wherein the method comprises:
and displaying the correct posture when the difference exceeds the determined level.

Images