WO2020145665A1 - Method for providing product information using electronic device and augmented reality - Google Patents

Abstract

A method for providing product information using augmented reality may comprise: an operation of collecting or receiving user health information; an operation of, when it is determined that a location of an electronic device is related to a predetermined location, driving a camera module; an operation of determining whether a user selects a product through a photographed image while driving the camera module; an operation of, when it is determined that the user selects a product, comparing information of the selected product and a product component information database from an image of the selected product to identify components of the selected product; an operation of comparing the components of the selected product and the user health information to determine whether the selected product affects user health; and an operation of, when it is determined that the selected product affects user health, searching for an alternative product. Figure 3

Description

Method for providing product information using electronic devices and augmented reality

Various embodiments of the present invention relate to a method for providing product information using augmented reality and an electronic device including the method.

Augmented reality is a technology that allows you to mix virtual images with the physical environment of the real world. In addition, augmented reality is used using devices that can be viewed by one or a few people. Thus, augmented reality systems can provide personalized experiences.

A method of providing product information using augmented reality according to various embodiments of the present invention and an electronic device including the method have an object to propose a product suitable for user health based on user health information and augmented reality.

A method of providing product information using augmented reality according to various embodiments of the present disclosure includes collecting or receiving user health information; When it is determined that the location of the electronic device is related to a predetermined location, driving the camera module; Determining whether a user selects a product through a photographed image while driving the camera module; When it is determined that the user selects a product, comparing the selected product information and the product component information database from the selected product image to check the component of the selected product; Comparing the components of the selected product and the user health information to determine whether the selected product affects the user health; And when it is determined that the selected product affects the user's health, searching for an alternative product.

An electronic device including a method for providing product information using augmented reality according to various embodiments of the present disclosure includes a sensor module; Communication module; Display device; Camera module; And a processor, wherein the processor collects or receives user health information, drives the camera module when it is determined that the location of the electronic device is related to a predetermined location, and uses an image captured while driving the camera module. It is determined whether the user has selected a product, and when the user determines that the product is selected, the selected product information is compared with the product information database to compare the selected product information and the product component information database from the image of the selected product. By comparing the components of the product with the user's health information, it is determined whether the selected product affects the user's health, and if it is determined that the selected product affects the user's health, an alternative product may be searched.

A method of providing product information using augmented reality according to various embodiments of the present invention and an electronic device including the method may propose a product suitable for user health.

1 is a block diagram of an electronic device in a network environment, according to various embodiments.

2A to 2C are diagrams illustrating electronic devices according to various embodiments of the present disclosure.

3 is a flowchart illustrating a method of providing product information using augmented reality of an electronic device according to various embodiments of the present disclosure.

4 is a diagram illustrating a method of providing information on an alternative product searched by an electronic device according to various embodiments of the present disclosure.

5 is a diagram illustrating a method of providing online product purchase information on an alternative product searched by an electronic device according to various embodiments of the present disclosure.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through the first network 198 (eg, a short-range wireless communication network), or the second network 199. It may communicate with the electronic device 104 or the server 108 through (eg, a remote wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, a sensor module ( 176), interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or antenna module 197 ). In some embodiments, at least one of the components (for example, the display device 160 or the camera module 180) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components may be implemented in one integrated circuit. For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illuminance sensor) may be implemented while being embedded in the display device 160 (eg, a display).

The processor 120, for example, executes software (eg, the program 140) to execute at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and can perform various data processing or operations. According to one embodiment, as at least part of data processing or computation, the processor 120 may receive instructions or data received from other components (eg, the sensor module 176 or the communication module 190) in the volatile memory 132. Loaded into, process instructions or data stored in volatile memory 132, and store result data in non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (eg, a central processing unit or an application processor), and an auxiliary processor 123 (eg, a graphics processing unit, an image signal processor) that can be operated independently or together. , Sensor hub processor, or communication processor). Additionally or alternatively, the coprocessor 123 may be set to use lower power than the main processor 121, or to be specialized for a specified function. The coprocessor 123 may be implemented separately from the main processor 121 or as a part thereof.

The coprocessor 123 may replace, for example, the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 may be active (eg, execute an application) ) With the main processor 121 while in the state, at least one component of the components of the electronic device 101 (eg, the display device 160, the sensor module 176, or the communication module 190) It can control at least some of the functions or states associated with. According to one embodiment, the coprocessor 123 (eg, image signal processor or communication processor) may be implemented as part of other functionally relevant components (eg, camera module 180 or communication module 190). have.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176). The data may include, for example, software (eg, the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

The program 140 may be stored as software in the memory 130, and may include, for example, an operating system 142, middleware 144, or an application 146.

The input device 150 may receive commands or data to be used for components (eg, the processor 120) of the electronic device 101 from outside (eg, a user) of the electronic device 101. The input device 150 may include, for example, a microphone, mouse, keyboard, or digital pen (eg, a stylus pen).

The audio output device 155 may output an audio signal to the outside of the electronic device 101. The audio output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from, or as part of, the speaker.

The display device 160 may visually provide information to the outside of the electronic device 101 (eg, a user). The display device 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to an embodiment, the display device 160 may include a touch circuitry configured to sense a touch, or a sensor circuit (eg, a pressure sensor) configured to measure the strength of the force generated by the touch. have.

The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 acquires sound through the input device 150, or an external electronic device (eg, directly or wirelessly connected to the sound output device 155 or the electronic device 101) Sound may be output through the electronic device 102 (eg, speakers or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biological sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that can be used for the electronic device 101 to be directly or wirelessly connected to an external electronic device (eg, the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or movement) or electrical stimuli that the user can perceive through tactile or motor sensations. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and videos. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101. According to one embodiment, the power management module 388 may be implemented, for example, as at least part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishing and performing communication through the established communication channel. The communication module 190 operates independently of the processor 120 (eg, an application processor) and may include one or more communication processors supporting direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg : Local area network (LAN) communication module, or power line communication module. The corresponding communication module among these communication modules includes a first network 198 (for example, a short-range communication network such as Bluetooth, WiFi direct, or infrared data association (IrDA)) or a second network 199 (for example, a cellular network, the Internet, or It can communicate with external electronic devices through a computer network (eg, a telecommunication network, such as a LAN or WAN). These various types of communication modules may be integrated into a single component (eg, a single chip), or may be implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses a subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be verified and authenticated.

The antenna module 197 may transmit a signal or power to the outside (eg, an external electronic device) or receive it from the outside. According to an embodiment, the antenna module may include a single antenna including a conductor formed on a substrate (eg, a PCB) or a radiator made of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network 198 or the second network 199, is transmitted from the plurality of antennas by, for example, the communication module 190. Can be selected. The signal or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, other components (eg, RFIC) other than the radiator may be additionally formed as part of the antenna module 197.

At least some of the components are connected to each other through a communication method between peripheral devices (for example, a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)). Ex: command or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the electronic devices 102 and 104 may be the same or a different type of device from the electronic device 101. According to an embodiment, all or some of the operations performed on the electronic device 101 may be performed on one or more external devices of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead executes the function or service itself. In addition or in addition, one or more external electronic devices may be requested to perform at least a portion of the function or the service. The one or more external electronic devices receiving the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and deliver the result of the execution to the electronic device 101. The electronic device 101 may process the result, as it is or additionally, and provide it as at least part of a response to the request. To this end, cloud computing, distributed computing, or client-server computing technology can be used, for example.

2A to 2C are diagrams illustrating electronic devices according to various embodiments of the present disclosure.

The electronic device of FIG. 2A may be a smartphone 201. The electronic device of FIG. 2B may be a smart watch 203 among wearable devices. The electronic device of FIG. 2C may be a smart glass 205 among wearable devices. 2A to 2C show a smart phone 201, a smart watch 203, and a smart glass 206. The smart phone 201, the smart watch 203, and the smart glasses 206 may be the same as the electronic device 101 disclosed in FIG. 1 and may include the electronic device 101 components disclosed in FIG. 1.

3 is a flowchart illustrating a method of providing product information using augmented reality of the electronic device 101 according to various embodiments of the present disclosure.

According to various embodiments of the present disclosure, in operation 301, under the control of the processor 120, the electronic device 101 may collect or receive user health information from an external device.

The user health information is information related to the health of the user of the electronic device 101, for example, body information such as weight, height, blood sugar, blood pressure, heart rate of the user of the electronic device 101, disease information such as medical diagnosis, or It may be health management information such as exercise amount, calories ingested during meals, and weight management. In addition, user health information may include other religious or vegetarian information. For example, disease information among user health information may be information such as allergy.

In operation 301, the electronic device 101 may collect health information of the user of the electronic device 101 using the sensor module 176. For example, the electronic device 101 may measure a user's heart rate and the like through the sensor module 176 and collect it as user health information. The sensor module 176 may further include a Heart Rate Monitoring (HRM) sensor for measuring a user's heart rate.

The electronic device 101 may receive the health information of the user of the electronic device 101 as well as the health information of the family of the user of the electronic device 101 from the external device 101. For example, the electronic device 101 may receive health information of a person other than the user of the electronic device 101 collected from the external device.

According to various embodiments of the present disclosure, in operation 301, the electronic device 101 may receive and store health information from an external device. According to various embodiments, the external device may be an electronic device related to the user's family as well as the electronic device 101. According to various embodiments, the external device may be an electronic device related to the electronic device 101 and other users. The electronic device 101 may be one of the smart phone 201, the smart watch 203, and the smart glasses 206, and the smart phone 201, the smart watch 203, and the smart glasses 206 may be used for user health. Information can be shared, transmitted, and received. When the electronic device 101 is the smartphone 201, the external device may be a smart watch 203 or a smart glass 205. When the electronic device 101 is the smart watch 203, the external device may be a smart phone 201 or a smart glass 205. When the electronic device 101 is the smart glass 205, the external device may be a smart phone 201 or a smart watch 203.

In operation 301, the electronic device 101 may collect health information of the user of the electronic device 101 by a user input. For example, the electronic device 101 may collect user health information (eg, body information, disease information, health care information, other information, etc.) by user input.

According to various embodiments of the present disclosure, in operation 303, under the control of the processor 120, the electronic device 101 may determine whether it is a store or a market of the electronic device 101.

According to various embodiments of the present disclosure, in operation 303, under the control of the processor 120, the electronic device 101 determines whether the location of the electronic device 101 is a predetermined position (eg, a store or a market). can do.

According to various embodiments of the present disclosure, in operation 303, under the control of the processor 120, the electronic device 101 determines whether the location of the electronic device 101 is related to a predetermined location (eg, a store or a market). The electronic device 101 may determine whether the location of the electronic device 101 is a store or a market through the communication module 190. For example, the electronic device 101 detects the position of the electronic device 101 using at least one of a GNSS communication module, a short-range wireless communication module, and a cellular communication module of the communication module 190, and the detected position is located It may be determined whether the location of the electronic device 101 is a store or a market by using whether it matches the store or market stored in the information database.

The electronic device 101 may determine whether the location of the electronic device 101 is related to a predetermined location (eg, a store or market) through the communication module 190. For example, the electronic device 101 detects the position of the electronic device 101 using at least one of a GNSS communication module, a short-range wireless communication module, and a cellular communication module of the communication module 190, and the detected position is located It is possible to determine whether the location of the electronic device 101 is a store or a market by using whether it matches a predetermined location (eg, a store or market) stored in the information database.

According to various embodiments of the present disclosure, in operation 303, if the location of the electronic device 101 is determined to be a store or a market under the control of the processor 120, the electronic device 101 may branch to operation 305.

According to various embodiments of the present disclosure, in operation 303, if it is determined that the location of the electronic device 101 is related to a predetermined location (eg, a store or market) under the control of the processor 120, the operation branches to operation 305. can do.

According to various embodiments of the present disclosure, when it is determined that the location of the electronic device 101 is a store or a market, the electronic device 101 may control the camera module 180 of the electronic device 101 under the control of the processor 120 in operation 305. It can be driven. When the camera module 180 is driven, an environment outside the electronic device 101 may be photographed. The electronic device 101 may analyze an image of the photographed external environment under the control of the processor 120.

According to various embodiments of the present disclosure, if it is determined that the location of the electronic device 101 is related to a predetermined location (eg, a store or a market), the electronic device 101, in operation 305, under the control of the processor 120, the electronic device The camera module 180 of 101 can be driven. When the camera module 180 is driven, an environment outside the electronic device 101 may be photographed. The electronic device 101 may analyze an image of the photographed external environment under the control of the processor 120.

In various embodiments, the electronic device 101 may determine, in operation 307, whether an operation of selecting a product by a user in a store or market is detected under the control of the processor 120.

In various embodiments, the electronic device 101 may determine, in operation 307, whether an operation of selecting a product by a user at a predetermined location (eg, a store or a market) under the control of the processor 120 is detected.

The electronic device 101 may drive the camera module 180 to photograph and analyze an image of an external environment. When the camera module 180 is driven, the electronic device 101 photographs a specific product for a certain period of time, or if a user gesture to pick up a specific product is detected, in operation 307, in a store or market under the control of the processor 120 It may be determined that the operation of the user selecting the product has been detected.

The electronic device 101 may drive the camera module 180 to photograph and analyze an image of an external environment. When the camera module 180 is driven, the electronic device 101 photographs a specific product for a certain period of time, or if a user gesture to pick up a specific product is detected, in operation 307, a predetermined location under the control of the processor 120 ( For example, it may be determined that a user's operation of selecting a product in a store or market) has been detected.

In various embodiments, when the camera module 180 is driven, the electronic device 101 photographs a specific product for a certain period of time, or if a user gesture to pick up the specific product is detected, in operation 307, the processor 120 is controlled In response, it is determined that an operation of selecting a product by a user in a store or market is detected, and the operation may branch to operation 309.

In various embodiments, when the camera module 180 is driven, the electronic device 101 photographs a specific product for a certain period of time, or if a user gesture to pick up the specific product is detected, in operation 307, the processor 120 is controlled In step 309, it is determined that an operation of selecting a product by a user at a predetermined location (eg, a store or a market) has been detected.

In various embodiments, the electronic device 101 may branch to operation 305 when the operation of selecting a product in the store or market is not detected by the processor 120 under the control of the processor 120 in operation 307.

In various embodiments, the electronic device 101 may branch to operation 305 if the user selects a product at a predetermined location (eg, a store or market) is not detected in operation 307 under the control of the processor 120. have.

When an operation of selecting a product by a user in a store or market is detected, the electronic device 101 may acquire an image of the selected product under control of the processor 120 in operation 309. In operation 309, the electronic device 101, under the control of the processor 120, acquires an image of the selected product, for example, an operation of obtaining external information of the selected product, text information written on the product, and image information Can.

In various embodiments, when an operation of selecting a product by a user at a predetermined location (eg, a store or a market) is detected, the electronic device 101 acquires an image of the selected product under control of the processor 120 in operation 309. can do. In operation 309, the electronic device 101, under the control of the processor 120, acquires an image of the selected product, for example, an operation of obtaining external information of the selected product, text information written on the product, and image information Can.

In various embodiments, in operation 311, the electronic device 101 may check the components of the selected product using the acquired image and the product component information database under the control of the processor 120. The electronic device 101 may specify the name of the product, etc. by using the appearance information of the selected product included in the acquired image, text information displayed on the product, image information, and the like. The electronic device 101 may search the product component information database using the specified product name and check the selected product component. The product component information database may be stored in the memory 130 of the electronic device 101 or may be information stored in a storage device of the external device (eg, the server 108). The product ingredient information database may be information on ingredients for each product previously established by a private organization related to food and drug safety or a governmental organization related to food and drug safety. The product component information may be component information regarding at least one of a nutritional component, a constituent component, and a pharmaceutical component. In various embodiments, in operation 313, under the control of the processor 120, the electronic device 101 may check a component of the selected product and determine whether the identified component has an effect on user health.

In various embodiments, the electronic device 101, in operation 313, under the control of the processor 120, checks the components of the selected product and compares the identified components with previously collected or received user health information to affect user health. Can determine whether there is.

In operation 313, the electronic device 101, under the control of the processor 120, checks a component of the selected product and determines whether the identified component has an effect on user health, for example, based on user health information Therefore, it may be an operation of determining whether a product contains a component to be avoided by the user. For example, the electronic device 101 may check health information of an allergic user, and determine that a product containing an allergen-causing component may affect user health. If it is determined that the user's religion is Islam using the information included in the user's health information, the electronic device 101 may determine that the user's health may be affected for the product containing the beef component. The electronic device 101 may determine that a user's health may be affected for a product containing a high-calorie component by using the user's weight and health management information among information included in the user's health information.

In various embodiments, the electronic device 101, in operation 313, under the control of the processor 120, confirming a component of the selected product and determining whether the identified component has an effect on user health, augmented reality to the user It may be an operation of displaying or guiding at least one of the ingredients and warning phrases of the selected product using (AR, AUGMENTED REALITY) on the display device 160.

In operation 313, the electronic device 101 may check the component of the selected product under the control of the processor 120, and if it is determined that the identified component affects the user's health, branch to operation 315.

In operation 315, the electronic device 101 may search for an alternative product based on the user's intention and user health information under the control of the processor 120.

In operation 315, the electronic device 101 determines, under the control of the processor 120, an operation of determining a user's intention by using a preferred product or the like using a pre-stored product purchase list.

In operation 317, the electronic device 101 may determine whether the searched alternative product is in a store or market, under the control of the processor 120.

In various embodiments, in operation 317, the electronic device 101 may determine whether the searched alternative product is in a predetermined location (eg, a store or market) under the control of the processor 120.

In operation 317, the electronic device 101, under the control of the processor 120, determines whether the searched alternative product is in the store or market, searches the database regarding the product stock and product location of the store or market. You can judge. A database of product stock and product location of a store or market may be stored in advance in an external device (eg, server 108).

In operation 317, the electronic device 101, under the control of the processor 120, uses the database of the product stock and product location of the store or market, and the product inventory of the store or market corresponding to the searched alternative product Can determine whether there is. In operation 317, the electronic device 101 receives, through the communication module 190, a database regarding the product stock and product location of the store or market under the control of the processor 120 and receives the product inventory of the received store or market. It is possible to determine whether there is a product inventory of a store or market corresponding to the searched alternative product, by using a database about the stock and product location.

In various embodiments, in operation 317, the operation of determining whether the searched alternative product is in a predetermined location (eg, a store or a market) under the control of the processor 120 is determined in operation 317 (eg, in a predetermined location). It can be determined by searching the database on the product stock and product location of the store or market). The database regarding product stock and product location of a predetermined location (eg, a store or market) may be stored in advance in an external device (eg, server 108). In operation 317, the electronic device 101, under the control of the processor 120, uses a database of product stock and product location at a predetermined location (eg, a store or market) to correspond to the searched alternative product. It is possible to determine whether there is a product inventory of a store or market. In operation 317, the electronic device 101 receives and receives, through the communication module 190, a database regarding product stock and product location of a predetermined location (eg, a store or market) under the control of the processor 120. It is possible to determine whether there is a product inventory of a store or market corresponding to the searched alternative product using a database of the product stock and the product location of the predetermined location (eg, a store or market).

In operation 317, the electronic device 101 may branch to operation 319 if it is determined that the searched alternative product is in a store or market, under the control of the processor 120.

In various embodiments, the electronic device 101 may branch to operation 319 when it is determined in step 317 that the searched alternative product is in a predetermined location (eg, a store or market) under the control of the processor 120.

In operation 317, the electronic device 101 may branch to operation 321 if the searched alternative product is not in the store or market under the control of the processor 120.

In various embodiments, in operation 317, the electronic device 101 may branch to operation 321 if the searched alternative product is not in a predetermined location (eg, a store or market) under the control of the processor 120.

If it is determined that the searched alternative product is in a store or market, the electronic device 101 provides information on the searched alternative product using AR (Augmented Reality) under the control of the processor 120 in operation 319. can do.

In various embodiments, if it is determined that the searched alternative product is in a predetermined location (eg, a store or a market), the electronic device 101, in operation 319, under the control of the processor 120, augmented reality (AR, AUGMENTED REALITY) Use to provide information about the searched alternative products.

In operation 319, the electronic device 101, under the control of the processor 120, provides information regarding an alternative product searched for using augmented reality (AR), alternatively, for example, in a store or market It may be an operation of displaying or guiding a location where a product is located on the display device 160 using augmented reality.

In various embodiments, in operation 319, the operation of providing information regarding an alternative product searched for using augmented reality (AR) under the control of the processor 120, in operation 319, for example, It may be an operation of displaying or guiding the location of the alternative product on the display device 160 using augmented reality in a location where an alternative product is located at a predetermined location (eg, a store or market).

If it is determined that the searched alternative product is not in the store or market, the electronic device 101 purchases an online product related to the searched alternative product using augmented reality (AR) under control of the processor 120 in operation 321. Information may be provided on the display device 160.

In various embodiments, if it is determined that the searched alternative product is not in a predetermined location (eg, a store or market), the electronic device 101, in operation 321, under the control of the processor 120, AR (Augmented Reality) Online product purchase information regarding the searched alternative product may be provided on the display device 160.

In operation 321, the electronic device 101, under the control of the processor 120, provides online product purchase information regarding an alternative product searched for using augmented reality (AR), for example, It may be an operation of displaying or guiding at least one of information and payment information about an online store selling alternative products on the display device 160 using augmented reality (AR).

4 is a diagram illustrating a method of providing information on alternative products searched by the electronic device 101 according to various embodiments of the present disclosure.

In 401, the electronic device 101 photographs a specific product 420 for a certain period of time, or if a user 410 gesture to pick up the specific product 420 is detected, the user in the store or market under the processor 120 control ( 410) may determine that the operation of selecting the product has been detected.

In various embodiments, in 401, when the electronic device 101 photographs a specific product 420 for a certain time or when a gesture of the user 410 picking up the specific product 420 is detected, the electronic device 101 is previously controlled under the processor 120. It may be determined that the operation of the user 410 selecting a product at a designated location (eg, a store or market) has been detected.

In 403, the electronic device 101 may check the components of the selected product 420 and determine whether the identified components affect the health of the user 410. If it is determined that the identified component affects the health of the user 410, the electronic device 101 displays the image 430 specifying the selected product 420 to the user 410 using augmented reality, or the selected product At least one of the components 431 and warning phrases 431 of 420 may be displayed or guided on the display device 160. The image 430 specifying the selected product 420 may have a window shape, and the component 431 and the warning phrase 431 of the selected product 420 may have a message bubble shape.

At 405 and 407, the electronic device 101 checks the components of the selected product 420, determines that the identified component has an effect on the health of the user 410, and determines that an alternative product 460 exists in the store or market If it is, the location of the alternative product 460 in the store or market may be displayed or guided on the display device 160 using the navigation information 450 and the information 440 on the alternative product on augmented reality. When the user 410 reaches the location of the alternative product 460 in the store or market according to the navigation information 450, the electronic device 101 displays the image 470 specifying the alternative product 460 ( 160) can be displayed or guided.

In various embodiments, at 405 and 407, the electronic device 101 checks the components of the selected product 420, determines that the identified components have an effect on the health of the user 410, and the alternative product 460 is previously designated. If it is determined that the location (eg, a store or a market) exists, the location of the alternative product 460 in the store or market is displayed using navigation information 450 and information about the alternative product 440 using augmented reality ( 160) can be displayed or guided. When the user 410 reaches the location where the alternative product 460 is located at a predetermined location (for example, a store or market) according to the navigation information 450, the electronic device 101 images the alternative product 460 The 470 may be displayed or guided on the display device 160.

FIG. 5 is a diagram illustrating a method of providing online product purchase information on an alternative product searched by the electronic device 101 according to various embodiments of the present disclosure.

In 501, the electronic device 101 photographs the specific product 520 for a certain time, or when a user 510 gesture to pick up the specific product 520 is detected, the user in the store or market under the processor 120 control ( 510) may determine that an operation of selecting a product has been detected.

In various embodiments, in 501, when the electronic device 101 photographs a specific product 520 for a certain time, or when a user 510 gesture to pick up the specific product 520 is detected, the electronic device 101 may be previously controlled under the processor 120 control. It may be determined that the operation of the user 510 selecting a product at a designated location (eg, a store or market) has been detected.

In 503, the electronic device 101 may check the components of the selected product 520 and determine whether the identified components affect the health of the user 510. If it is determined that the identified component has an effect on the health of the user 510, the electronic device 101 displays the image 530 specifying the selected product 520 using the augmented reality to the user 510, or the selected product At least one or more of the components 531 and warning phrases 531 of 520 may be displayed or guided on the display device 160. The image 530 specifying the selected product 520 may have a window shape, and the component 531 and the warning phrase 531 of the selected product 520 may have a message bubble shape.

In 505 and 507, the electronic device 101 checks the components of the selected product 520 and determines that the identified components affect the user 510 health, and determines that the alternative product 560 is not in the store or market , Display at least one or more of the information 532 and payment information 533 regarding the online store selling the searched alternative products 560 on the display device 160 using AR (Augmented Reality), or Can guide you.

In various embodiments, at 505 and 507, the electronic device 101 checks the components of the selected product 520 and determines that the identified components have an effect on the user 510 health, and the alternative product 560 is previously designated. If it is determined that it is not in a location (eg, a store or market), at least one or more of information 532 and payment information 533 regarding the online store selling the searched alternative product 560 is augmented reality (AR). It can be displayed or guided on the display device 160 using.

The electronic device according to various embodiments disclosed in the present document may be various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

It should be understood that various embodiments of the document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, and include various modifications, equivalents, or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B," "A, B or C," "at least one of A, B and C," and "A Each of the phrases such as "at least one of,, B, or C" may include any one of the items listed together in the corresponding phrase of the phrases, or all possible combinations thereof. Terms such as “first”, “second”, or “first” or “second” can be used to simply distinguish a component from other components, and to separate components from other aspects (eg, importance or Order). Any (eg, first) component is referred to as “coupled” or “connected” to another (eg, second) component, with or without the term “functionally” or “communically” When mentioned, it means that any of the above components can be connected directly to the other component (eg, by wire), wirelessly, or through a third component.

As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. The module may be an integrally configured component or a minimum unit of the component or a part thereof performing one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may include one or more instructions stored in a storage medium (eg, internal memory 436 or external memory 438) readable by a machine (eg, electronic device 401). It may be implemented as software (e.g., program 440) that includes. For example, a processor (eg, processor 420) of a device (eg, electronic device 401) may call and execute at least one of one or more commands stored from a storage medium. This enables the device to be operated to perform at least one function according to the at least one command called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The storage medium readable by the device may be provided in the form of a non-transitory storage medium. Here,'non-transitory' only means that the storage medium is a tangible device, and does not contain a signal (eg, electromagnetic waves). It does not distinguish between temporary storage cases.

According to one embodiment, a method according to various embodiments disclosed in this document may be provided as being included in a computer program product. Computer program products can be traded between sellers and buyers as products. The computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store ^TM ) or two user devices ( For example, it can be distributed directly (e.g., downloaded or uploaded) between smartphones). In the case of online distribution, at least a portion of the computer program product may be stored at least temporarily on a storage medium readable by a device such as a memory of a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, modules or programs) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components the same or similar to that performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order, or omitted Or, one or more other actions can be added.

Claims (15)

1. In the method of providing product information using augmented reality,

   Collecting or receiving user health information;

   When it is determined that the location of the electronic device is related to a predetermined location, driving the camera module;

   Determining whether a user selects a product through a photographed image while driving the camera module;

   When it is determined that the user selects a product, comparing the selected product information and the product component information database from the selected product image to check the component of the selected product;

   Comparing the components of the selected product and the user health information to determine whether the selected product affects the user health; And

   And if it is determined that the selected product affects the user's health, searching for an alternative product.
2. According to claim 1,

   The operation of determining whether the selected product affects the user's health is

   And using augmented reality to display at least one or more of the selected product components and warning documents on the display device.
3. According to claim 1,

   The user health information

   Characterized in that it is collected through a user input of the electronic device, or is collected using a sensor module of the electronic device, and receiving the user health information is

   And receiving health information of the user or the family of the user from an external device.
4. According to claim 1,

   The operation of determining that the location of the electronic device is related to a predetermined location

   And comparing and determining a database of a location and a predetermined location of the electronic device identified through a communication module.
5. According to claim 1,

   The operation of determining whether a user has selected a product through the photographed image

   The method further includes determining that the user has selected the product when a user gesture for picking up the specific product is detected or a specific product is photographed for a predetermined time.
6. According to claim 1,

   Determining whether the searched alternative product exists at a predetermined location by comparing information on the searched alternative product and a database of product inventory and product location at a predetermined location; And

   If it is determined that the searched alternative product exists at a predetermined location, providing information about the searched alternative product using augmented reality information; And

   And when it is determined that the searched alternative product does not exist at a predetermined location, using the augmented reality information to provide online product purchase information about the searched alternative product.
7. The method of claim 6,

   Using the augmented reality information to provide information about the searched alternative product is

   Displaying or guiding navigation information on a location with an alternative product at a predetermined location on a display device using augmented reality; And

   The method further includes displaying or guiding at least one of information and payment information related to an online store selling the searched alternative product on a display device.
8. In the electronic device including a method for providing product information using augmented reality,

   Sensor module;

   Communication module;

   Display device;

   Camera module; And

   Includes a processor,

   The processor

   Collecting or receiving user health information, and when it is determined that the location of the electronic device is related to a predetermined location, drives the camera module, and determines whether a user selects a product through an image captured while driving the camera module And when it is determined that the user has selected the product, compares the selected product information and the product component information database from the image of the selected product to check the component of the selected product, and the component of the selected product and the user health information. An electronic device that compares and determines whether the selected product affects the user's health, and searches for an alternative product if it is determined that the selected product affects the user's health.
9. The method of claim 8,

   The processor

   An electronic device configured to display on the display device at least one of a component and a warning document of the selected product using augmented reality.
10. The method of claim 8,

    The user health information

    An electronic device, which is collected through a user input of the electronic device or is collected using the sensor module.
11. The method of claim 8,

    The processor

    An electronic device that determines a location of the electronic device by comparing a database of the electronic device location and a predetermined location identified through the communication module.
12. The method of claim 8,

    The processor

    An electronic device configured to determine that a user has selected a product when a user gesture to pick up the specific product is detected or a specific product is photographed for a predetermined time using the camera module.
13. The method of claim 8,

    The processor

    Compares the information on the searched alternative product with a database of product inventory and product location at a predetermined location to determine whether the searched alternative product exists at a predetermined location,

    If it is determined that the searched alternative product exists at a predetermined location, it is set to provide information about the searched alternative product using augmented reality information,

    If it is determined that the searched alternative product does not exist at a predetermined location, an electronic device configured to provide online product purchase information for the searched alternative product using the augmented reality information.
14. The method of claim 13,

    The processor

    It is set to display or guide navigation information on a location with an alternative product at a predetermined location on the display device using augmented reality,

    An electronic device configured to display or guide at least one of information and payment information related to an online store selling the searched alternative product on the display device.
15. The method of claim 8,

    The processor

    An electronic device configured to receive health information of the user or the family of the user from an external device through the communication module.

Images