JP2014032589A - Electronic device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which conventionally, a code carried in, for example, a catalogue, a flyer and the like has to be photographed and when there is no code, a simulation of an arrangement state of an object cannot be performed.SOLUTION: The electronic device comprises: an imaging section that photographs a first object; an input section that inputs first information relating to a size of the first object and second information relating to a size of a second object instead of the first object or used together with the first object; and a display section that displays the first object photographed by the imaging section and performs a display relating to the second information on the basis of the first and second information.

Description

  The present invention relates to an electronic device.

Patent Document 1 describes an arrangement simulation system for simulating an object arrangement state.
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2008-191751

  However, in the above prior art, for example, a code posted in a catalog, a flyer, etc. must be photographed and a special scale must be placed in the real space. State simulation could not be performed.

  In an aspect of the present invention, an imaging unit that captures an image of the first object, first information about the size of the first object, and a second object used in place of the first object or together with the first object. An input unit that inputs second information about the size, and a display unit that displays the first object imaged by the imaging unit and displays the second information based on the first and second information And an electronic apparatus characterized by comprising:

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 shows a configuration of an electronic apparatus 10 according to the present embodiment. The operation | movement flow of the electronic device 10 which concerns on this embodiment is shown. The operation | movement flow of the electronic device 10 which concerns on this embodiment is shown. The operation | movement flow of the electronic device 10 which concerns on this embodiment is shown. An example in which the display unit 700 displays a frame 702 indicating the dimensions of the first object is shown. An example is shown in which a display unit 700 displays a frame 704 indicating the dimension of a second object so as to overlap the image of the first object.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows a configuration of an electronic apparatus 10 according to the present embodiment. The electronic device 10 is, for example, a device such as a smartphone, a camera, or a mobile phone. For example, for a captured image including the first object such as an electric product, a second object that is a replacement product of the first object is used. Information relating to the size or the like, that is, for example, an image indicating the range occupied by the second object is added and displayed. Thereby, the electronic device 10 enables simulation when the second object is placed in the user's room or the like.

  The electronic device 10 includes an imaging unit 100, a CPU 200, a frame memory 300, a wireless communication unit 400, an attitude sensor 500, an image composition unit 600, a display unit 700, an operation unit 800, a resizing unit 900, A memory unit 1000.

  The imaging unit 100 is used for imaging the arrangement of a room including a first object in accordance with a user operation, and includes a lens unit 110 and an imaging element 120 (for example, CCD or CMOS). The lens unit 110 is connected to the CPU 200 and is driven by the imaging control unit 202, and enters light from the subject to form an image on the imaging element 120. The lens unit 110 may be fixed to or removable from the imaging unit 100. The lens unit 110 includes an AF lens 112 and a diaphragm 114.

  The AF lens 112 is driven by a drive control signal from the imaging control unit 202 and changes the subject distance to be focused. The diaphragm 114 is driven by a drive control signal from the imaging control unit 202 and adjusts the amount of subject light incident on the imaging device 120.

  The image sensor 120 images subject light received through the AF lens 112 and the diaphragm 114. The frame memory 300 is a buffer memory connected to the image sensor 120, and sequentially receives and temporarily stores captured image data captured by the image sensor 120, and functions as a temporary memory for image processing and image display. To do.

  The wireless communication unit 400 inputs the first information related to the size of the first object and the second information related to the size of the second object used instead of or together with the first object. Specifically, the wireless communication unit 400, in response to an instruction from the CPU 200, first information on the size or the like of the first object (for example, external dimension information of the first object, the model number of the first object, and the first object Appearance feature information [contour, color, and / or image] may be included) from a server on the network and input to the CPU 200.

  In addition, the wireless communication unit 400 uses a second object (for example, a television stand) used in place of a first object (for example, a currently used television) or a second object (for example, a television stand) that is used together with the first object. Including second information (eg, external dimension information of the second object, model number of the second object, and external feature information [contour, color, and / or image] of the second object) regarding the size of the newly purchased television) May be acquired from a server or the like on the network and supplied to the CPU 200.

  The attitude sensor 500 detects the attitude of the electronic device 10. The posture sensor 500 is an acceleration sensor as an example, and detects the posture of the electronic device 10 from the tilt of the electronic device 10 at the time of shooting. The attitude sensor 500 outputs the detected attitude of the electronic device 10 to the CPU 200 as attitude data.

  The attitude sensor 500 has a vertical tilt around the left and right of the electronic device 10, a horizontal tilt around the vertical of the electronic device 10, and a vertical and horizontal tilt around the optical axis of the image sensor 120. At least one of the inclinations may be detected.

  The CPU 200 is connected to various hardware components inside the electronic device 10 and controls these components. The CPU 200 includes an imaging control unit 202 that controls imaging of a subject by the imaging unit 100 through cooperation of hardware and software. The imaging control unit 202 controls the AF lens 112 and the diaphragm 114 based on the captured image data of the subject input from the imaging element 120 to detect the distance from the imaging unit 100 to the first object that is the subject, and focuses. Adjust. The imaging control unit 202 may detect the distance from the imaging unit 100 to the first object based on a change in contrast of the image captured by the imaging unit 100 (contrast AF method), and instead of this, the phase difference AF method Thus, the distance to the first object may be detected.

  Specifically, in the case of the contrast AF method, the imaging control unit 202 captures the first object while changing the focusing distance of the AF lens 112, and sets the distance with the highest contrast as the subject of the first object. Detect as distance. In the case of the phase difference AF method, the imaging control unit 202 detects the subject distance from the imaging unit 100 to the first object based on the interval between the two images formed on the AF sensor.

  Further, the CPU 200 determines the size of the first object based on the size occupied by the first object in the captured image, the distance information from the imaging unit 100 to the first object, and / or the focal length or the zoom amount of the AF lens 112. First estimated information including information related to the estimated value of height is generated. The CPU 200 accesses a server or the like on the network via the wireless communication unit 400 based on the estimated first information, searches for at least one candidate of the first object to be matched with the estimated first information, and is matched. The first object candidate data including the first information corresponding to each of the at least one first object is input from the server.

  Further, the CPU 200 accesses a server or the like on the network via the wireless communication unit 400 based on the first information of the first object selected from the first object candidate data, and selects the selected first object. At least one candidate for the corresponding second object is searched, and second object candidate data including second information corresponding to each of the matched at least one second object is input.

  As an example, the CPU 200 requests a search from the server using the product type, size, and / or color attribute of the first object (for example, a television currently used), and is suitable as a newly purchased product. Second information including external dimension information, a model number, external characteristic information, and the like of a second object (for example, a newly purchased television) that is a replacement product is acquired via the wireless communication unit 400 and the network. The CPU 200 instructs the image composition unit 600 to display the second information on the second object selected from the second object candidate data.

  The image composition unit 600 receives a display instruction related to the second information from the CPU 200, combines the display related to the second information with the captured image stored in the frame memory 300, and supplies the composite to the display unit 700. For example, when receiving an instruction to display the second information from the CPU 200, the image composition unit 600 synthesizes the image of the second object that is a replacement product with the captured image stored in the frame memory 300. To supply.

  The display unit 700 is a liquid crystal display screen as an example, and displays a composite image based on the composite image data input from the image composition unit 600. Thereby, the display unit 700 displays the first object imaged by the imaging unit 100 and displays the second information based on the first and second information.

  As an example, while the imaging unit 100 is imaging a television as the first object, the display unit 700 combines a captured image data of the television with a composite image obtained by synthesizing a television image that is a replacement product as second information. indicate. With such a configuration, when the electronic device 10 captures an image of the first object, the electronic device 10 can perform an arrangement simulation of the second object to be a replacement product.

  As an example, the operation unit 800 includes a touch panel provided on the surface of the display unit 700, receives a user operation, and inputs it to the CPU 200. As an example of such an operation, the operation unit 800 specifies an instruction to specify or change the first object photographed in the captured image, an instruction to specify one first object from a plurality of candidates for the first object, An instruction to specify a desired size of the second object, an instruction to specify one second object from a plurality of candidates for the second object used together with or instead of the first object, etc. And supplied to the CPU 200.

  When receiving an instruction to manually change the range of the first object detected from the captured image, the resizing unit 900 changes the range of the first object in the captured image in accordance with the instruction.

  When the CPU 200 receives an image capturing instruction through the operation unit 800, the memory unit 1000 receives the first captured image obtained by capturing the first object from the frame memory 300 and stores it. Note that the memory unit 1000 may be a storage medium built in the electronic device 10 or a removable storage medium.

  2 to 4 show an operation flow of the electronic device 10 according to the present embodiment. In this operation flow, the electronic device 10 identifies the current product in response to imaging of the product currently used as the first object (“current product”) according to the user's instruction, and sets it as the second object. Information related to the size of a new product that is to be used in place of the current product is combined with image data of the current product and displayed.

  In step S <b> 1, the imaging unit 100 images a target space including, for example, the current product owned by the user as the first object, in accordance with the user's imaging operation. The imaging unit 100 sends captured image data obtained by imaging a space including the current product to the CPU 200 via the frame memory 300.

  The imaging control unit 202 detects the subject distance of each subject such as the current product included in the captured image from the captured image data stored in the frame memory 300, for example, using the contrast AF method. For the purpose of improving the distance measurement accuracy by the contrast AF method, the imaging control unit 202 increases the depth of field by opening the diaphragm 114 more in imaging of an image for detecting the distance from the electronic device 10 to the subject. You may control to make it shallower.

  Then, the imaging control unit 202 detects the subject distance from the captured image data while changing the subject distance in focus. By such control, the imaging control unit 202 can detect a more accurate distance from the electronic device 10 to the subject. The attitude sensor 500 sends attitude data of the electronic device 10 detected at the time of imaging to the CPU 200.

  In step S <b> 2, the CPU 200 performs image processing such as white balance on the captured image data, detects an object that is the current product from the captured image, and creates detection data for the current product. The detection data of the current product includes, as an example, an estimation range of the current product in the captured image, an estimation category of the current product, and feature information of the current product.

  The CPU 200 extracts feature information of the current product from the captured image data using, for example, a contour detection technique. As an example, the CPU 200 detects the contour of the current product from the distribution of the color and distance in the captured image data, and extracts the color and the shape of the contour as feature information of the current product.

  The CPU 200 detects the estimated category of the current product and the estimated range of the current product based on the feature information of the current product. For example, if the extracted feature information of the current product is, for example, black and rectangular outlines, the CPU 200 detects the estimated category of the current product as TV, and the extracted feature information of the current product is, for example, brown and rectangular outlines. If it is, the estimated category of the current product is detected as “tance”. The CPU 200 detects the estimated category of the current product and the data area of the detected captured image as the estimated range of the current product.

  Further, the CPU 200 specifies the imaging direction from the posture data indicating the tilt of the electronic device 10 at the time of imaging, and determines the estimated category of the current product based on the information on the color, the shape of the contour, and the imaging direction. Good. For example, when the CPU 200 detects information indicating that the white color, the circular outline, and the imaging direction are upward, the CPU 200 detects the estimated category of the current product as a lighting device.

  In step S3, the CPU 200 causes the display unit 700 to display a frame indicating an area determined to be the estimated range of the current product based on the detection data of the current product created in step S2. As an example, as illustrated in FIG. 5, the CPU 200 causes the image composition unit 600 to superimpose the frame 702 on the captured image data and causes the display unit 700 to display an image in which the frame 702 is superimposed on the captured image data.

  In step S4, the CPU 200 inquires of the user whether or not the estimated range of the current product indicated by the frame 702 is correct. The CPU 200 may proceed by assuming that the frame 702 is correct unless an instruction to change the frame 702 is input from the operation unit 800. Instead, the CPU 200 displays a confirmation input pop-up on the display unit 700 to perform the inquiry. Alternatively, the inquiry may be made by controlling the display mode (for example, blinking display) of the frame 702.

  When the answer to No is input in response to the inquiry, that is, when the frame 702 does not correctly display the estimated range of the current product (No in Step S4), the CPU 200 advances the process to Step S5, and from the user. An instruction to change the position and / or range of the frame 702 is input via the operation unit 800 to correct the estimated range of the current product.

  In step S5, the resizing unit 900 corrects the estimated range of the current product displayed as 702 in response to the CPU 200 receiving a change instruction to change the estimated range of the current product. For example, the resizing unit 900 changes the size of the frame 702 by moving the vertex of the frame 702 to a designated position, and corrects the estimated range of the current product.

  Thereafter, the CPU 200 executes the process of step S4 again, and inquires of the user whether the frame 702 correctly displays the estimated range of the current product. When the answer to No is input to the inquiry, that is, when the frame 702 does not correctly display the estimated range of the current product (No in Step S4), the CPU 200 performs the process of Step S5 on the resizing unit 900 again. Let it run.

  On the other hand, when the frame 702 correctly displays the estimated range of the current product (Yes in step S4), the CPU 200 advances the process to step S6. In step S6, the CPU 200 determines the estimated range of the current product as the current product designated range after confirmation by the user.

  The CPU 200 corrects the feature information of the current product and the estimated category of the current product included in the detection data of the current product based on the determined current product designation range. As an example, the CPU 200 corrects the estimated category of the current product from the TV to the chest when the current product designation range is changed from the black and rectangular contour to the brown and rectangular contour, The estimated category of the current product is the category of the current product. Further, the CPU 200 corrects the feature information of the current product from black to brown.

  When the display unit 700 displays the current product, the CPU 200 may specify the specified range as the current product in response to an input specifying the range of the current product from the operation unit 800. . For example, when the user inputs an instruction to specify or change the current product from the operation unit 800 by touching the touch panel to enclose the range of the current product, the CPU 200 may set the range corresponding to the operation as the current product specified range.

  The operation unit 800 includes, for example, an operation button, moves a frame indicating the current product on the display unit 700 in response to the user pressing the move button, and responds to pressing the enter button. Then, the range displayed in the area within the frame may be input as the current product designation range. In response to this, the CPU 200 may determine the feature information of the current product and the category of the current product included in the detection data of the current product from the designated current product designation range.

  In step S <b> 7, the CPU 200 detects the tilt of the electronic device 10 at the time of imaging from the attitude data detected by the attitude sensor 500, and determines whether the captured image data is captured vertically or horizontally. To detect. The CPU 200 specifies the vertical direction and the horizontal direction of the current product designation range based on the detected orientation of the captured image data.

  In step S8, the CPU 200 determines the posture data, the current product designation range, that is, the size occupied by the current product in the captured image, the distance from the imaging unit 100 to the current product, and / or the focal length or zoom amount of the AF lens 112, and the like. Based on this, estimated first information is calculated. The estimated first information includes, for example, the estimated size (eg, height, width, etc.) of the current product, the category of the current product, and feature information of the current product.

  In step S9 (see FIG. 3), for example, the CPU 200 transmits the estimated size of the current product included in the estimated first information to the server on the network via the wireless communication unit 400, and the size is determined by the server. Is searched for current product candidate data, and current product candidate data estimated to correspond to the current product is received. The CPU 200 transmits not only the estimated size of the current product but also the estimated category of the current product and the color included in the feature information of the current product as the estimated first information. The current product candidate data may be searched including the above features.

  The server that has received the estimated first information from the electronic device 10 in step S9, for example, that the category matches the estimated category received from the electronic device 10 from the product database that holds the first information for a plurality of products. The evaluation is based on at least one condition that the size approximates the estimated size and the appearance feature information matches or resembles the appearance feature information received from the electronic device 10, and the evaluation value is at least one higher than the reference. A process of including one information in the current product candidate data is performed. At this time, the server may identify a product having a size within a range of a predetermined ratio (for example, 95% to 105%) with respect to the estimated size as a product having a size approximating the estimated size, A product having a chromaticity in a predetermined range with respect to the appearance feature information received from the electronic device 10 may be specified as a product whose color approximates.

  Instead, the CPU 200 once receives first information on the sizes of a plurality of current products held by a server on the network via the wireless communication unit 400, and receives all the data received on the electronic device 10 side. Current product candidate data may be detected from the inside. In the above, the current product candidate data received by the electronic device 10 is the product name of the current product candidate, the model number of the current product candidate, the product image of the current product candidate, and the size of the current product candidate for at least one current product candidate. At least one of the first information regarding the size (for example, the external dimension information of the current product candidate and the external feature information [contour, color, and / or image] of the current product candidate, etc.) may be included.

  In step S10, the CPU 200 causes the display unit 700 to display a list of received current product candidate data (for example, at least one of the first information regarding the product name, model number, product image, size, etc. of the current product candidate). .

  In step S <b> 11, the CPU 200 inquires of the user whether there is a target product in the list of current product candidate data displayed on the display unit 700. The CPU 200 may display the confirmation input pop-up on the display unit 700 to make the inquiry, or may make the inquiry by controlling the display mode (for example, blinking display) of the list of current product candidate data. If it is input that the current product does not exist in the list of current product candidates in response to the inquiry (No in step S11), the CPU 200 advances the process to step S12 to inform the user of the current product name and / or Enter the model number of the current product.

  On the other hand, when it is input that there is an item corresponding to the current product in the list of current product candidate data in response to the inquiry in step S11 (Yes in step S11), or the current product name and / or the model number of the current product is input. When it is determined (step S12), the CPU 200 advances the process to step S13, and causes the operation unit 800 to select the current product from the list of current product candidate data. Thus, the CPU 200 receives an instruction from the user when the display unit 700 displays the current product, and associates the current product with the first information regarding the size of the current product.

  Further, when a product image of the current product candidate is input from the server or the like as the current product candidate data via the wireless communication unit 400, the CPU 200 is based on the product image of the current product candidate and the image captured by the imaging unit 100. Thus, when the matching degree of the target object is higher than the reference, it may be specified that the product image of the current product candidate matches the product image of the current product.

  For example, the CPU 200 compares the product image data of the current product candidate input through the wireless communication unit 400 with the data of the captured image using the contour detection technique, and the product of the current product candidate having the most common color and contour. The image data is identified as current product data. According to such a process, it is possible to save the user from having to specify the current product data from among a large number of current product candidate data.

  Further, the CPU 200 may cause the display unit 700 to display distance information from the end of the current product to the display end of the display unit 700 in response to determining the current product data. As an example, the CPU 200 calculates distance information from the end of the current product to the display end based on the ratio of the size of the display unit 700 to the size of the current product, and outputs the distance information to the image composition unit 600. The distance information is superimposed on the captured image data, and an image in which the distance information is superimposed on the captured image data is displayed on the display unit 700.

  In step S <b> 14, the CPU 200 allows the user to specify the size of a new product that the user wants to replace on the captured image through the operation unit 800. As an example, the operation unit 800 inputs an area enclosed by a user on the captured image by touching the touch panel as an instruction for a range in which a new product is to be placed. The CPU 200 receives an instruction for a new product range from the operation unit 800, and sets the range corresponding to the operation as a new product designation range desired by the user.

  For example, as illustrated in FIG. 6, the CPU 200 causes the image composition unit 600 and the display unit 700 to display a frame 704 indicating the designated new product designation range on the captured image. In order to distinguish the frame 704 from the frame 702 indicating the size of the current product, the CPU 200 differs from the frame 702 in at least one of the display modes of the frame 704, for example, the line color, line width, and broken line interval. You may display on the display part 700. FIG.

  In step S15, the CPU 200 calculates the expected size of the new product based on the first information related to the size of the current product included in the current product data and the new product designation range. For example, the CPU 200 calculates the expected size of the new product based on the ratio between the size of the current product and the new product designation range from the external dimension information of the current product included in the first information.

  The CPU 200 transmits the expected size of the new product and the current product data to a server on the network via the wireless communication unit 400, and matches the category and / or appearance feature information indicated in the current product data. And searching for at least one new product candidate belonging to a predetermined size range including the size expected for the new product. The wireless communication unit 400 receives new product candidate data retrieved by a server or the like. Here, the new product candidate data is the second information regarding the product name of the new product candidate, the model number of the new product candidate, the product image of the new product candidate, the size of the new product candidate, etc. for at least one new product candidate. (E.g., external dimension information of a new product candidate, and external feature information [contour, color, and / or image] of the new product candidate, etc.) may be included.

  In step S16 (see FIG. 4), the CPU 200 causes the display unit 700 to display a list of received new product candidate data (for example, a list including at least one of the product name, model number, and image of the new product candidate). If no new product candidate data is detected, the CPU 200 may cause the display unit 700 to display a pop-up indicating that no candidate corresponding to the new product has been found.

  In step S <b> 17, the CPU 200 inquires of the user whether there is a new product candidate in the list of new product candidate data displayed on the display unit 700. The CPU 200 may display the confirmation input popup on the display unit 700 to make the inquiry, or may make the inquiry by controlling the display mode (for example, blinking display) of the list of new product candidate data. . When the user inputs an instruction indicating that there is no new product candidate list in response to the inquiry (No in step S17), the CPU 200 advances the process to step S18 and prompts the user. The new product name and / or model number of the new product is input.

  On the other hand, when it is input that the item corresponding to the new product is in the list of new product candidate data in response to the inquiry in step S17 (Yes in step S17), the new product name and / or the new product model number are input. When it is determined (step S18), the CPU 200 advances the process to step S19, and causes the operation unit 800 to select a desired new product from the list of new product candidate data.

  In step S20, the CPU 200 creates image data of a new product for composition using captured image data, posture data, a distance from the imaging unit 100 to the current product, current product data, new product data, and the like. As an example, the CPU 200 uses the outline and color shown as the feature information of the current product extracted from the captured image data, the external dimension information included in the first information of the current product data, and the attitude data, and the feature of the current product. The inclination of the current product with respect to the imaging surface of the captured image data (that is, the inclination of the current product when the imaging unit 100 is viewed from the viewpoint) is detected by comparing the information with the external dimension information.

  The CPU 200 converts the orientation of the image of the new product according to the detected inclination of the current product, and generates an image in which the new product is viewed from the same angle as the current product. The CPU 200 instructs the image composition unit 600 to display the second information, and supplies the generated image of the new product to the image composition unit 600.

  When the image composition unit 600 receives an instruction to display the image of the new product generated in step S <b> 20 as a display related to the second information from the CPU 200, the image composition unit 600 superimposes the image of the new product on the captured image stored in the frame memory 300. Combined and supplied to the display unit 700. In step S <b> 21, the display unit 700 displays the composite image supplied from the image composition unit 600. That is, the display unit 700 performs display regarding the second information according to the detection result of the attitude sensor 500.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  As an example of a form in which the above embodiment is modified, the electronic apparatus 10 may change the composite display according to the difference between the size of the current product and the size of the new product. In order to realize this, in step S20, when the new product used in place of the current product is larger than the current product, the CPU 200 places the new product on the image of the current product in the image captured by the imaging unit 100. When an image of a new product is created to superimpose and display an image showing the appearance of the product, and the new product used in place of the current product is smaller than the current product, the image of the current product in the image captured by the imaging unit 100 You may create the frame which shows the range which a new product occupies.

  In this case, in step S21, when the new product is larger than the current product, the display unit 700 displays the image of the new product over the image of the current product, and when the new product is smaller than the current product. Displays a frame indicating the size of the new product over the image of the current product. Thereby, the electronic device 10 enables an arrangement simulation while comparing the size of a new product to be a replacement product with the size of the current product.

  In addition, the electronic device 10 temporarily stores a captured image of a space including the current product imaged at the user's home or the like in the memory unit 1000, and previously captured when the user images the new product at a store or the like later. The image of the new product currently captured may be superimposed on the captured image of the space including the current product. In order to realize this, for example, when the imaging unit 100 captures an image of a new product, the CPU 200 reads a captured image of a space including the current product from the memory unit 1000.

  The CPU 200 detects the current product in the same category as the image of the new product from the read captured image by the contour detection technique, and further detects the inclination of the current product with respect to the imaging surface as in step S20. The CPU 200 converts the orientation of the image of the new product according to the detected inclination of the current product, and generates an image in which the new product is viewed from the same angle as the current product. The CPU 200 supplies an image synthesizing unit 600 with an instruction for synthesizing the generated image of the new product and the captured image of the space including the current product.

  The display unit 700 displays the composite image supplied from the image composition unit 600. As a result, the electronic device 10 generates the second imaged data and the combined data using the first captured image including the current product that has been captured in advance by the image composition unit 600, so that the user's room It is possible to simulate the placement of new products even in places other than, for example, on the go such as storefronts.

  Further, the electronic device 10 recognizes the third object adjacent to the current product from the captured image including the current product based on the first information related to the size of the current product, and responds to the size of the third object. New product candidates determined in this way may be presented to the user. For example, here, the third object is arranged adjacent to the current product such as a product for installing or storing the current product or a product that supplements the function of the current product, for example, the current product. It is a TV stand on which a TV is mounted.

  In order to realize this, as an example, the electronic device 10 detects the current product from the captured image including the current product in the same manner as Steps S1 to S13 of FIGS. When the CPU 200 detects that the category of the current product is a television based on the outline and color included in the appearance characteristic information of the first information related to the size and the like of the current product, the third object adjacent below the television is detected. Is recognized as a TV stand.

  Further, the CPU 200 detects the size and the like of the third object arranged adjacent to the current product in the captured image based on the first information related to the size and the like of the current product. For example, when the current product is a television, the CPU 200 determines the size of the television stand based on the ratio of the size of the television to the size of the television stand based on the external dimension information of the television included in the first information regarding the size of the current product. Is detected. The electronic device 10 may also detect the size of the third object in the same manner as steps S1 to S13 in FIGS.

  Next, the electronic device 10 is suitable for replacement from the current product based on the current product data including the first information on the size of the current product and the information on the size of the third object, etc. A candidate for a new product suitable for use with the third object is acquired from a server or the like. Specifically, the electronic device 10 transmits information related to the size and the like of the third object in addition to the current product data to the server or the like via the wireless communication unit 400, and the category, size, and / or current product is transmitted. At least one new product candidate that matches the appearance feature information and has a size that matches the size of the third object is retrieved. Then, the electronic device 10 displays an image related to the second object superimposed on the captured image obtained by capturing the first object and the third object.

  Furthermore, the electronic device 10 performs an arrangement simulation in the case of purchasing a second object, which is a new product used together with the first object, in addition to an arrangement simulation in the case where the current product as the first object is replaced with a new product as the second object. May be used. Here, the first object and the second object are two or more products arranged in the vicinity of each other, such as a product and a product that assists the installation of the product or supplements the function of the product. As an example, when the user owns a television stand as an example of the first object, a case of simulating the arrangement of a television set to be newly purchased as an example of the second object can be cited.

  In order to realize this, as an example, the electronic device 10 detects the current product from the captured image including the current product in the same manner as Steps S1 to S13 of FIGS. Then, the CPU 200 detects the category (for example, a TV stand) of the current product based on the outline and color included in the appearance feature information of the first information related to the size and the like of the current product.

  Next, the electronic device 10 transmits the current product data including the first information about the size of the current product to the server, and a candidate for a new product (related product) suitable for use with the current product is transmitted from the server. get. Here, the server receives the current product data, identifies the product category (eg, television) to be used with the current product according to the category of the current product (eg, TV stand), and the size and / or appearance of the current product. Search for at least one new product candidate that matches the feature information. Then, the electronic device 10 displays an image related to the second object superimposed on the captured image obtained by capturing the first object.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  DESCRIPTION OF SYMBOLS 10 Electronic device, 100 Imaging part, 110 Lens part, 112 AF lens, 114 Aperture, 120 Imaging element, 200 CPU, 202 Imaging control part, 300 Frame memory, 400 Wireless communication part, 500 Attitude sensor, 600 Image composition part, 700 Display unit, 702 frame, 704 frame, 800 operation unit, 900 resize unit, 1000 memory unit

Claims (12)

An imaging unit for imaging the first object;
An input unit for inputting first information related to the size of the first object, and second information related to the size of the second object used in place of or in conjunction with the first object;
An electronic apparatus comprising: a display unit that displays the first object imaged by the imaging unit and performs display related to the second information based on the first and second information. An attitude detection sensor for detecting the attitude of the electronic device;
The electronic device according to claim 1, wherein the display unit displays the second information according to a detection result of the posture detection sensor.   The electronic apparatus according to claim 1, further comprising: a designation unit that designates the first object when the display unit displays the first object.   4. The apparatus according to claim 1, further comprising a processing unit that associates the first object with the first information when the display unit displays the first object. 5. The electronic device according to the item. The input unit inputs image data of the first object,
The specific part which specifies the image of the said 1st object based on the image data of the said 1st object and the image which the said imaging part imaged is provided, The any one of Claim 1 to 3 characterized by the above-mentioned. The electronic device as described in.   The electronic device according to claim 1, wherein the display unit outputs distance information from an end portion of the first object to a display end of the display unit.   The electronic apparatus according to claim 1, further comprising a recognition unit that recognizes a third object adjacent to the first object based on the first information.   7. The electron according to claim 1, further comprising: a size detection unit that detects a size of a third object adjacent to the first object based on the first information. machine. A distance detection unit for detecting a distance from the imaging unit to the first object;
The electronic device according to claim 1, wherein the display unit displays information on the second object based on the distance detected by the distance detection unit. The imaging unit includes a diaphragm that adjusts the amount of light incident on the imaging device,
The distance detection unit detects a distance to the first object based on a change in contrast of an image captured by the imaging unit,
The electronic apparatus according to claim 9, further comprising an adjustment unit that opens the diaphragm more when capturing an image for distance detection by the distance detection unit. The display unit
When the second object used in place of the first object is larger than the first object, the image of the first object in the image captured by the imaging unit is overwritten with an image showing the appearance of the second object And display
The range occupied by the second object is displayed together with the image of the first object in the image captured by the imaging unit when the second object is smaller than the first object. The electronic device as described in. A storage unit for storing a first captured image obtained by imaging the first object;
When the imaging unit captures a second captured image including the second object, the display unit displays an image of the second object captured in the second captured image as the first captured image. The electronic apparatus according to any one of claims 1 to 11, wherein the electronic apparatus is displayed in an overlapping manner.

Images