TWI712938B - Auxiliary teaching method for product installation and portable electronic device - Google Patents

Abstract

An auxiliary teaching method for product installation is implemented by a portable electronic device that stores a first feature template and a virtual element. The first feature template is related to an installation part formed on a product body Appearance, the virtual element is related to an accessory suitable for installation in the installation part of the product body. The auxiliary teaching method for product installation includes: (A) continuous shooting to generate an original image; (B) when it is determined that the current original image has a first feature part that matches the first feature template, at least according to the The original image and the virtual element generate and display an augmented reality image, the augmented reality image presents the original image and the virtual element, and the virtual element and the first characteristic part are in position to match each other.

Description

Auxiliary teaching method for product installation and portable electronic device

The invention relates to an auxiliary teaching method for product installation, in particular to an auxiliary teaching method for product installation involving a portable electronic device. The invention also relates to a portable electronic device capable of implementing the auxiliary teaching method for product installation

Many products on the market have other accessories besides the product body. After purchasing the product, users need to install the accessories on the product body before they can be used normally.

In the prior art, in order to assist the user in installing accessories, a paper manual is usually attached to the product package for the user's reference. However, attaching a paper manual to each set of products will not only incur additional costs, but also does not conform to the current society's environmental awareness of saving resources. Therefore, the existing technology still has room for improvement.

Therefore, one of the objectives of the present invention is to provide an auxiliary teaching method for product installation that can improve the disadvantages of the prior art.

Therefore, the auxiliary teaching method for product installation of the present invention is implemented by a portable electronic device that includes a storage unit, an image capturing unit, a display unit, and a processing unit, and the storage unit stores a first feature Template and a virtual element corresponding to the first feature template, the first feature template being related to the appearance of a mounting part formed on a product body, and the virtual element being related to a mounting part suitable for being arranged on the product body The accessory; the auxiliary teaching method for product installation includes: (A) the processing unit controls the image capturing unit to continuously shoot to generate an original image; (B) when the processing unit determines that the current original image has a When the first feature part of the first feature template is the first feature part, the processing unit generates an augmented reality image at least according to the original image and the virtual element, and controls the display unit to display the augmented reality image, wherein the augmented reality image The environment image presents the original image and the virtual element, and in the augmented reality image, the virtual element and the first characteristic part match each other in position.

In some implementation aspects of the auxiliary teaching method for product installation of the present invention, in step (A), the processing unit also controls the display unit to display the original image.

In some implementation aspects of the product installation assistant teaching method of the present invention, the first feature template is a three-dimensional point cloud data indicating the appearance of the installation part, and the virtual element includes a virtual image and a virtual image related to the accessory. / Or an installation prompt message.

In some implementation aspects of the product installation assistant teaching method of the present invention, the storage unit further stores a second feature template, the second feature template being related to the appearance of the accessory and the installation part of the product body; the product installation The auxiliary teaching method also includes after step (A): (C) when the processing unit determines that the current original image has a second feature part that matches the second feature template, the processing unit controls the display unit An installation complete message is displayed.

In some implementation aspects of the auxiliary teaching method for product installation of the present invention, in step (B), when the processing unit determines that the current original image has the first characteristic part, the processing unit first performs The position of a feature part in the original image calculates the display coordinates corresponding to the virtual element, and then generates the augmented reality image based on the original image, the virtual element, and the display coordinate, wherein, in the augmented reality In the image, the virtual element is displayed at the position indicated by the display coordinates.

Another object of the present invention is to provide a portable electronic device capable of implementing the auxiliary teaching method for product installation.

The portable electronic device of the present invention includes a storage unit, an image capturing unit, a display unit, and a processing unit electrically connected to the storage unit, the image capturing unit, and the display unit. The storage unit stores a first feature template and a virtual element corresponding to the first feature template. The first feature template is related to the appearance of a mounting portion formed on a product body, and the virtual element is related to a The accessories installed in the installation part of the product body. The processing unit controls the image capturing unit to perform continuous shooting to generate an original image, and when the processing unit determines that the current original image has a first characteristic part that matches the first characteristic template, the processing unit at least The original image and the virtual element generate an augmented reality image, and the display unit is controlled to display the augmented reality image, wherein the augmented reality image presents the original image and the virtual element, and in the expanded reality image In the augmented reality image, the virtual element and the first characteristic part are matched in position with each other.

In some embodiments of the portable electronic device of the present invention, when the processing unit controls the image capturing unit to generate the original image, it also controls the display unit to display the original image.

In some embodiments of the portable electronic device of the present invention, the first feature template is a three-dimensional point cloud data indicating the appearance of the mounting part, and the virtual element includes a virtual image and a virtual image related to the accessory. / Or an installation prompt message.

In some embodiments of the portable electronic device of the present invention, the storage unit further stores a second feature template, the second feature template is related to the appearance of the accessory and the installation part of the product body, when the processing When the unit determines that the current original image has a second characteristic part that matches the second characteristic template, the processing unit controls the display unit to display an installation completion message.

In some embodiments of the portable electronic device of the present invention, when the processing unit determines that the current original image has the first characteristic part, the processing unit first performs the first characteristic part on the original image according to the first characteristic part. Calculate the display coordinates corresponding to the virtual element, and then generate the augmented reality image based on the original image, the virtual element, and the display coordinates, where, in the augmented reality image, the virtual element is It is displayed at the position indicated by the display coordinates.

The effect of the present invention is that the portable electronic device can generate and display the augmented reality image when it is determined that the original image has the first characteristic part, thereby guiding the user to install the accessory to the product body The upper installation part can completely overcome the shortcomings of the prior art.

Before the present invention is described in detail, it should be noted that the "electrical connection" mentioned in this patent specification generally refers to the "direct electrical connection" directly formed between two electronic devices/devices/components, and two electronic devices/devices / "Indirect electrical connection" between components through other electronic equipment/devices/components.

Referring to FIG. 1, an embodiment of the portable electronic device 1 of the present invention is suitable for implementing an auxiliary teaching method for product installation, and thereby assists the user to install an accessory to an installation portion formed on a product body.

For ease of understanding, in this embodiment, the product body takes a dehumidifier as an example, and the mounting portion is, for example, a mounting groove formed on the housing of the dehumidifier. In addition, the accessory is, for example, a wireless communication module that matches the shape of the installation slot. Specifically, the wireless communication module is suitable for being installed in the installation slot and electrically connected to the control circuit inside the dehumidifier, In this way, the dehumidifier equipped with the wireless communication module can receive the wireless control commands issued by the portable electronic device 1 or other remote control devices.

It is added that in other embodiments, the product body is not limited to a dehumidifier, but can also be other types of household appliances (such as air conditioners, refrigerators, air purifiers, etc.), or various types that need to be assembled manually Supplies or furniture. Moreover, the accessory is not limited to a wireless communication module, but can also be a functional electronic module used to cooperate with different household appliances, or a hardware component used to cooperate with various appliances or furniture, instead of using the original The embodiment is limited.

The portable electronic device 1 includes a storage unit 11, an image capture unit 12, a display unit 13, an input unit 14, and an electrical connection to the storage unit 11, the image capture unit 12, the display unit 13, and the The processing unit 15 of the input unit 14, wherein the image capturing unit 12 is implemented as a camera module, for example, and the display unit 13 and the input unit 14 are implemented as a touch screen, for example, but not so Is limited.

The storage unit 11 is implemented as a flash memory, for example, and stores a first feature template D1, a second feature template D2, and a virtual element D3 corresponding to the first feature template D1.

In this embodiment, the first characteristic template D1 indicates the appearance of the mounting part, and the second characteristic template D2 indicates the appearance of the assembly of the accessory and the mounting part of the product body. In more detail, The first feature template D1 and the second feature template D2 are respectively implemented as two pieces of three-dimensional point cloud data in this embodiment, and each includes a plurality of three-dimensional coordinates, but it is not limited thereto.

With reference to Figure 3, the virtual element D3 in this embodiment includes an installation prompt message e1 expressed in text, and a virtual image e2 showing the appearance of the accessory. The installation prompt message e1 can be used in this embodiment. For example, it indicates "insert the module plug from here", and the virtual image e2 is implemented as a three-dimensional virtual image in this embodiment, for example. However, in other embodiments, the virtual element D3 may also include only one of the installation prompt message e1 and the virtual image e2, and the virtual image e2 may also be implemented as a two-dimensional virtual image , And not limited to this embodiment.

It is added that, in this embodiment, the portable electronic device 1 obtains the first feature template D1 and the first feature template D1 by installing and executing an application software issued by the developer of the dehumidifier, for example. The second feature template D2 and the virtual element D3, but not limited to this.

Referring to FIG. 1, FIG. 2 and FIG. 3 at the same time, how the portable electronic device 1 of this embodiment implements the auxiliary teaching method of product installation will be described in detail below by way of example.

First, when the processing unit 15 of the portable electronic device 1 receives an auxiliary teaching opening instruction generated by the input unit 14 according to the user's operation, step S1 is performed.

In step S1, the processing unit 15 controls the image capturing unit 12 to perform continuous shooting to generate an original image, and controls the display unit 13 to display the original image and a shooting position prompt message. Specifically, the original image presents the content currently captured by the image capturing unit 12, and the shooting position prompt message indicates, for example, "please aim the lens at the installation slot on the back of the dehumidifier" to guide the user The image capturing unit 12 is aligned with the installation part of the product body (that is, the installation groove of the dehumidifier). Next, proceed to step S2.

In step S2, the processing unit 15 determines whether the current original image has a first feature part that meets the first feature template D1, or has a second feature part that meets the second feature template D2. If the judgment result of the processing unit 15 is no, step S2 is executed again after a predetermined period of time. On the other hand, if the processing unit 15 determines that the current original image has the first characteristic part, then step S3 is performed. On the other hand, if the processing unit 15 determines that the current original image has the second characteristic part, then step S5 is performed next.

In step S3 following step S2, once the processing unit 15 determines that the current original image has the first characteristic part, it means that the image capturing unit 12 has been aligned and captured the installation without the accessory unit. At this time, the processing unit 15 calculates a display coordinate corresponding to the virtual element D3 according to the position of the first characteristic part in the original image, and the display coordinate indicates a display coordinate relative to the first characteristic part in the original image The display position in the position. In this embodiment, the display position is, for example, the same as the position of the first characteristic part in the original image. However, in other embodiments, the display position may also be, for example, the first characteristic part in the original image. The position in is deviated by a predetermined distance and is not limited to this embodiment. After the processing unit 15 calculates the display coordinates, step S4 follows.

In step S4, the processing unit 15 generates an augmented reality image i as shown in FIG. 3 according to the original image, the virtual element D3 and the display coordinates, and controls the display unit 13 to display the augmented reality image i. In more detail, the augmented reality image i simultaneously presents the original image and the virtual element D3 (that is, the installation prompt message e1 and the virtual image e2), and the virtual element D3 is displayed in the augmented reality image. The environment image i is displayed at the display position indicated by the display coordinates, and matches the position of the first characteristic part presented in the augmented reality image i.

In particular, if the relative positional relationship between the image capturing unit 12 and the mounting part of the product body changes, which causes the position of the first characteristic part in the augmented reality image i to move, the processing unit 15 will also update the display coordinates in real time, so that the position of the virtual element D3 in the augmented reality image i always matches with the first feature part, thereby effectively guiding the user to the accessory (In this embodiment, the wireless communication module is taken as an example) to be installed on the installation part of the product body (in this embodiment, the installation slot of the dehumidifier is taken as an example).

In addition, since the virtual image e2 contained in the virtual element D3 is a three-dimensional virtual image in this embodiment, in the augmented reality image i, the display angle of the three-dimensional virtual image is, for example, the same as that of the image The angle at which the photographing unit 12 photographs the mounting part matches. For example, suppose the processing unit 15 determines from the original image that the image capturing unit 12 is capturing the mounting portion from the upper right of the mounting portion, then the processing unit 15 will control the display unit 13 in this step The viewing angle from the upper right to the lower left displays the three-dimensional virtual image e2, thereby further enhancing the effect of guiding the user, but not limited to this.

In step S5 following step S2, once the processing unit 15 determines that the current original image has the second characteristic part, it means that the accessory (in this embodiment, the wireless communication module is taken as an example) has been It is correctly installed on the installation part of the product body (in this embodiment, the installation slot of the dehumidifier is taken as an example). At this time, the processing unit 15 controls the display unit 13 to display an installation completion message, the installation completion message For example, it can indicate "the wireless communication module has been installed", but it is not limited to this.

In summary, the portable electronic device 1 of the present embodiment can generate and display the augmented reality image when it is determined that the original image has the first characteristic part by implementing the product installation assistant teaching method i, to guide the user to install the accessory (in this embodiment, the wireless communication module is taken as an example) to the installation part on the product body (in this embodiment, the installation groove of the dehumidifier is taken as an example) Therefore, the portable electronic device 1 of this embodiment can completely overcome the shortcomings of the prior art, and indeed achieve the purpose of the invention.

However, the above are only examples of the present invention. When the scope of implementation of the present invention cannot be limited by this, all simple equivalent changes and modifications made in accordance with the scope of the patent application of the present invention and the content of the patent specification still belong to This invention patent covers the scope.

1: Portable electronic device 11: storage unit 12: Image capture unit 13: display unit 14: Input unit 15: processing unit D1: The first feature template D2: Second feature template D3: virtual element e1: Installation prompt message e2: virtual image i: Augmented reality images S1~S5: steps

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, in which: FIG. 1 is a block diagram of an embodiment of the portable electronic device of the present invention; Figure 2 is a flow chart illustrating how the embodiment implements a product installation assistant teaching method; and FIG. 3 is a schematic diagram exemplarily showing that this embodiment displays an augmented reality image.

S1~S5: steps

Claims (10)

An auxiliary teaching method for product installation is implemented by a portable electronic device. The portable electronic device includes a storage unit, an image capturing unit, a display unit, and a processing unit. The storage unit stores a first feature template and A virtual element corresponding to the first feature template, the first feature template indicating the appearance of a mounting portion formed on a product body, and the virtual element is related to a mounting portion suitable for being set on the product body The accessory includes a three-dimensional virtual image showing the appearance of the accessory; the auxiliary teaching method for product installation includes: (A) the processing unit controls the image capturing unit to perform continuous shooting to generate an original image; and (B) When the processing unit determines that the current original image has a first characteristic part that matches the first characteristic template, the processing unit generates an augmented reality image at least according to the original image and the virtual element, and controls the The display unit displays the augmented reality image, wherein the augmented reality image presents the original image and the virtual element, and in the augmented reality image, the three-dimensional virtual image and the first feature part are mutually The positions are matched, and the display angle of the three-dimensional virtual image is matched with the angle at which the image capturing unit shoots the mounting portion. The auxiliary teaching method for product installation according to claim 1, wherein, in step (A), the processing unit also controls the display unit to display the original image. The auxiliary teaching method for product installation according to claim 1, wherein the first feature template is a piece of three-dimensional point cloud data indicating the appearance of the installation part. The auxiliary teaching method for product installation according to claim 1, wherein the storage unit further stores a second feature template, the second feature template being related to the appearance of the accessory and the installation part of the product body; the product installation The auxiliary teaching method also includes after step (A): (C) when the processing unit determines that the current original image has a second feature part that matches the second feature template, the processing unit controls the display unit An installation complete message is displayed. The product installation assistant teaching method according to claim 1, wherein, in step (B), when the processing unit determines that the current original image has the first characteristic part, the processing unit first The position of a feature part in the original image calculates the display coordinates corresponding to the virtual element, and then generates the augmented reality image based on the original image, the virtual element, and the display coordinate, wherein, in the augmented reality In the image, the virtual element is displayed at the position indicated by the display coordinates. A portable electronic device comprising: a storage unit storing a first characteristic template and a virtual element corresponding to the first characteristic template, the first characteristic template indicating an installation part formed on a product body The virtual element is related to an accessory suitable for the installation part of the product body, and includes a three-dimensional virtual image showing the appearance of the accessory; an image capturing unit; a display unit; and a processing Unit, electrically connected to the storage unit, the image capturing unit and the display unit; Wherein, the processing unit controls the image capturing unit to perform continuous shooting to generate an original image, and when the processing unit determines that the current original image has a first characteristic part that matches the first characteristic template, the processing unit At least generate an augmented reality image based on the original image and the virtual element, and control the display unit to display the augmented reality image, wherein the augmented reality image presents the original image and the virtual element, and In the augmented reality image, the position of the three-dimensional virtual image and the first characteristic part are matched with each other, and the display angle of the three-dimensional virtual image is matched with the angle at which the image capturing unit captures the mounting portion. The portable electronic device according to claim 6, wherein when the processing unit controls the image capturing unit to generate the original image, it also controls the display unit to display the original image. The portable electronic device according to claim 6, wherein the first feature template is a piece of three-dimensional point cloud data indicating the appearance of the mounting part. The portable electronic device according to claim 6, wherein the storage unit further stores a second characteristic template, and the second characteristic template is related to the appearance of the accessory and the installation part of the product body when the processing When the unit determines that the current original image has a second characteristic part that matches the second characteristic template, the processing unit controls the display unit to display an installation completion message. The portable electronic device according to claim 6, wherein, when the processing unit determines that the current original image has the first characteristic part, the processing unit first applies the first characteristic part to the original image Calculate the display coordinates of the corresponding virtual element based on the position in the original image The image, the virtual element, and the display coordinates generate the augmented reality image, where in the augmented reality image, the virtual element is displayed at the position indicated by the display coordinates.

Images