TW201528049A - Correction method based on pattern and electronic apparatus - Google Patents

Abstract

A correction method based on a pattern and an electronic apparatus are provided. A pattern is displayed in a display screen of a display unit, and an image is captured by an image capturing unit which faces the display screen, wherein the image includes a polygon graph corresponding to the pattern. The image is analyzed to obtain a correction information. An operational plane related to the display screen is established based on the correction information, where the size of the operational plane has a proportional relationship with the size of the display screen. And a corresponding relationship of each of coordinates between the operational plane and the display screen is generated according to the proportional relationship.

Description

Pattern-based correction method and electronic device

The present invention relates to an interactive sensing technology, and more particularly to a three-dimensional space-based and pattern-based correction method and electronic device.

Most of the traditional electronic products only have input devices such as a remote controller, a keyboard, a mouse, and the like for the user to operate. With the advancement of technology, more and more development and research are devoted to the improvement of the operation interface. The new generation of operating interfaces is becoming more user-friendly and more convenient. In recent years, conventional input devices for electronic products have gradually been replaced by other input devices, and the replacement of conventional input devices with gesture operations is most popular.

Gesture operations are widely used in a variety of human-computer interaction interfaces, such as robot remote control, electrical remote control, and slide presentation operations. The user can directly manipulate the user interface in a three-dimensional space by gestures without touching an input device such as a keyboard, a mouse, or a remote controller, and can drive the electronic product in an intuitive manner. According to this, how to control the display screen in a three-dimensional space is more convenient and conforms to the diversified use situation, which is an important part of the current development.

The invention provides a pattern-based correction method and an electronic device, which can automatically establish a manipulation plane in the space for the user to manipulate the content of the display screen by using the pattern of the display screen.

The pattern-based correction method of the present invention is for defining a manipulation plane in space. The correction method includes: displaying a pattern on a display screen of the display unit; capturing an image toward the display screen along the specified direction through the image capturing unit, wherein the image includes a polygon pattern corresponding to the pattern, and the specified direction and display The normal directions of the screens are not parallel to each other; the correction information is obtained by analyzing the polygon graphics; and based on the correction information, a manipulation plane corresponding to the display screen is established in the space, wherein the size of the manipulation plane has a size corresponding to the size of the display screen a proportional relationship; and establishing a correspondence between the control plane and each coordinate position of the display screen according to the above proportional relationship, thereby controlling the content of the display screen through the manipulation plane.

In an embodiment of the invention, the step of obtaining the correction information by analyzing the polygon graphic comprises: obtaining position information of each of the plurality of vertices of the polygon graphic, and using the position information as the correction information.

In an embodiment of the invention, the step of establishing a manipulation plane corresponding to the display screen in the space based on the correction information comprises: based on the position information and an angle information (ie, between the specified direction and the normal direction of the display screen) Angle information), defining multiple boundaries of the control plane; calculating the boundary length of each boundary; The proportional relationship is calculated according to the length of the boundary and the size of the display screen.

In an embodiment of the invention, the plurality of display pixels of the pattern are evenly distributed over a designated area on the display screen. The above polygon pattern is, for example, a trapezoid. The image capturing unit described above includes a depth camera.

The electronic device of the present invention includes: an image capturing unit that captures an image toward a display screen of the display unit along a specified direction, wherein the image is displayed with a pattern, and the image includes a polygon pattern corresponding to the pattern, and The processing direction is not parallel to the normal direction of the display screen; and the processing unit is coupled to the image capturing unit and the display unit, wherein the processing unit obtains the correction information by analyzing the polygon graphic, and based on the correction information, A control plane corresponding to the display screen is established in the space, wherein the size of the manipulation plane has a proportional relationship with the size of the display screen, and the correspondence relationship between the control plane and each coordinate position of the display screen is established according to the proportional relationship, thereby Plane to control the content of the display.

Based on the above, the present invention can automatically define a manipulation plane in the space before the user performs the operation, so that the user can perform gesture operations in the manipulation plane to control the content of the display screen, thereby improving the operation of the display screen in the space. Convenience.

The above described features and advantages of the invention will be apparent from the following description.

100‧‧‧Electronic devices

110‧‧‧Image capture unit

120‧‧‧Processing unit

130‧‧‧storage unit

131‧‧‧ pattern setting module

132‧‧‧Analysis module

133‧‧‧Flat setting module

210‧‧‧Display unit

410‧‧‧ pattern

420‧‧ images

430‧‧‧ Polygon graphics

Vertex 431~433‧‧‧

D‧‧‧ spacing

D1‧‧‧Specified direction

DB1~DB4, TB1~TB4‧‧‧ boundary

DS‧‧‧ display

ND‧‧‧ normal method

TP‧‧‧ control plane

S305~S325‧‧‧Steps of calibration method

1 is a functional block diagram of an electronic device in accordance with an embodiment of the present invention.

2 is a schematic diagram of a configuration of a three-dimensional interactive system in accordance with an embodiment of the present invention.

3 is a flow chart of a pattern-based correction method in accordance with an embodiment of the present invention.

4A is a schematic diagram of displaying a pattern on a display screen in accordance with an embodiment of the present invention.

4B is a schematic diagram of an image having a polygonal pattern in accordance with an embodiment of the present invention.

FIG. 5 is a schematic diagram of a pattern-based correction method in accordance with an embodiment of the present invention.

In order to make the disclosure of the present disclosure easier to understand, the following specific embodiments are examples that can be implemented by the present disclosure.

1 is a functional block diagram of an electronic device in accordance with an embodiment of the present invention. 2 is a schematic diagram of a configuration of a three-dimensional interactive system in accordance with an embodiment of the present invention. Referring to FIG. 1 and FIG. 2 simultaneously, the electronic device 100 is configured to control the display screen DS of the display unit 210 in the three-dimensional interactive system. In the embodiment, the display unit 210 is, for example, a projection display, and the electronic device 100 is disposed below the display screen DS. However, in other embodiments, the display unit 210 can also be any type of display. The display device is, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, or a flexible display.

Here, the electronic device 100 is disposed below the display unit 210 and captures an image toward the display screen DS of the display unit 210 along the designated direction D1 (the image capturing direction). However, in other embodiments, the electronic device 100 may also be disposed around the display screen DS such as upper, left, and right of the display unit 210. That is, the position of the image capturing unit 110 can be captured along the display direction DS along a specified direction D1, and the specified direction D1 and the normal direction ND of the display screen DS are not parallel to each other.

The angle between the specified direction D1 and the normal direction ND may be set to be within an angle range, wherein the angle range is determined based on the type of the lens of the image capturing unit 110. The angle range is, for example, 90°±θ, and the above θ is determined according to the type of lens of the image capturing unit 110. For example, the larger the wide angle of the lens, the larger the above θ. For example, the above angle range is 90°±45°, that is, 45° to 135°; or the above angle range is 90°±30°, that is, 60° to 120°. Further, the angle between the specified direction D1 and the normal direction ND is preferably 90 degrees. In the present embodiment, the specified direction D1 is substantially perpendicular to the normal direction ND of the display screen DS.

The electronic device 100 includes an image capturing unit 110, a processing unit 120, and a storage unit 130. The image capturing unit 110 is, for example, a depth camera or a stereo camera, or the image capturing unit 110 may be any of a charge coupled device (CCD) lens and a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor). Transistors, CMOS) Camera/camera with lens or infrared lens. The image capturing unit 110 is configured to capture an image toward the display screen DS of the display unit 210 along the specified direction D1. In the case where a pattern is displayed on the display screen DS, the captured image includes a polygon pattern corresponding to the above pattern.

The processing unit 120 is coupled to the image capturing unit 110 and the storage unit 130. Moreover, the processing unit 120 can also be coupled to the display unit 210 in a wired or wireless manner. The processing unit 120 performs image processing and analysis according to the image captured by the image capturing unit 110, and controls the content displayed on the display screen DS. In this embodiment, the processing unit 130 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), or other programmable microprocessor (Microprocessor).

The storage unit 130 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (Flash memory), hard A disc or other similar device or combination of these devices is used to record a plurality of modules executable by the processing unit 120 to implement a method of correcting the control plane in space. For example, the module includes a pattern setting module 131, an analysis module 132, and a plane setting module 133. However, this is merely an example and is not limited thereto.

The pattern setting module 131 is configured to provide a pattern to the display screen DS for display, so that the image capturing unit 110 captures the image of the display screen DS on which the pattern is displayed. The analysis module 132 is configured to analyze the image captured by the image capturing unit 110. The plane setting module 133 establishes a correspondence in the space based on the analysis result of the analysis module 132. The display plane of the display screen DS.

The correction method for defining the control plane in space is described below with reference to FIG. 1 and FIG. 2 described above. 3 is a flow chart of a pattern-based correction method in accordance with an embodiment of the present invention. Referring to FIG. 1 to FIG. 3 simultaneously, first, a pattern is provided to the display unit 210 through the pattern setting module 131, and the pattern is displayed on the display screen DS of the display unit 210 (step S305). The plurality of display pixels of the pattern are evenly distributed in a designated area on the display screen DS, and the designated area may be the entire display screen DS or a partial area in the display screen DS. That is, the size of the designated area is less than or equal to the display screen DS.

Next, the image capturing unit 110 captures the image having the polygon pattern corresponding to the pattern toward the display screen DS in the specified direction D1 (step S310). That is, the polygon pattern formed in the image captured by the image capturing unit 110 is because the specified direction D1 has an angle with the normal direction ND of the display image DS, and thus the image capturing unit 110 is displayed in the captured image. When the image of the display screen DS of the above pattern is displayed, the pattern is deformed in the captured image due to the angle relationship, and the polygonal pattern is formed.

For example, FIG. 4A is a schematic diagram of displaying a pattern on a display screen according to an embodiment of the invention. 4B is a schematic diagram of an image having a polygonal pattern in accordance with an embodiment of the present invention. In the present embodiment, a rectangle 410 filled with a checkered pattern inside is used as the pattern 410, and the pattern 410 is displayed on the display screen DS as shown in FIG. 4A. However, the pattern 410 shown in FIG. 4A is only one example, and other filling patterns may be selected in other embodiments, which are not limited herein.

The image capturing unit 110 is photographed from the bottom up, and an image 420 as shown in FIG. 4B can be obtained. The image 420 includes a polygonal graphic 430 that is a trapezoid formed corresponding to the pattern 410. In addition, in other embodiments, the image capturing unit 110 may be disposed above the display screen DS, and the image is taken from the top to the bottom, which is not limited herein.

Thereafter, the polygon graphic 430 in the image 420 is analyzed by the processing unit 120 to obtain correction information (step S315). For example, the analysis module 132 analyzes the image to obtain the position information of each of the plurality of vertices 431 433 433 of the polygon pattern 430 , and uses the position information as the correction information.

Next, the processing unit 120 establishes a manipulation plane corresponding to the display screen DS in the space based on the correction information (step S320), wherein the size of the manipulation plane has a proportional relationship with the size of the display screen DS. For example, the plane setting module 133 defines the multiple boundaries of the manipulation plane based on the position information of the vertices 431 434 434 and an angle information, that is, the angle information between the specified direction D1 and the normal direction ND of the display screen DS. And calculate the boundary length of each boundary, and then calculate the proportional relationship according to the length of the boundary and the size of the display screen DS. Then, a correspondence relationship between the manipulation plane and each coordinate position of the display screen DS is established according to the proportional relationship (step S325), whereby the content of the display screen DS is controlled through the manipulation plane.

For example, FIG. 5 is a schematic diagram of a pattern-based correction method in accordance with an embodiment of the present invention. As shown in FIG. 5, a pattern 410 is displayed on the display screen DS, and the processing unit 120 performs the above steps S315 to S325 to obtain the manipulation plane TP. In FIG. 5, the spacing d between the control plane TP and the display screen DS can also be set in advance, Wherein the above spacing d can be greater than or equal to zero.

In practical applications, the image capturing unit 110 may include a depth camera and a color camera. Since the depth image obtained by the depth camera only has a depth value, the color image can be obtained through a general color camera, so that the analysis module 132 recognizes the color image to find the outline of the polygon pattern 430, thereby obtaining the vertex 431. ~434. Then, the plane setting module 133 can know the position information of the image 420 according to the position information of the vertices 431 to 434 of the polygon pattern 430, and the angle information between the respective depth values and the specified direction D1 and the normal direction ND of the display screen DS. The position of the pattern 410 in space further defines the boundaries TB1~TB4 of the control plane TP and their boundary lengths. The boundaries TB1 to TB4 of the control plane TP also correspond to the boundaries DB1 to DB4 of the display screen DS, respectively.

Thereafter, the plane setting module 133 calculates the length ratio of the boundaries TB1 TB4 of the control plane TP and the corresponding boundaries DB1 □ DB4 on the display screen DS, and uses the calculation result as the control plane TP and the display screen DS. The proportional relationship further establishes a correspondence between the manipulation plane TP and each coordinate position on the display screen DS according to the proportional relationship.

For example, the steering plane TP has a height of H2 and a width of W2. The height of the display screen DS is H1 and the width is W1. Therefore, the proportional relationship between the display screen DS calculated by the plane setting module 133 and the manipulation plane TP may include a height proportional relationship H1/H2 and a width proportional relationship W1/W2. The plane setting module 133 can further establish a correspondence relationship between the coordinate position (X1, Y1) of the manipulation plane TP and the coordinate position (X2, Y2) of the display screen DS according to the above proportional relationship. E.g, X2 = X1 × (W1/W2), Y2 = Y1 × (H1/H2). However, this is merely an example and is not limited thereto.

In FIG. 5, the proportional relationship between the control plane TP and the display screen DS is, for example, 1:1, that is, the control plane TP has the same area as the display screen DS, so that the coordinate position on the manipulation plane TP can be directly converted into the display screen DS. The coordinate position on the above, but the invention is not limited to this.

In other embodiments, the size of the manipulation plane TP established via the above-described correction method is not limited to be the same as the size of the display screen DS, and may be determined by the size of the designated area in which the display pixels of the view 410 are evenly distributed. That is to say, the pattern 410 can be displayed in a size smaller than the designated area of the display screen DS, and the control plane having a size smaller than the display screen DS can be obtained through the above steps S310 to S325.

In summary, the above embodiment provides a pattern-based correction method and an electronic device, which uses a pattern of a display screen to define a manipulation plane in space, and controls a content of the display screen through a manipulation plane. Through the above embodiments, the user can automatically define a control plane in the space before the interactive operation, thereby improving the convenience of the user to operate the three-dimensional interactive system.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

S305~S325‧‧‧Steps based on the calibration method

Claims (10)

A pattern-based correction method for defining a manipulation plane in a space, the correction method comprising: displaying a pattern on a display screen of a display unit; and facing the display screen through a frame-taking unit along a specified direction An image is captured, wherein the image includes a polygon pattern corresponding to the pattern, and the specified direction is not parallel to a normal direction of the display screen; and the correction information is obtained by analyzing the polygon pattern; Establishing, in the space, the manipulation plane corresponding to the display screen, wherein the size of the manipulation plane has a proportional relationship with the size of the display screen; and establishing the manipulation plane and each coordinate of the display screen according to the proportional relationship A correspondence between the locations by which the content of the display screen is controlled. The correction method according to claim 1, wherein the step of obtaining the correction information by analyzing the polygon pattern comprises: obtaining position information of each of the plurality of vertices of the polygon pattern, and using the position information as the correction News. The method of claim 2, wherein the step of establishing the manipulation plane corresponding to the display screen in the space based on the correction information comprises: based on the location information and the specified direction and the display screen The normal side An angle information between the directions defines a plurality of boundaries of the manipulation plane; calculates a boundary length of each of the boundaries; and calculates the proportional relationship according to the boundary lengths and the size of the display screen. The correction method according to claim 1, wherein the plurality of display pixels of the pattern are evenly distributed in a designated area on the display screen. The correction method of claim 1, wherein the polygon pattern is a trapezoid. The correction method of claim 1, wherein the image capturing unit comprises a depth camera. An electronic device includes: an image capturing unit that captures an image along a display direction of a display unit along a specified direction, wherein the image is displayed with a pattern, and the image includes a polygon corresponding to the pattern And the processing unit is coupled to the image capturing unit and the display unit, wherein the processing unit obtains a correction information by analyzing the polygon graphic, and the processing unit is coupled to the image capturing unit and the display unit. And establishing, according to the correction information, a manipulation plane corresponding to the display screen in a space, wherein the size of the manipulation plane has a proportional relationship with the size of the display screen, and establishing the manipulation plane and the display according to the proportional relationship The correspondence between each coordinate position of the screen, through which the content of the display screen is controlled. The electronic device of claim 7, further comprising: a storage unit, comprising a plurality of modules for execution by the processing unit, wherein the The module comprises: a pattern setting module, wherein the pattern is provided for display in the display screen; an analysis module analyzes the polygon pattern to obtain the correction information; and a plane setting module, based on the correction information, in the space Establishing the manipulation plane corresponding to the display screen, wherein the size of the manipulation plane has a proportional relationship with the size of the display screen; and establishing a relationship between the manipulation plane and each coordinate position of the display screen according to the proportional relationship Corresponding relationship, by which the content of the display screen is controlled through the manipulation plane. The electronic device of claim 8, wherein the analysis module obtains position information of each of the plurality of vertices of the polygon graphic, and uses the position information as the correction information; the plane setting module is based on the positions Information and an angle information between the specified direction and the normal direction of the display screen, defining a plurality of boundaries of the manipulation plane, and calculating a boundary length of each of the boundaries, and displaying the boundary according to the boundary lengths Calculate the scale relationship by the size of the screen. The electronic device of claim 8, wherein the plurality of display pixels of the pattern are evenly distributed on a designated area on the display screen; the polygon pattern is a trapezoid; and the image capturing unit comprises a depth camera.