WO2013161261A1

WO2013161261A1 - Display control system and indicating device

Info

Publication number: WO2013161261A1
Application number: PCT/JP2013/002708
Authority: WO
Inventors: 山田　和宏
Original assignee: パナソニック株式会社
Priority date: 2012-04-26
Filing date: 2013-04-22
Publication date: 2013-10-31

Abstract

In this display control system (100), display control is performed in accordance with positions on a display panel (24) indicated by an indicating device (10). Said indicating device (10) is provided with a reading unit (15) that takes an image at an indicated position, said image including a position-information pattern. An identification unit (16a) identifies the position on the basis of the position-information pattern included in the image taken by the reading unit (15). An orientation-detection unit (16b) detects distortion in the taken image, and from said distortion, detects the orientation of the indicating device (10).

Description

Display control system and pointing device

This disclosure relates to a display control system capable of handwriting input on a display surface of a digital display.

Patent Document 1 discloses a technique for digitizing information entered on paper and transmitting the digitized information to a server or a terminal when characters or the like are entered on the paper using a pen.

JP 2007-226577 A

This disclosure provides a display control system capable of executing handwriting input with high accuracy on the display surface of a display device.

The display control system according to the present disclosure includes a display device having a display unit and an instruction device for indicating a position on the display unit, and performs display control according to the position indicated by the instruction device. The display unit of the display device is formed with a position information pattern representing a planar position on the display unit, and the pointing device has an instruction unit for indicating a position on the display unit, and a position indicated by the instruction unit. A display unit that captures an image including a position information pattern, and the display control system specifies a position in the display unit based on the position information pattern included in the captured image by the reading unit; A posture detection unit that detects distortion of the captured image and detects the posture of the pointing device with respect to the display unit from the detected distortion;

The display control system according to the present disclosure can realize highly accurate handwriting input without hindering downsizing of the pointing device and cost reduction of the display control system.

It is a schematic diagram of a display control system concerning an embodiment. 1 is a block diagram of a display control system according to Embodiment 1. FIG. It is a schematic sectional drawing of a display panel. It is a schematic sectional drawing of a digital pen. It is a top view of a color filter. (A) to (D) are diagrams showing examples of dot arrangement corresponding to each code. (A), (b) is a figure for demonstrating the relationship between the inclination of a digital pen, and the positional information read. It is an example of the captured image in the state of FIG. 4 is a flowchart illustrating a processing flow of the display control system according to the first embodiment. 6 is a block diagram of a display control system according to Embodiment 2. FIG. 10 is a flowchart illustrating a processing flow of the display control system according to the second embodiment. It is a schematic sectional drawing which shows the other structure of a digital pen.

Recently, a system capable of handwriting input has been developed in which a writing instrument such as a stylus is used to write characters on the display surface of the display device so that the locus of the writing instrument is directly displayed on the display surface. However, such a system is still developing. In particular, there is still room for development in terms of high-definition handwriting input.

As one of such display control systems, a position information pattern indicating a planar position is formed on the display unit, and the indicated position is detected by optically reading the pattern by a pointing device, and a locus display or the like is performed. Can be considered.

In the display control system using optical reading as described above, the following problems are expected to occur. That is, there is a certain gap between the surface of the display unit with which the pointing device abuts and the layer on which the position information pattern is formed, depending on the thickness of the glass substrate or the like. For this reason, for example, when using an instruction device that performs reading at the tip of the instruction unit, when the instruction device is placed perpendicular to the display unit surface, the indicated position matches the reading position, but the display unit surface When the pointing device is tilted with respect to the position, the indicated position and the reading position are displaced. For this reason, the position indicated by the instruction unit and the position displayed on the display device do not necessarily match, and a shift occurs, which may give the user a sense of discomfort. In addition, since the magnitude of the positional deviation changes depending on the inclination of the pointing device, it is not appropriate to fix the display position in a fixed manner.

For this problem, if the actual posture such as the tilt of the pointing device with respect to the display unit can be detected, for example, the display position can be corrected according to the posture information. However, adding some device to detect the attitude of the pointing device is not preferable in terms of downsizing the pointing device and reducing the cost of the display control system.

Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

The inventor (s) provides the accompanying drawings and the following description in order for those skilled in the art to fully understand the present disclosure, and is intended to limit the subject matter described in the claims. Not what you want.

(Embodiment 1)
[1. Overview of display control system]
FIG. 1 is a schematic diagram illustrating an appearance of a display control system 100 according to the embodiment. The display control system 100 includes an optical digital pen (hereinafter simply referred to as “digital pen”) 10 as a pointing device and a display device 20. As will be described in detail later, the display device 20 is a liquid crystal display, for example, and can display various objects on the display unit 21. Further, the display unit 21 is formed with a dot pattern as a position information pattern representing a planar position in the display unit 21. The digital pen 10 optically reads the dot pattern to detect position information of the designated position and transmits the position information to the display device 20. The display device 20 receives the position information as input and performs various processes. That is, the digital pen 10 functions as a reading device and also functions as a data input device to the display control system 100.

For example, the display device 20 can display the locus of the tip of the digital pen 10 by continuously displaying points at designated positions on the display unit 21 following the movement of the digital pen 10. That is, it is possible to enter characters, figures, and the like on the display unit 21 using the digital pen 10. In addition, the display device 20 follows the movement of the digital pen 10 and continuously erases the display at the designated position on the display unit 21, thereby displaying a portion that matches the locus of the tip of the digital pen 10. Can be erased. That is, the digital pen 10 can be used like an eraser. Furthermore, the display device 20 can use the digital pen 10 as an input device such as a mouse by displaying a specified position on the display unit 21. In this way, in the display control system 100, the position of the digital pen 10 is input as an input to the display device 20 by moving the digital pen 10 on the display unit 21 of the display device 20, and the display device 20 responds to the input. To change the displayed contents.

[2. Configuration of display device]
Hereinafter, the display device 20 will be described. FIG. 2 is a block diagram illustrating a schematic configuration of the display control system 100.

The display device 20 includes a receiving unit 22 that receives a signal from the outside, a display-side microcomputer 23 that controls the entire display device 20, and a display panel 24 that displays an image. The display panel 24 of the present embodiment is a liquid crystal panel.

The receiving unit 22 receives a signal transmitted from the digital pen 10, which will be described in detail later. The signal received by the receiving unit 22 is sent to the display-side microcomputer 23.

The display-side microcomputer 23 is composed of a CPU, a memory, and the like, and a program for operating the CPU is also mounted. The display-side microcomputer 23 is an example of a control unit. For example, the display-side microcomputer 23 controls the display panel 24 based on a signal transmitted from the digital pen 10 and changes the content displayed on the display unit 21.

FIG. 3 is a schematic sectional view of the display panel 24. The basic configuration of the display panel 24 is the same as that of a general liquid crystal panel. Specifically, the display panel 24 includes a pair of glass substrates 25, a polarizing filter 26 provided on the outer surface of each glass substrate 25, a pair of alignment films 27 provided between the pair of glass substrates 25, and a pair A liquid crystal layer 28 provided between the alignment films 27, a transparent electrode 29 provided on each alignment film 27, and a color filter 30 provided between the glass substrate 25 on the surface side and the transparent electrode 29. Have. Various images are displayed on the display unit 21. Although described later in detail, the dot pattern described above is written on the color filter 30. The dot pattern is an example of a position information pattern.

[3. Configuration of digital pen]
Next, a detailed configuration of the digital pen 10 will be described. FIG. 4 is a cross-sectional view illustrating a schematic configuration of the digital pen 10.

The digital pen 10 includes a cylindrical main body 11, a pen tip 12 as an instruction unit attached to the tip of the main body 11, a pressure sensor 13 that detects pressure acting on the pen tip 12, and an infrared ray. Irradiation unit 14 that emits light, reading unit 15 that reads incident infrared light, a control unit 16 that controls the digital pen 10, a transmission unit 17 that outputs a signal to the outside, and each member of the digital pen 10 And a power source 19 for supplying power to the power source.

The main body 11 is formed of a cylinder similar to a general pen. The pen tip portion 12 has a tapered shape, and its tip is formed to be round so as not to damage the display portion 21. Moreover, it is preferable that the shape of the pen point part 12 is a shape in which the user can easily recognize the image displayed on the display unit 21. In the configuration of FIG. 4, the pen tip portion 12 is formed of a material that can transmit infrared light.

The pressure sensor 13 is built in the main body 11 and connected to the proximal end of the pen tip 12. The pressure sensor 13 detects the pressure applied to the pen tip unit 12 and transmits the detection result to the control unit 16. Specifically, the pressure sensor 13 detects the pressure applied to the pen tip portion 12 when the user enters characters or the like on the display portion 21 using the digital pen 10. That is, the pressure sensor 13 is used when determining whether or not there is an input intention of the user using the digital pen 10.

The irradiation unit 14 is provided at the tip of the main body 11 and in the vicinity of the pen tip 12. The irradiation part 14 is comprised by infrared LED, for example, and is comprised so that infrared light may be irradiated from the front-end | tip of the main-body part 11. FIG. In the configuration of FIG. 4, a plurality of (for example, four) irradiation units 14 are arranged so as to surround the pen tip unit 12. The number of irradiation units 14 can be set as appropriate. Moreover, the irradiation part 14 may be formed in ring shape.

The reading unit 15 is provided at the tip of the main body 11 and in the vicinity of the pen tip unit 12. The reading unit 15 includes an objective lens 15a, a lens 15b, and an image sensor 15c built in the tip of the pen tip unit 12, and the objective lens 15a and the lens 15b constitute an optical system. The objective lens 15a and the lens 15b form incident light on the image sensor 15c. Infrared light emitted from the irradiation unit 14 and reflected by the display device 20 is incident on the objective lens 15a. The image sensor 15c is provided on the optical axes of the objective lens 15a and the lens 15b. The imaging element 15 c converts an optical image formed on the imaging surface into an electrical signal and outputs the electrical signal to the control unit 16. The image sensor 15c is configured by, for example, a CCD image sensor or a CMOS image sensor. Although the details will be described later, since the dot pattern is formed of a material that absorbs infrared light, the dot pattern does not return infrared light. As a result, an optical image in which the dot pattern is expressed in black is captured by the image sensor 15c.

The control unit 16 includes a specifying unit 16a, a posture detection unit 16b, and a pen-side microcomputer 16c, as shown in FIG. The specifying unit 16 a specifies position information on the display unit 21 of the digital pen 10 based on the image signal from the reading unit 15. Specifically, the specifying unit 16a acquires a dot pattern from the image signal acquired by the reading unit 15, and specifies the position of the pen tip unit 12 on the display unit 21 based on the dot pattern. Information regarding the position of the pen tip part 12 specified by the specifying part 16a is sent to the pen-side microcomputer 16c. The posture detection unit 16 b detects posture information with respect to the display unit 21 of the digital pen 10 based on the image signal from the reading unit 15. Specifically, the posture detection unit 16b detects image distortion from the image signal acquired by the reading unit 15 based on the arrangement of dot patterns and the like, and the detected distortion information indicates the normal direction of the surface of the display unit 21. The tilt of the digital pen 10 is detected. Information about the posture of the digital pen 10 detected by the posture detection unit 16b is sent to the pen-side microcomputer 16c. The pen side microcomputer 16 c controls the entire digital pen 10. The pen side microcomputer 16c is constituted by a CPU, a memory, and the like, and a program for operating the CPU is also mounted.

The transmission unit 17 transmits a signal to the outside. Specifically, the transmission unit 17 wirelessly transmits the position information specified by the specifying unit 16a and the posture information detected by the posture detection unit 16b to the outside. The transmission unit 17 performs near field communication with the reception unit 22 of the display device 20. The transmitter 17 is provided at the end of the main body 11 opposite to the pen tip 12.

[4. Detailed structure of color filter]
Next, the detailed structure of the color filter 30 will be described. FIG. 5 is a plan view of the color filter 30.

The color filter 30 includes a black matrix 31, a plurality of pixel regions 32 that are partitioned by the black matrix 31 and transmit light of a specific color, and dots 33 provided in the pixel region 32. The pixel area 32 includes a red pixel area 32r that transmits red (R) light, a green pixel area 32g that transmits green (G) light, and a blue pixel area 32b that transmits blue (B) light. Contains. Each pixel region 32 corresponds to a subpixel of the display panel 24. Note that when the colors to be transmitted are not distinguished, they are simply referred to as “pixel region 32”. The black matrix 31 includes a vertical line extending in the longitudinal direction of the pixel region 32 and a horizontal line extending in the short direction of the pixel region 32 and is formed in a lattice shape. The horizontal line is formed thicker than the vertical line. The black matrix 31 and the dots 33 are formed of a material mainly composed of carbon black. The dots 33 are formed in a solid circle. The dots 33 are provided in some pixel regions 32 instead of all the pixel regions 32. In the color filter 30, a plurality of dots 33 are collected to form a dot pattern. This dot pattern differs depending on the position of the color filter 30.

The dot pattern will be described in detail below.

First, a first reference line 34 and a second reference line 35 are defined on the color filter 30. These first and

second reference lines

34 and 35 are virtual lines and are not actually existing lines. The first reference line 34 is a straight line extending in the short direction of the pixel region 32. A plurality of first reference lines 34 are arranged in parallel in the longitudinal direction of the pixel region 32 every two pixel regions 32. Each first reference line 34 is located at the center in the longitudinal direction of the pixel region 32. The second reference line 35 is a straight line extending in the longitudinal direction of the pixel region 32. The second reference line 35 is provided on the green pixel region 32g, and a plurality of second green cells 32g are arranged in parallel in the short direction of the pixel region 32. Each second reference line 35 is located in the center in the lateral direction of the green pixel region 32g. A grid is defined on the color filter 30 by the first reference line 34 and the second reference line 35.

Each dot 33 is arranged around the intersection of the first reference line 34 and the second reference line 35. FIG. 6 is a diagram showing an arrangement pattern of the dots 33. The dots 33 are arranged at positions shifted from the intersection in any of four directions orthogonal to each other (up and down, left and right in FIGS. 5 and 6). Specifically, the dots 33 are arranged in any one of FIGS. 6A to 6D. In the arrangement shown in FIG. 6A, the dot 33 is arranged at a position shifted to the right on the first reference line 34 from the intersection of the first reference line 34 and the second reference line 35. At this time, the dots 33 are arranged on the blue pixel region 32b. When this arrangement is digitized, it is represented by “1”. In the arrangement of FIG. 6B, the dot 33 is arranged at a position shifted upward on the second reference line 35 from the intersection of the first reference line 34 and the second reference line 35. At this time, the dots 33 are arranged on the green pixel region 32g. When this arrangement is digitized, it is represented by “2”. In the arrangement of FIG. 6C, the dot 33 is arranged at a position shifted to the left on the first reference line 34 from the intersection of the first reference line 34 and the second reference line 35. At this time, the dots 33 are arranged on the red pixel region 32r. When this arrangement is digitized, it is represented by “3”. In the arrangement of FIG. 6D, the dot 33 is arranged at a position shifted downward on the second reference line 35 from the intersection of the first reference line 34 and the second reference line 35. At this time, the dots 33 are arranged on the green pixel region 32g. When this arrangement is digitized, it is represented by “4”.

Then, using 6 dots × 6 dots as one unit area, one dot pattern is formed by 36 dots 33 included in the unit area. By arranging each of the 36 dots 33 included in the unit area in any one of “1” to “4”, a huge number of dot patterns can be formed. The dot patterns in each unit area are all different.

Information is added to each of these dot patterns. Specifically, each dot pattern represents a position coordinate for each unit area. That is, when the color filter 30 is divided into unit areas of 6 dots × 6 dots, each dot pattern represents the position coordinates of the unit area. As such dot pattern patterning and coordinate conversion methods, for example, a publicly known method as disclosed in Japanese Patent Application Laid-Open No. 2006-141067 can be used.

[5. Operation]
Next, the operation of the display control system 100 configured as described above will be described.

FIG. 7 is an enlarged view showing a contact state between the digital pen and the display unit of the display device in the present embodiment. In FIG. 7A, the orientation of the pen tip portion 12 of the digital pen 10 is perpendicular to the display portion 21, and the imaging direction dc of the reading portion 15 of the digital pen 10 is the normal direction of the surface of the display portion 21. It matches dn. On the other hand, in FIG. 7B, the direction of the pen tip portion 12 of the digital pen 10 is inclined with respect to the display portion 20, and the imaging direction dc of the reading portion 15 of the digital pen 10 is the method of the display portion 20. It deviates from the line direction dn. For this reason, in the state of FIG. 7B, the position A captured by the color filter 30 on which the dot pattern is formed is shifted from the position B that is directly below the pen tip portion 12. This is because the glass substrate 25 and the polarization filter 26 on the surface of the display unit 21 have a thickness.

On the other hand, in the state of FIG. 7B, since the imaging direction dc of the reading unit 15 has an inclination with respect to the normal direction dn of the surface of the display unit 21, trapezoidal distortion occurs in the image captured by the reading unit 15. Arise. Therefore, in the present embodiment, trapezoidal distortion is detected from the captured image, and the tilt of the digital pen 10 is detected from the degree of the distortion.

FIG. 8 is an example of an image captured by the reading unit 15, (a) is an image captured in the state of FIG. 7 (a), and (b) is FIG. 7 (b), that is, the pen tip unit 12 is tilted. It is the image imaged in the state. In the image of FIG. 8B, trapezoidal distortion is generated as can be seen in comparison with FIG. By detecting the degree of the trapezoidal distortion by image processing, it is possible to detect how much the pen tip portion 12 is inclined with respect to the display portion 21. The degree of trapezoidal distortion can be obtained from the pitch size of the vertical arrangement of dot patterns, for example. That is, in the image of FIG. 8A, the vertical arrangement of dot patterns is arranged substantially in parallel. On the other hand, in the image of FIG. 8B, the vertical arrangement of the dot patterns has an inclination, and the inclination gradually increases as it approaches the edge of the image. That is, the pitch of the vertical arrangement of dot patterns is large on the near side and small on the far side. From the difference in the pitch of the dot pattern arrangement, the inclination of the imaging direction dc of the reading unit 15, that is, the inclination of the pen tip unit 12 can be calculated by geometric calculation.

In practice, rotation is often added to the captured image of the reading unit 15 in addition to the trapezoidal distortion. Therefore, it is preferable to detect the trapezoidal distortion after correcting the rotation of the image. Further, the trapezoidal distortion detection method is not limited to the one shown here.

FIG. 9 is a flowchart showing a processing flow of the display control system 100. Below, the case where a user writes a character into the display apparatus 20 using the digital pen 10 is demonstrated.

First, when the power of the display control system 100 is turned on, the pen side microcomputer 16c of the digital pen 10 monitors whether or not pressure is applied to the pen tip portion 12 in step S11. This pressure is detected by the pressure sensor 13. When the pressure is detected (Yes), the pen-side microcomputer 16c determines that the user is inputting characters on the display unit 21 of the display device 20, and proceeds to step S12. While the pressure is not detected (No), the pen side microcomputer 16c repeats Step S11. Note that when the power source of the digital pen 10 is turned on, the irradiation unit 14 starts irradiation of infrared light.

In step S12, the reading unit 15 of the digital pen 10 detects a dot pattern formed on the display unit 21. Infrared light is emitted from the irradiation unit 14, and the infrared light is absorbed by at least the dots 33 provided in the color filter 30 of the display device 20, and is reflected by the image region 32 and the like. The reflected infrared light is received by the image sensor 15c through the objective lens 15a and the lens 15b. As a result, a dot pattern is imaged by the image sensor 15c. In this way, the reading unit 15 optically reads the dot pattern. The image signal acquired by the reading unit 15 is transmitted to the specifying unit 16a and the posture detecting unit 16b.

In step S13, the specifying unit 16a acquires a dot pattern from the image signal, and specifies the position of the pen tip unit 12 on the display unit 21 based on the dot pattern. Specifically, the specifying unit 16a acquires a dot pattern by performing predetermined image processing on the obtained image signal. For example, since the black matrix 31 is formed of carbon black like the dots 33, it absorbs infrared light. Therefore, the black matrix 31 is also included in the image from the reading unit 15 in the same state as the dots 33. Therefore, the image signal from the reading unit 15 is subjected to predetermined image processing so that the dots 33 are easily discriminated from the black matrix 31, and an array of a plurality of dots 33 is acquired from the processed image signal. Subsequently, the specifying unit 16a determines a unit area of 6 dots × 6 dots from the acquired arrangement of the dots 33, and specifies the position coordinates (position information) of the unit area from the dot pattern of the unit area. The specifying unit 16a converts the dot pattern into position coordinates by a predetermined calculation corresponding to the dot pattern coding method. The specified position information is transmitted to the pen-side microcomputer 16c.

In step S14, the posture detection unit 16b detects the trapezoidal distortion of the image from the image signal, and detects the inclination of the pen tip unit 12 based on the degree of the trapezoidal distortion. For example, as described above, after performing rotation correction on the image, the inclination and pitch of the vertical arrangement of the dot pattern are obtained, and the inclination in the imaging direction of the reading unit 15 is calculated from the inclination and pitch by geometric calculation. . The calculated inclination is transmitted to the pen-side microcomputer 16c as posture information.

Subsequently, in step S15, the pen side microcomputer 16c transmits the position information and the posture information to the display device 20 via the transmission unit 17.

The position information and posture information transmitted from the digital pen 10 are received by the receiving unit 22 of the display device 20. The received position information and posture information are transmitted from the receiving unit 22 to the display-side microcomputer 23. When the display-side microcomputer 23 receives the position information, in step S16, the display-side microcomputer 23 controls the display panel 24 to change the display content of the position corresponding to the position information. In this example, since a character is input, a point is displayed at a position corresponding to the position information on the display unit 21.

At this time, the display-side microcomputer 23 corrects the position corresponding to the position information in consideration of the posture information. For example, in the state of FIG. 7B, the position information indicates the position A, but the actual position where the pen tip portion 12 is in contact with the display portion 21 is the position B, and a deviation occurs. This deviation is caused by the inclination of the pen tip portion 12. For this reason, the display-side microcomputer 23 changes the display content change position to, for example, the position B in consideration of the posture information, that is, the inclination of the pen tip portion 12 so as not to give the user a sense of incongruity. It should be noted that the corrected position is not necessarily set to the position B immediately below the pen tip portion 12, and is set to a position between the position A and the position B where the user is assumed not to feel uncomfortable. It doesn't matter.

Subsequently, in step S17, the pen side microcomputer 16c determines whether or not the input by the user is continued. When the pressure sensor 13 detects the pressure, the pen side microcomputer 16c determines that the input by the user is continued, and returns to step S11. Then, by repeating the above flow, dots are continuously displayed at the position of the pen tip 12 on the display unit 21 following the movement of the pen tip 12 of the digital pen 10. Finally, characters corresponding to the locus of the pen tip portion 12 of the digital pen 10 are displayed on the display portion 21 of the display device 20.

On the other hand, if the pressure sensor 13 does not detect the pressure in step S17, the pen side microcomputer 16c determines that the input by the user is not continued, and ends the process.

Thus, when the display device 20 displays the locus of the tip of the digital pen 10 on the display unit 21 on the display unit 21, handwriting input to the display unit 21 using the digital pen 10 can be performed.

In addition, although the case where a character was entered was demonstrated above, the usage of the display control system 100 is not restricted to this. Of course, not only characters but also numbers, symbols, figures, etc. can be entered, but the digital pen 10 can be used like an eraser to erase characters, figures, etc. displayed on the display unit 21. That is, in the above example, the point is displayed at the position corresponding to the position information on the display unit 21, but the point at the position may be deleted. Furthermore, the digital pen 10 can be used like a mouse to move a pointer displayed on the display unit 21 or to select an icon displayed on the display unit 21.

[6. Effects of the embodiment]
As described above, according to the present embodiment, the position of the pen tip unit 12 on the display unit 21 based on the dot pattern included in the image captured by the reading unit 15 by the specifying unit 16a provided in the digital pen 10. Is identified. At the same time, a trapezoidal distortion of the image captured by the reading unit 15 is detected by the posture detection unit 16b provided in the digital pen 10, and based on the degree of the trapezoidal distortion, the display unit 21 of the pen tip unit 12 is detected. Tilt is detected. The display device 20 determines the change position of the display information on the display unit 21 based on the position information specified by the specifying unit 16a and taking into account the posture information detected by the posture detection unit 16b.

Thereby, even when the posture of the digital pen 10 is tilted, the display position can be corrected so that there is no deviation between the position of the pen tip portion 12 and the display position on the display. In addition, since the tilt of the pen tip 12 is detected from the image captured by the reading unit 15, it is not necessary to add a new device to obtain the posture information of the digital pen 10. Therefore, it is possible to eliminate the user's uncomfortable feeling without increasing the size and cost of the digital pen 10.

That is, in this embodiment, the position on the display unit 21 is instructed by the digital pen 10 to the display device 20 having the display unit 21. At this time, in the digital pen 10, the reading unit 15 captures an image including a dot pattern representing a planar position on the display unit 21. And the position in the display part 21 is pinpointed by the specific | specification part 16a based on the dot pattern contained in this captured image. Further, the posture detection unit 16b detects the distortion of the captured image, and the posture of the digital pen 10 with respect to the display unit 21 is detected from the detected distortion. For this reason, for example, even if there is a deviation between the designated position and the reading position of the dot pattern due to the posture of the digital pen 10, the detected posture information is taken into consideration in the display unit 21. The display position can be corrected. Therefore, using the posture information detected from the captured image, it is possible to correct the deviation between the indicated position and the display position due to the posture of the digital pen 10 in advance. The detected posture information can also be used for setting a display information change mode on the display unit 21.

It should be noted that, even in the position detection of the pen tip portion 12 in the specifying portion 16a, image distortion correction and rotation correction are necessary in order to accurately recognize the position information pattern. For this reason, it is considered that the distortion correction amount of the image is detected during the processing of the specifying unit 16a. In this case, it is possible to detect the inclination of the pen tip 12 from the distortion correction amount. Therefore, in the configuration of FIG. 2, the specifying unit 16a and the posture detection unit 16b are separate blocks, but these may be integrated to perform processing.

In the above-described embodiment, the position information and the posture information are transmitted from the digital pen 10 and the display device 20 corrects the position indicated by the position information based on the posture information. The position indicated by the position information may be corrected by the pen 10 based on the posture information, and the corrected display position may be transmitted to the display device 20. However, when the type of the display device 20 is changed, there is a possibility that the distance from the surface of the display unit 21 to the layer on which the dot pattern is formed (here, the color filter 30) may change, and the display position is corrected accordingly. Need to be changed. For this reason, it can be said that the correction process is preferably performed on the display device 20 side.

(Embodiment 2)
Next, the display control system 200 according to the second embodiment will be described. FIG. 10 is a block diagram illustrating a schematic configuration of the display control system 200. The display control system 200 is different from the first embodiment in that the display device 220, not the digital pen 210, identifies the position of the digital pen 210 and detects the posture. Hereinafter, the description of the configuration substantially similar to that of the first embodiment may be omitted.

As shown in FIG. 10, the digital pen 210 has a pressure sensor 13, an irradiation unit 14, a reading unit 15, a control unit 216, and a transmission unit 17. The configurations of the pressure sensor 13, the irradiation unit 14, the reading unit 15, and the transmission unit 17 are the same as those in the first embodiment. The control unit 216 includes the pen-side microcomputer 16c and does not include the specifying unit 16a and the posture detection unit 16b of the first embodiment. That is, the control unit 216 outputs the image signal input from the image sensor 15c to the transmission unit 17 without specifying the position information of the digital pen 210 from the image signal. Thus, the image signal picked up by the image pickup device 15c is transmitted from the digital pen 210.

As shown in FIG. 10, the display device 220 includes a receiving unit 22 that receives an external signal, a display-side microcomputer 23 that controls the entire display device 20, a display panel 24 that displays an image, and a digital pen 10. It has a specifying unit 240 that specifies the position, and a posture detecting unit 241 that detects the posture of the digital pen 10. The configurations of the receiving unit 22, the display-side microcomputer 23, and the display panel 24 are the same as those in the first embodiment. A dot pattern as shown in FIG. 5 is formed on the display unit 21 of the display panel 24. The receiving unit 22 receives a signal transmitted from the digital pen 210 and transmits the signal to the specifying unit 240. The specifying unit 240 has the same function as the specifying unit 16a of the digital pen 10 in the first embodiment. That is, in this embodiment, since the transmission signal from the digital pen 210 is an image signal acquired by the image sensor 15c, the specifying unit 240 specifies the position of the digital pen 210 from the image signal. That is, the specifying unit 240 acquires a dot pattern from the image signal, and specifies the position coordinates of the pen tip unit 12 on the display unit 21 based on the dot pattern, similarly to the specifying unit 16a. The specifying unit 240 transmits the specified position information to the display-side microcomputer 23.

The posture detection unit 241 has the same function as the posture detection unit 16b of the digital pen 10 in the first embodiment. That is, in this embodiment, since the transmission signal from the digital pen 210 is an image signal acquired by the image sensor 15c, the posture detection unit 241 detects the posture of the digital pen 210 from the image signal. That is, the posture detection unit 241 detects image distortion based on the arrangement of dot patterns and the like from the image signal, as in the posture detection unit 16b, and digitally displays the normal direction of the display unit 21 from the detected distortion information. The inclination of the pen 10 is detected. The posture detection unit 241 transmits the detected posture information to the display-side microcomputer 23. The display-side microcomputer 23 controls the display panel 24 to change the display information displayed on the display unit 21 based on the position information and the posture information.

Next, the operation of the display control system 200 will be described. FIG. 11 is a flowchart showing a process flow of the display control system 200. Below, the case where a user writes a character in the display apparatus 220 using the digital pen 210 is demonstrated.

When the power of the display control system 200 is turned on, in step S21, the pen side microcomputer 16c of the digital pen 210 monitors whether or not pressure is applied to the pen tip portion 12. When the pressure is detected (Yes), the pen-side microcomputer 16c determines that the user is inputting characters to the display unit 21 of the display device 220, and proceeds to step S22. In step S <b> 22, the reading unit 15 of the digital pen 210 acquires a dot pattern image formed on the display unit 21. The image signal acquired by the reading unit 15 is transmitted to the display device 220 via the transmission unit 17 in step S23.

The image signal transmitted from the digital pen 210 is received by the receiving unit 22 of the display device 220 in step S24. The received image signal is sent to the specifying unit 240 and the posture detecting unit 241. The specifying unit 240 acquires a dot pattern based on the image signal and specifies the position of the digital pen 210. The position information specified by the specifying unit 240 is sent to the display-side microcomputer 23.

In step S25, the posture detection unit 241 detects the trapezoidal distortion of the image from the image signal, and detects the inclination of the pen tip unit 12 based on the degree of the trapezoidal distortion. For example, as described above, after performing rotation correction on the image, the inclination of the vertical arrangement and pitch of the dot pattern is obtained, and the inclination in the imaging direction of the reading unit 15 is calculated from the inclination by geometric calculation. The calculated inclination is sent to the display-side microcomputer 23 as posture information.

Subsequently, in step S26, when receiving the position information and the posture information, the display-side microcomputer 23 controls the display unit 24 to change the display contents of the position corresponding to the position information and the posture information. In this example, since a character is input, a point is displayed at a position corresponding to the position information on the display unit 21. At this time, the display-side microcomputer 23 corrects the position corresponding to the position information to a position where it is estimated that the user does not feel uncomfortable, for example, taking the posture information into consideration.

Thereafter, in step S27, the pen side microcomputer 16c determines whether or not the input by the user is continued. If the input continues (Yes), the process returns to step S21 and the above flow is repeated. On the other hand, if the input is not continued, the process is terminated. In this way, characters corresponding to the locus of the pen tip 12 of the digital pen 210 are displayed on the display unit 21 of the display device 220.

By performing such processing, the display control system 200 can detect the position of the digital pen 210 operated by the user with high definition and reflect the position on the display unit 21 with high definition.

That is, in this embodiment, the position on the display unit 21 is instructed by the digital pen 210 to the display device 220 having the display unit 21. At this time, in the digital pen 210, the reading unit 15 captures an image including a dot pattern representing a planar position on the display unit 21. In the display device 220, the specifying unit 240 specifies the position on the display unit 21 based on the dot pattern included in the captured image. Further, the posture detection unit 241 detects the distortion of the captured image, and the posture of the digital pen 210 with respect to the display unit 21 is detected from the detected distortion. For this reason, for example, even if a deviation occurs between the designated position and the dot pattern reading position due to the posture of the digital pen 210, the detected posture information is taken into consideration in the display unit 21. The display position can be corrected. Therefore, using the posture information detected from the captured image, it is possible to correct the deviation between the indicated position and the display position due to the posture of the digital pen 210 in advance. The detected posture information can also be used for setting a display information change mode on the display unit 21.

(Other embodiments)
As described above,

Embodiments

1 and 2 have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to an embodiment in which changes, replacements, additions, omissions, and the like are appropriately performed. In addition, it is possible to combine the components described in the first and second embodiments to form a new embodiment.

Therefore, other embodiments will be exemplified below.

In the first and second embodiments, the optical digital pen has been described as an example of the pointing device. The indicating device is for indicating a position on the display unit of the display device, and includes an indicating unit for indicating a position on the display unit, and an image including a position information pattern at the position specified by the indicating unit. It is only necessary to include a reading unit that captures images. Therefore, the pointing device is not limited to the optical digital pen. Further, the configuration of the instruction unit and the reading unit is not limited to that shown in the first and second embodiments.

In the first and second embodiments, the dot pattern has been described as an example of the position information pattern. The position information pattern may be formed on the display unit of the display device and represents a planar position on the display unit. Therefore, the position information pattern is not limited to a dot pattern. Further, the way of expressing the position coordinates and the division form of the unit area are not limited to those shown in the first and second embodiments.

The dots 33 are provided in the color filter 30, but are not limited thereto. The dots 33 may be provided on the glass substrate 25 or the polarizing filter 26. Furthermore, the display panel 24 may be configured to include a sheet different from the color filter 30, the glass substrate 25, and the polarizing filter 26 in which the dots 33 are formed.

In each embodiment described above, the posture information of the digital pen 10 is used for correcting the display position. That is, based on the position information output from the specifying unit, the change position of the display information on the display unit is determined in consideration of the posture information of the pointing device. However, the use form of the posture information is not limited to this. For example, the thickness of the line to be displayed, the touch of the line, or the like may be switched according to the inclination of the pen tip portion 12. That is, it is possible to set a display information change mode in the display unit according to the posture information.

In each of the above-described embodiments, the inclination of the pen tip 12 is obtained from the trapezoidal distortion of the image. However, for example, the rotation angle around the axis of the pen tip 12 is determined from the rotation correction amount of the image. It can also be detected as information. In this case, as a use form of the posture information, for example, a multicolor pen can be realized by changing the displayed color according to the rotation angle of the pen tip unit 12.

Further, in each of the above-described embodiments, the digital pen 10 having a configuration in which the pen tip and the reading position are matched as illustrated in FIG. 4 has been described as an example, but the present invention is not limited to this. For example, as shown in FIG. 12, even when the pen is configured so that the pen tip and the reading position are separated by a predetermined length, if the pen is tilted with respect to the display unit 21, the pen tip position and the reading position are Will be deviated from the predetermined length. Therefore, the method as described above is effective. In FIG. 12, the same reference numerals as those in FIG. That is, the digital pen 10 shown in FIG. 12 has a cylindrical main body 11, a pen tip 12 as an indicator attached to the tip of the main body 11, and a pressure for detecting pressure acting on the pen tip 12. A sensor 13, an irradiation unit 14 that emits infrared light, a lens 15 b, and an image sensor 15 c, a reading unit 15 that reads incident infrared light, and a control unit 16 that controls the digital pen 10. And a transmitter 17 for outputting a signal to the outside, and a power source 19 for supplying power to each member of the digital pen 10.

In addition, the specific unit and the posture control unit are assumed to be provided in the digital pen 10 in the first embodiment and are provided in the display device 20 in the second embodiment, but are not limited thereto. Alternatively, it may be provided as a control device separate from the digital pen 10 and the display device 20.

As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided.

Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above technique. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings and detailed description.

In addition, since the above-described embodiments are for illustrating the technique in the present disclosure, various modifications, replacements, additions, omissions, and the like can be made within the scope of the claims and the equivalents thereof.

This disclosure is applicable to a display control system that can realize highly accurate handwriting input. Specifically, the present disclosure can be applied to tablets, smartphones, notebook PCs, and the like.

10,210 Optical digital pen (indicator)
12 Pen tip (instruction unit)
15

Reading unit

16a, 240

Identification unit

16b, 241

Attitude detection unit

20, 220 Display device 21 Display unit 24

Display panel

100, 200 Display control system

Claims

A display control system comprising: a display device having a display unit; and an instruction device for indicating a position on the display unit, and performing display control according to the position instructed by the instruction device,
The display unit of the display device has a position information pattern representing a planar position in the display unit,
The pointing device is
An instruction unit for indicating a position on the display unit;
A reading unit that captures an image including the position information pattern at a position instructed by the instruction unit;
The display control system includes:
A specifying unit for specifying a position in the display unit based on a position information pattern included in a captured image by the reading unit;
An attitude detection unit that detects distortion of the captured image and detects an attitude of the pointing device with respect to the display unit from the detected distortion;
Display control system.
The display control system according to claim 1,
The display device
Based on the position information output from the specifying unit, taking into account the posture information output from the posture detection unit, to determine the change position of the display information in the display unit,
Display control system.
The display control system according to claim 1 or 2,
The display device
The display control system which can set the change form of the display information in the said display part according to the said attitude | position information.
The display control system according to any one of claims 1 to 3,
The posture detection unit is a display control system that detects a trapezoidal distortion of the captured image and detects an inclination of the pointing device with respect to the display unit based on the detected degree of the trapezoidal distortion.
The display control system according to any one of claims 1 to 3,
The pointing device has a pen shape,
The posture detection unit is a display control system that detects a rotation correction amount of the captured image and detects a rotation angle around the axis of the pointing device from the detected rotation correction amount.
An instruction device for instructing a position on the display unit with respect to a display device having a display unit and a position information pattern representing a planar position in the display unit formed on the display unit,
An instruction unit for indicating a position on the display unit;
A reading unit that captures an image including the position information pattern at a position instructed by the instruction unit;
A specifying unit for specifying a position in the display unit based on a position information pattern included in a captured image by the reading unit;
An attitude detection unit that detects distortion of the captured image and detects an attitude of the pointing device with respect to the display unit from the detected distortion;
Pointing device.