JP2017117434A - Coordinate detection system, coordinate detection method, image processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform detection of an instruction by an electronic pen and an instruction by a finger at the same time at a low cost.SOLUTION: A coordinate detection system for detecting a coordinate instructed by an electronic pen or a finger performing an instruction operation with respect to a display screen comprises: first light irradiation means of irradiating a coordinate region for detecting a coordinate from an outer periphery of the display screen with light; light detection means of detecting light that the first light irradiation means radiates and light that second light irradiation means, which emits light from the electronic pen, radiates; determination means of determining whether the electronic pen or the finger is used according to a brightness level on the basis of brightness information on brightness of the light detected by the light detection means; and position specification means of specifying a position of the electronic pen or the finger determined by the determination means.SELECTED DRAWING: Figure 13

Description

  The present invention relates to a coordinate detection system, a coordinate detection method, an image processing apparatus, and a program.

  There are methods that use a touch panel to detect coordinates of an electronic pen or a finger on an electronic blackboard or electronic whiteboard, and an optical system that uses a light source and an optical sensor such as an infrared LED and an optical camera. There are already known systems that use a mutual capacitive projection capacitive touch panel to distinguish between multipoint detection and instructions with an electronic pen and a finger.

  However, in the case of systems up to now, in the method using a projected capacitive touch panel, the touch panel has to be configured on the entire surface of the screen, which is the coordinate detection area. Especially when dealing with large screens such as electronic whiteboards, the number of wires and terminals increases, the wiring length to the detection terminals becomes longer, and the normally used transparent electrodes have too high electrical resistance, so the metal wiring layer has low electrical resistance. Becomes necessary, and the manufacturing cost increases. In addition, since these electrodes and wiring must be formed on the screen, the influence on the brightness is inevitable.

  In an electronic whiteboard, a display device body can be equipped with a touch panel, but it is difficult to provide an interactive device using a projector or to provide an existing display device in the form of a retrofit unit. Special technology is required to attach the touch panel to the screen (* note), and there is a problem that tuning in a state where it is attached to the display device is essential.

  (* Note) The current product is about “the diagonal of the screen is 149.86 cm (50-inch)”.

  Patent Document 1 discloses a touch panel system intended to simultaneously perform touch operations with a plurality of electronic pens and fingers. This system is a system using transmission electrodes and reception electrodes (so-called mutual capacitive projection capacitive touch panel) arranged in a grid pattern. This document discloses a method for distinguishing between an electronic pen and a finger by sending a pen identification signal to the receiving electrode based on a synchronization signal received by the electronic pen through the transmitting electrode, and further identifying the electronic pen. Yes.

  However, the invention described in Patent Document 1 is difficult to provide in the form of a retrofitting unit because the manufacturing cost increases and the influence on the brightness is unavoidable when an electronic whiteboard with a large screen of 50 inches or more is considered. This problem has not been solved.

  Therefore, an object of the present invention is to simultaneously detect an instruction with an electronic pen and detect an instruction with a finger at low cost.

  In order to solve the above-mentioned problem, the invention described in claim 1 is a coordinate detection system that detects coordinates indicated by an electronic pen or a finger that performs an instruction operation on a display screen, from the outer periphery of the display screen. From a first light irradiating means for irradiating light to a coordinate detection area for detecting coordinates, light emitted from the first light irradiating means, and second light irradiating means for emitting light from the electronic pen A light detection means for detecting the irradiated light, a determination means for determining whether the electronic pen or the finger is in accordance with a brightness level based on luminance information relating to the brightness of the light detected by the light detection means, And a position specifying means for specifying the position of the electronic pen or the finger determined by the determining means.

  According to the present invention, it is possible to simultaneously detect an instruction with an electronic pen and an instruction with a finger at low cost.

(A) is an external view of the coordinate detection system which concerns on 1st Embodiment, (b) is an enlarged view in the ellipse A of (a). It is a figure which shows the relationship between the electronic whiteboard main body 101, the electronic pen 107, and the user's finger | toe 108 which were shown in FIG. It is an example of the functional block diagram of the electronic whiteboard 200 which concerns on 1st Embodiment. It is an example of the hardware block diagram of the coordinate detection system which concerns on 1st Embodiment. (A) is a conceptual diagram of the electronic pen 107 which concerns on 1st Embodiment, (b) is a functional block diagram of the electronic pen shown to (a), (c) is (b) It is a hardware block diagram. It is an example of the camera image at the time of detecting the finger | toe 108 which concerns on 1st Embodiment. It is an example of the camera image at the time of detecting the electronic pen 107 which concerns on 1st Embodiment. It is an example of the light source inside a screen frame that shines with uniform brightness as viewed from the camera L103 according to the first embodiment. (A), (b), (c), (d) is explanatory drawing about correction | amendment of the light source image inside the frame of the screen which concerns on 1st Embodiment. It is explanatory drawing about the simultaneous detection of the electronic pen and finger | toe which concerns on 1st Embodiment. (A), (b) is explanatory drawing about the detection method of the electronic pen which concerns on 1st Embodiment, and a finger | toe. It is explanatory drawing about deletion of the finger stick which concerns on 1st Embodiment. It is an example of the sequence diagram which shows the operation | movement of the electronic whiteboard shown in FIG. It is a figure which shows an example of the external appearance of the coordinate detection system which concerns on 2nd Embodiment. It is an example of the hardware block diagram of the coordinate detection system which concerns on 2nd Embodiment. It is a figure which shows the example of the hardware constitutions of the electronic pen which concerns on 2nd Embodiment. It is an example of the functional block diagram of the electronic whiteboard which concerns on 2nd Embodiment. It is an example of the functional block diagram of the electronic pen which concerns on 2nd Embodiment. It is a flowchart which shows the example of a process of the electronic pen which concerns on 2nd Embodiment. It is a figure for demonstrating the communication timing of the electronic pen which concerns on 2nd Embodiment. It is a flowchart which shows the example of a process of the coordinate detection system which concerns on 2nd Embodiment. It is a sequence diagram which shows the example of a process of the coordinate detection system which concerns on 2nd Embodiment.

<Overview>
An embodiment of the present invention will be described. The coordinate detection system according to the present invention has the following characteristics in processing of coordinate detection of an electronic pen and a finger on a display screen including a projection surface of an electronic whiteboard (also referred to as an electronic blackboard) or a projector device.

  In short, the coordinate detection system uses an optical member such as a light-emitting element or a light-receiving element to avoid the influence of luminance on the display screen, and can handle almost as it is even if the screen size changes.

  The coordinate detection system simultaneously performs detection of the light source mounted on the electronic pen and finger detection for detecting the shadow of the light source inside the screen frame.

  In addition, the coordinate detection system is configured to make the luminance of the light source mounted on the electronic pen higher than the luminance of the light source inside the screen frame of the display device so as to have a difference in luminance.

  In addition, the coordinate detection system can distinguish whether the light source is a pen light source or a light source inside the screen frame by having a binary luminance threshold value in the same camera image.

  The coordinate detection system moves away from the optical camera if the brightness of the light sources inside the screen frame are all the same, that is, the brightness detected by the camera decreases as it approaches the diagonal. When the luminance of the light source inside the screen frame looks constant when viewed from a good camera, the luminance difference from the pen light source is easier to discriminate with a threshold, so the luminance of the light source inside the screen frame is corrected to make it appear uniform.

  Since the pen light source may be weakened due to the drop in the power supply (battery) voltage of the pen or the variation in the characteristics of the light source device (LED), the pen light source is measured by transmitting the brightness of the light source inside the pen and transmitting it to the main unit. The brightness of the light source inside the screen frame is adjusted so that it can be easily detected.

  LED is an abbreviation for Light Emitting Diode, and in the present embodiment, an LED is a light emitting element that emits infrared light.

  A structure related to coordinate detection is unnecessary on the display surface, and it can be retrofitted by separating it from the display device.

[First Embodiment]
Hereinafter, the present embodiment will be described in the case of a display device.

<Configuration>
FIG. 1A is an external view of the coordinate detection system according to the first embodiment, and FIG. 1B is an enlarged view of an ellipse A in FIG.

  An electronic whiteboard 200 as a display device shown in FIG. 1A includes an electronic whiteboard body 101 as a display device body and legs 160.

  The electronic whiteboard body 101 shown in FIG. 1B includes a screen 102 as display means, a camera L103 as first light detection means, a camera R104 as second light detection means, and a first light. And infrared LED arrays 105L, 105C, and 105R, which are light sources as irradiation means.

  FIG. 2 is a diagram showing a relationship among the electronic whiteboard body 101, the electronic pen 107, and the user's finger 108 shown in FIG.

  The electronic pen 107 includes an infrared LED 146 as a second light irradiation unit that emits light at the tip.

  Using the screen 102 of the electronic whiteboard body 101 as a coordinate detection area, two or more cameras L103 and R104 are installed at the corners of the screen frame, and the coordinates of the electronic pen 107 and the finger 108 pointing point are detected from the camera image.

FIG. 2 shows an example in which cameras L103 and R104 are installed at both ends on the screen. Since the installation distance of the base line between the cameras 103 and 104 and the installation angle of the camera L103 and camera R104 are known, the angle between the base line and the detected object is obtained from the camera image, and the position of the object is specified by triangulation therefrom. .
In order to improve the detection accuracy, cameras may be installed at both lower ends of the screen 102, and the position of the object may be specified from a combination of a plurality of cameras.

  Here, infrared cameras are used as the camera L103 and the camera R104, and an object is detected using infrared light.

  The camera L103 and the camera R104 can detect two opposite sides at an angle of view of 90 degrees (or more). A light source is installed inside the screen frame that is the two opposite sides, and the camera L103 and the camera R104 detect this light source.

  When a camera is installed on the lower side of the screen, a light source is also required inside the frame on the screen. In the above example, an example in which the infrared LED arrays 105L, 105c, and 105R are used as light sources is described. Instead of the infrared LED arrays 105L, 105c, and 105R, a light source that uniformly emits light using a light guide plate may be used.

  When a finger or the like is put on the coordinate detection area of the screen 102, the camera L103 and the camera R104 can detect it as a shadow of the opposing light source, and the position of this shadow is specified by triangulation.

  In the case of detecting the electronic pen 107, a light source (infrared LED) is attached to the pen tip, and the position is specified by detecting the point of the light source.

  Although an example of the configuration around the screen of the electronic whiteboard body 101 is shown, since the display unit is not related to coordinate detection, only the screen frame in which the camera L103, camera R104 and the light source (infrared LED) are installed is made independent. It may be configured (* Note).

  (* Note) To function as a pointing device for the PC-based main unit.

<Functional block diagram>
FIG. 3 is an example of a functional block diagram of the electronic whiteboard 200 according to the first embodiment.

  The electronic whiteboard (image processing apparatus) 200 includes a screen 102 as a display unit and a coordinate detection system 201.

  The coordinate detection system 201 includes a control unit 111, a first light irradiation unit 112, a first light detection unit 113, a second light detection unit 114, a detection level correction unit 115, a determination unit 116, a communication unit 117, and a position specification. Means 118 and adjustment means 119 are provided.

<Hardware block diagram>
FIG. 4 is an example of a hardware block diagram of the coordinate detection system 201 according to the first embodiment.

  The coordinate detection system 201 includes a CPU 120, ROM 121, RAM 122, wireless Md 123, I / F 124, camera L103, camera R104, and infrared LED arrays 105L, 105C, and 105R.

  CPU is an abbreviation for Central Processing Unit, and CPU 120 is an arithmetic device that performs overall control of coordinate detection system 201 by executing a program for coordinate detection system 201 stored in ROM 121 or the like, for example.

  ROM is an abbreviation for Read Only Memory, and ROM 121 is a non-volatile storage element that stores a control program for the coordinate detection system 201, and includes, for example, a mask ROM or a flash ROM.

  RAM is an abbreviation for Random Access Memory, and the RAM 122 is a storage element that develops a control program read from the ROM 121.

  The wireless Md (module) 123 is a communication unit that exchanges data with the wireless Md 142 built in the electronic pen 107. The wireless Md123 includes an antenna, a transmission unit, and a reception unit. The transmission unit of the wireless Md 123 transmits pen identification information, press detection information, light source luminance information of the second light irradiation unit, and the like to the display apparatus main body. The reception unit of the wireless Md 123 receives a response signal from the electronic pen 107.

  The I / F 124 is an element that exchanges data with an external device.

  Since the camera L103, the camera R104, and the infrared LED arrays 105L, 105C, and 105R have been described above, description thereof will be omitted.

  Here, the display means 110 shown in FIG. 3 is realized by the screen 102 shown in FIG. 3 is realized by the CPU 120, the ROM 121, the RAM 122, the program executed by the CPU 120, and the like shown in FIG. The first light irradiation means 112 is realized by the infrared LED arrays 105L, 105c, 105R shown in FIG. The first light detection means 113 is realized by the camera L103 shown in FIG. The second light detection means 114 is realized by the camera R104 shown in FIG.

  The detection level correction means 115 has a function of correcting the detection level at the time of detecting the instruction unit based on the luminance level detected by the first and second light detection means by the first light irradiation means, as shown in FIG. The CPU 120, the ROM 121, the RAM 122, and a program executed by the CPU 120 are implemented.

  The determining unit 116 has a function of determining whether the electronic pen 107 or the finger 108 is in accordance with the brightness level based on the luminance information regarding the luminance of the light detected by the cameras L103 and R104 which are the first and second light detecting units. And realized by the CPU 120, the ROM 121, the RAM 122, the program executed by the CPU 120, and the like shown in FIG.

  The communication means 117 is realized by the wireless Md 123 shown in FIG.

  The position specifying unit 118 has a function of specifying the position of the electronic pen 107 or the finger 108 determined by the determining unit 116, and is realized by the CPU 120, the ROM 121, the RAM 122, the program executed by the CPU 120, and the like shown in FIG. Is done.

  The adjusting unit 119 has a function of adjusting the light emission luminance of the first light irradiation unit based on the light source luminance information of the second light irradiation unit sent from the electronic pen 107, and includes the CPU 120 shown in FIG. This is realized by a program executed by the ROM 121, the RAM 122, and the CPU 120.

  Here, in the hardware block diagram, the camera L103, R104 is directly connected to the main processing unit including the CPU 120 via the I / F 124, but an FPGA (field-programmable gate array) is used for high-speed image processing. ) Etc. may be used separately.

  FIG. 5A is a conceptual diagram of the electronic pen 107 according to the first embodiment, and FIG. 5B is a functional block diagram of the electronic pen shown in FIG. FIG. 5C is a hardware block diagram of FIG.

  An electronic pen 107 shown in FIG. 5A has a wireless Md 142 inside, and a light source (infrared LED) 146 at the tip.

  The electronic pen 107 shown in FIG. 5B includes a control unit 131, a communication unit 132, a battery unit 133, a pressure detection unit 134, a signal intensity detection unit 135, and a signal transmission unit 136.

  An electronic pen 107 illustrated in FIG. 5C includes a microcomputer 141, a wireless Md 142, a battery 143, a pressure sensor 144, an infrared sensor 145, and an infrared LED 146.

  The microcomputer 141 includes, for example, a CPU, a RAM, a flash ROM, and the like, and is an element that performs overall control of the electronic pen 107 by executing a program for the electronic pen 107 stored in the flash ROM or the like.

  The wireless Md 142 includes an antenna, a transmission unit, and a reception unit.

  The battery 143 is a power source for the electronic pen 107, and a primary battery or a secondary battery is used.

  The pressure sensor 144 is a sensor that detects whether or not the user is holding the electronic pen 107.

  The infrared sensor 145 is a sensor that detects infrared light from the electronic whiteboard body 101.

  The infrared LED 146 is an element that irradiates the electronic whiteboard body 101 with infrared rays.

  Here, the control means 131 is realized by the microcomputer 141 and a program executed by the microcomputer 141, and controls the coordinate detection system 201 as a whole. The communication means 132 is realized by the wireless Md 142. The battery means 133 is realized by the battery 143. The pressure detecting means 134 is realized by the pressure sensor 144. The signal intensity detection means 135 is realized by an infrared sensor 145. The signal transmission means 136 is realized by an infrared LED 146.

  The electronic pen 107 has a light source (infrared LED) mounted on the tip of the pen, and when the electronic pen 107 is pressed against the screen, the pressure detection unit (pressure sensor) detects that the user is drawing Make the light source emit light.

  A wireless communication function (power-saving wireless: for example, Bluetooth: registered trademark) is mounted, and when the drawing operation is detected by pressing, the electronic whiteboard body 101 is notified that drawing is in progress.

  In order to simulate the operation of the eraser, another light source and a pressure detection unit may be provided on the pen bottom side of the electronic pen 107.

  The intensity of the signal to be transmitted (infrared) varies depending on the characteristics of the element (infrared LED 146) and the power supply voltage (the degree of consumption of the battery 143). For this reason, in order to prevent the detection accuracy from being lowered, the signal intensity is detected in the electronic pen 107, and it can be notified to the electronic whiteboard body 101 by wireless communication.

<Operation>
FIG. 6 is an example of a camera image when detecting the finger 108 according to the first embodiment.

  As an example, from the camera L103, infrared LED arrays 105R and 105C as light sources attached to the inner sides of the two opposing screen frames can be seen. When the finger 108 is struck on the screen 102, the light from the infrared LED arrays 105R and 105C as a light source is blocked by the finger 108 and detected as a shadow.

  FIG. 7 is an example of a camera image when detecting the electronic pen 107 according to the first embodiment.

  In the case of the electronic pen 107, the light of the infrared LED 146 as a light source mounted on the pen tip is detected. In this case, a shadow can be formed on the frame of the screen 102.

  In FIG. 7, it is difficult to distinguish the infrared LED arrays 105C and 105R that are the light sources of the screen frame and the infrared LED 146 that is the light source of the electronic pen 107 because there is no difference in luminance.

  FIG. 8 is an example of a screen frame inner light source that emits light with uniform brightness as viewed from the camera L103 according to the first embodiment.

  The brightness of the light source inside the frame of the screen 102 is reduced so that the infrared LED 146 that is the light source of the electronic pen 107 and the infrared LED array 105C, 105R that is the light source inside the frame of the screen 102 can be easily distinguished from the brightness (electronic pen 107 50% of the light source).

  If the light source inside the frame of the screen 102 emits light with uniform brightness, the brightness detected by the camera L103 becomes lower as the diagonal distance from the camera L103 becomes farther.

  In order to distinguish from the infrared LED 146, which is the light source of the electronic pen 107, using the threshold value from the detected image with high accuracy, it is better to look at uniform luminance.

  FIGS. 9A, 9 </ b> B, 9 </ b> C, and 9 </ b> D are explanatory diagrams for correcting the light source image inside the frame of the screen according to the first embodiment.

  A correction table (correcting the solid line in FIG. 9B as a dotted line) in the coordinates in each horizontal (H) direction is created from the original image, and correction is performed based on the correction table at the time of detection.

  The correction is performed by masking the light source part other than the screen frame inner side of the original image so that only the light source part inside the frame (infrared LED array 105R, 105C) of the screen 102 is applied (* Note).

  (* Note) Infrared LED 146, which is a pen light source that does not overlap with the light source of the frame, is not exposed. Although it is conceivable that correction is applied to a part of the pen light source, the pen light source has a higher luminance than the light source part inside the frame of the screen 102 and looks close, so there is no problem in detection.

  FIG. 10 is an explanatory diagram regarding the simultaneous detection of the electronic pen and the finger according to the first embodiment.

  FIG. 10 shows a state in which the luminance of the light source inside the frame of the screen 102 is corrected so that the luminance can be different from that of the pen light source.

  The protrusion of the finger 108 is detected as a shadow, and the electronic pen 107 is detected as a brighter point than the infrared LED arrays 105R and 105C that are the light sources of the frame.

  Furthermore, it is suitable for detection according to the light emission characteristics of the pen light source by making it possible to adjust the light emission brightness of the light source part inside the frame based on the intensity information of the infrared LED 146 that is the pen light source notified from the electronic pen 107 It can be adjusted to the light emission brightness.

  FIGS. 11A and 11B are explanatory diagrams of the electronic pen and finger detection method according to the first embodiment.

  Assume that an electronic pen 107 and a finger 108 are attached to the screen 102 as shown in FIG.

  When the luminances of the detected camera images in the vertical (V) direction are integrated and made one-dimensional, the result is as shown in FIG.

  Infrared LED arrays 105R and 105C as the light source inside the frame of the screen 102 appear to have almost constant brightness, and when the finger 108 is struck, it becomes a shadow, and when the electronic pen 107 is struck, the light source of the electronic pen 107 is Detected. When the shadow of the electronic pen 107 itself matches the frame light source of the screen 102 in the vicinity of the electronic pen 107 and the finger strike when drawing with the electronic pen 107, the shadow caused by the finger strike is detected.

  From this value, the determination means 116 of the coordinate detection system 201 determines that the position exceeding the pen detection threshold is the electronic pen 107 and determines the position below the finger detection threshold as the finger 108.

  Further, the position specifying means 118 of the coordinate detection system 201 specifies the angle of the electronic pen 107 with respect to the base line connecting the two cameras from the position of the electronic pen 107 in the H direction in FIG. The designated position (coordinates) is calculated.

  For example, the position specifying means 118 is based on the camera image acquired by the camera L103, and is based on the camera L103 from the position in the H direction corresponding to the point where the integrated value of the luminance in the V direction exceeds the pen detection threshold. The angle (first angle) of the electronic pen 107 with respect to is specified.

  As an example, the position specifying unit 118 stores in advance in the ROM 121 or the like correspondence information that associates the position and angle in the H direction starting from the camera. As a result, the position specifying means 118 can acquire the angle corresponding to the position in the H direction from the correspondence information stored in the ROM 121.

  Similarly, the position specifying means 118 is based on the camera R104 from the position in the H direction corresponding to the point where the integrated value of the luminance in the V direction exceeds the pen detection threshold based on the camera image acquired by the camera R104. The angle (second angle) of the electronic pen 107 with respect to the base line is specified.

  Further, the position specifying means 118 uses the distance, the first angle, and the second angle between the camera L103 and the camera R104, which are the base line of triangulation, to indicate the position (indicated by the electronic pen 107 by known triangulation ( (Coordinates) is calculated (specified).

  Further, the position specifying means 118 specifies the indicated position (coordinates) of the finger 108 by performing the same process on the point where the integrated value of the luminance in the V direction is below the finger detection threshold.

  Multi-point detection is possible by linking the same ID to each detection point combination whose position has been specified, so that detection points are added when new detection points appear in the presence of existing detection points. Also, assume that the same point never flies far away between scans.)

  FIG. 12 is an explanatory diagram regarding deletion of a finger butt according to the first embodiment.

  If a position that seems to be a finger is detected in a certain range in the vicinity of the position where the electronic pen 107 is detected as a result of the position specification, it is determined that the finger has been pushed and deleted.

<Sequence diagram>
FIG. 13 is an example of a sequence diagram showing the operation of the electronic whiteboard shown in FIG.

  The operation of the coordinate detection system 201 (coordinate detection method) includes an initialization and operation start process stage, a process stage performed only when using the electronic pen, and a process stage performed at regular intervals in the coordinate detection system; Have

The instructions of the electronic pen 107 and the finger 108 and the timing of the instruction end are asynchronous, but the detection process is performed simultaneously.
(Initialization and operation start processing stage)
The coordinate detection system 201 performs detection LED lighting, correction table creation, and brightness adjustment (step S1).

The coordinate detection system 201 starts detection (step S2).
(Processing stage only when using the electronic pen)
A coordinate detection area on the screen 102 (see FIG. 1) of the display device of the coordinate detection system 201 is designated by the electronic pen 107 and the finger 108 (steps S3 and S4).

  Drawing is detected by pressing the electronic pen 107 (step S5), the nib LED of the electronic pen 107 is turned on, and the luminance is measured (step S6).

  When the electronic pen 107 notifies the coordinate detection system 201 of the drawing state and the pen tip LED brightness (notified at regular intervals during drawing: step S7), the coordinate detection system 201 adjusts the brightness of the detection LED (step S7). S8).

  The coordinate detection system 201 acquires a camera image, corrects it with a correction table (step S9), and discriminates a bright part, an electronic pen 107, a dark part, and a finger 108 with a threshold value (step S10).

  The coordinate detection system 201 specifies the positions (coordinates) of the electronic pen 107 and the finger 108 (step S11), and deletes finger information as position information of the finger 108 near the electronic pen 107 (step S12).

  The coordinate detection system 201 compares the position information of the electronic pen 107 being tracked with the position information of the finger 108 (if the positions are close, it is determined that they are the same: step S13), and the position information of the electronic pen 107 and the finger 108 whose positions are newly specified Is stored (newly tracked) (step S14).

  The coordinate detection system 201 deletes the information of the extinguished electronic pen 107 and the finger 108 (indicating that the instruction has disappeared: step S15), specifies the indicated position of the electronic pen 107 (step S16), and instructs the finger 108 The position is specified (step S17).

  The finger 108 sends an instruction end signal to the coordinate detection system 201 (step S18), the electronic pen 107 sends an instruction end signal to the coordinate detection system 201 (step S19), and the electronic pen 107 finishes drawing when pressed. Is detected (step S20).

  The electronic pen 107 notifies the coordinate detection system 201 of the drawing state (drawing end notification: step S21).

<Program>
The coordinate detection system according to the present invention described above is realized by a program that causes a computer to execute processing. Therefore, as an example, a case where the function of the present invention is realized by a program will be described below.

For example,
A computer-readable program used in a coordinate detection system that detects coordinates indicated by an electronic pen or a finger that performs an instruction operation on a display screen,
A procedure in which the first light irradiation means irradiates light to a coordinate detection region for detecting coordinates from the outer periphery of the display screen;
A procedure in which the second light irradiation means emits light from the electronic pen;
A procedure in which the first light detection means detects the light emitted from the first light irradiation means;
A procedure in which the second light detection means detects the light emitted from the second light irradiation means;
A procedure for determining whether the determination unit is the electronic pen or the finger according to a brightness level based on luminance information relating to the luminance of light detected by the first light detection unit;
A procedure for specifying a position of the electronic pen or the finger determined by the determination means;
A program for executing

  Such a program may be stored in a computer-readable storage medium.

<Storage medium>
Here, examples of the storage medium include computer-readable storage media such as CD-ROM, flexible disk (FD), and CD-R, semiconductor memories such as flash memory, RAM, ROM, and FeRAM, and HDD.

  CD-ROM is an abbreviation for Compact Disc Read Only Memory. Flexible disk means Flexible Disk (FD). CD-R is an abbreviation for CD Recordable. FeRAM is an abbreviation for Ferroelectric RAM and means a ferroelectric memory. HDD is an abbreviation for Hard Disc Drive.

<Effect>
According to the present embodiment, by using an optical system, it is possible to avoid the influence of luminance on the screen, and to cope with the screen size as it is.

  The detection of the light source mounted on the electronic pen and the finger detection for detecting the shadow of the light source inside the screen frame are combined and performed simultaneously.

  The brightness of the light source mounted on the electronic pen is made higher than the brightness of the light source inside the screen frame, so that the brightness is different.

  In the same camera image, the light source of the pen is distinguished from the light source inside the screen frame by having a binary luminance threshold value.

  As long as all the light sources inside the screen frame have the same luminance, the luminance detected by the camera decreases as the distance from the camera increases (the closer to the diagonal). If the brightness of the light source inside the screen frame looks constant when viewed from the camera, it is easier to distinguish the brightness difference from the light source of the electronic pen by using the threshold value. .

  The light source of the electronic pen may be weakened due to the influence of a decrease in the power (battery) voltage of the electronic pen or the variation in characteristics of the light source device (LED). For this reason, the brightness of the light source inside the screen frame is adjusted so that the light source of the electronic pen can be easily detected by measuring the brightness of the light source inside the electronic pen and transmitting it to the main body.

  A structure related to coordinate detection is unnecessary on the display surface, and it can be retrofitted by separating it from the display device.

  As described above, in a coordinate detection system that detects the coordinates of an electronic pen or a finger, it is possible to simultaneously detect an instruction with the electronic pen and detect an instruction with the finger, and a large-screen system such as an electronic whiteboard with reduced manufacturing costs. Compatible with a retrofit coordinate detection system.

  The above-described embodiment shows an example of a preferred embodiment of the present invention, and the present invention is not limited thereto, and various modifications can be made without departing from the scope of the invention. is there. That is, in the above-described embodiment, the case of the display device has been described. However, it is needless to say that the present invention is not limited to this and may be a projector device.

[Second Embodiment]
In the first embodiment, the coordinate detection system 201 uses the wireless Md 123 to acquire information such as pen identification information and press detection information transmitted from the electronic pen 107. In this case, before the electronic pen 107 is used, a pairing operation for performing wireless communication between the coordinate detection system 201 and the electronic pen 107 is required.

  In the second embodiment, by making it possible to identify drawing data drawn by a plurality of electronic pens 107 without performing a pairing operation between the coordinate detection system 201 and the electronic pen 107, the user's An example of the coordinate detection system 201 that improves convenience will be described.

<Configuration>
FIG. 14 is an example of an external view of an electronic whiteboard body according to the second embodiment. The electronic whiteboard main body 101 according to the second embodiment is similar to the electronic whiteboard main body 101 according to the first embodiment shown in FIG. 1B, and includes a screen 102, a camera L103, a camera R104, and an infrared LED. It has arrays 105L, 105C, 105R, etc.

  The electronic whiteboard body 101 according to the second embodiment includes one or more photodiodes 1401-1 to 1401-6 around the screen 102. In the following description, “photodiode 1401” is used to indicate an arbitrary photodiode among the one or more photodiodes 1401-1 to 1401-6.

  The photodiode 1401 is an example of a light receiving element or a light receiving device for optical communication. For example, the photodiode 1401 may be another light receiving element such as a phototransistor or a light receiving device such as an infrared light receiving module.

  In the present embodiment, the electronic pen 107 transmits pen identification information of the electronic pen 107, information indicating a drawing state, and the like to the coordinate detection system 201 by an optical communication signal using light of the infrared LED 146 that is emitted during drawing. . For example, the electronic pen 107 transmits an optical communication signal by blinking light of the infrared LED 146 that emits light at the time of drawing.

  On the other hand, the coordinate detection system 201 receives an optical communication signal transmitted from the electronic pen 107 using the optical communication photodiode 1401, and displays pen identification information and a drawing state included in the received optical communication signal. The information shown is acquired.

  For example, as shown in FIG. 1, the photodiodes 1401 for optical communication have a plurality (six in the example of FIG. 1) of photodiodes 1401-1 to 1401- in order to improve detection accuracy of optical communication signals. It is desirable that 6 is installed. The number of photodiodes 1401 shown in FIG. 1 is an example, and may be one or more other numbers.

<Hardware configuration>
(Coordinate detection system)
FIG. 15 is an example of a hardware block diagram of a coordinate detection system according to the second embodiment. The coordinate detection system 201 according to the second embodiment includes a microcomputer 1501, one or more photodiodes 1401, and a storage device in addition to the hardware configuration of the coordinate detection system 201 according to the first embodiment shown in FIG. 1503 etc. Note that the coordinate detection system 201 according to the second embodiment may not include the wireless Md 123 illustrated in FIG.

  The other hardware configuration of the coordinate detection system 201 according to the second embodiment shown in FIG. 15 is the same as the hardware configuration of the coordinate detection system 201 according to the first embodiment shown in FIG. The description will focus on the differences from the embodiment.

  A microcomputer (microcomputer) 1501 is a device including a general computer configuration such as a CPU, RAM, and flash ROM. For example, the microcomputer 1501 receives a signal of optical communication using the photodiode 1401 by executing a program stored in a built-in flash ROM or the like by the CPU, and includes pen identification information and a drawing state included in the received signal. A function for acquiring information indicating the above is realized. Note that the function realized by the microcomputer 1501 may be realized by a program executed by the CPU 120.

  The photodiode 1401 is a light receiving element for optical communication as described above in the description of FIG.

  Preferably, the optical communication photodiode 1401 is provided with an optical filter 1502 that transmits an optical communication signal (light) transmitted from the electronic pen 107. This is because if the light emitted from the infrared LED arrays 105L, 105C, and 105R enters the photodiode 1401 for optical communication, it may be difficult to obtain an optical communication signal.

  Therefore, it is desirable that the wavelength of the light emitted from the infrared LED arrays 105L, 105C, and 105R is different from the wavelength of the optical communication light transmitted from the electronic pen 107. Accordingly, the photodiode 1401 can attenuate the light emitted from the infrared LED arrays 105L, 105C, and 105R using the optical filter 1502.

  On the other hand, the electronic pen 107, or the camera L103 and camera R104 for detecting the finger pointing position, the wavelength of the light emitted from the infrared LED arrays 105L, 105C, and 105R and the optical communication light transmitted from the electronic pen 107 Both wavelengths are detected.

  The reason why the camera L103 and the camera R104 are not used for detecting the light of the optical communication transmitted from the electronic pen 107 is that the frame rate of the camera L103 and the camera R104 is generally about 60 fps (at most, about 120 fps). It is because it is low. For example, if optical communication is performed in accordance with the frame rates of the camera L103 and the camera R104, since the extinguished bit cannot detect the indication position of the electronic pen 107, the detection rate of the indication position of the electronic pen 107 decreases. There's a problem.

  On the other hand, if the frame rate of the camera L103 and camera R104 is increased to a level that does not affect the detection rate of the indication position of the electronic pen 107, the cost of the camera L103 and camera R104 and the load related to image processing increase. There is.

  By using the photodiode 1401 for detecting light of optical communication transmitted from the electronic pen 107, it is possible to cope with a communication rate of several MHz or more with a relatively low cost and simple circuit configuration.

  The storage device 1503 is a storage device that holds a large amount of data, such as an HDD (Hard Disk Drive), an SSD (Solid Staste Drive), or a flash ROM.

(Electronic pen)
FIG. 16 is a diagram illustrating an example of a hardware configuration of the electronic pen according to the second embodiment.

  FIG. 16A shows an image of the appearance of the electronic pen 107 according to the second embodiment.

  The electronic pen 107 has a light source 1611 provided with an infrared LED 146 inside the pen tip. The electronic pen 107 detects that the pen tip has been pressed against the screen 102 with the pressure sensor 1411, and causes the infrared LED 146 to emit light when the user is performing a drawing operation. Further, the electronic pen 107 according to the second embodiment performs optical communication by blinking the infrared LED 146 at a high speed when the user performs a drawing operation, for example, pen identification information of the electronic pen 107 and Information indicating the drawing state is transmitted.

  Preferably, the electronic pen 107 includes a light receiving unit 1612 including a photodiode for detecting an optical communication signal output from the other electronic pen 107.

  FIG. 16B shows an example of a hardware block diagram of the electronic pen 107 according to the second embodiment. The electronic pen 107 according to the second embodiment includes, for example, a microcomputer 141, a battery 143, a pressure sensor 144, an infrared LED 146, a ROM 1601, a photodiode 1602, and the like. The microcomputer 141, the battery 143, the pressure sensor 144, and the infrared LED 146 are the same as those included in the electronic pen 107 according to the first embodiment shown in FIG. The description will focus on differences from the hardware configuration of the embodiment.

  The ROM 1601 is a nonvolatile memory and stores, for example, pen identification information of the electronic pen 107 and the like. The electronic pen 107 may store the pen identification information of the electronic pen 107 in a flash ROM or the like built in the microcomputer 141 instead of the ROM 1601.

  The photodiode 1602 is a light receiving element that detects an optical communication signal (light) output from another electronic pen 107.

  Preferably, the photodiode 1602 transmits an optical communication signal (light) transmitted from another electronic pen 107, and attenuates light output from the infrared LED arrays 105L, 105C, and 105R of the coordinate detection system 201. An optical filter 1603 is provided.

<Functional block diagram>
(Electronic whiteboard)
FIG. 17 is an example of a functional block diagram of an electronic whiteboard according to the second embodiment. In addition to the functional block diagram of the electronic whiteboard 200 according to the first embodiment shown in FIG. 3, the electronic whiteboard 200 according to the second embodiment includes an optical signal receiving means 1701, an information extracting means 1702, and a drawing data management. Means 1703 and storage means 1704 are provided. Note that the electronic whiteboard 200 according to the second embodiment may not include the communication unit 117 shown in FIG. Further, the other functional blocks are the same as the functional blocks of the electronic whiteboard 200 according to the first embodiment shown in FIG. 3, and therefore, here, the description will focus on differences from the first embodiment.

  The optical signal receiving unit 1701 is a unit that receives an optical communication signal in which the electronic pen 107 performs communication using the light emitted using the infrared LED 146. The optical signal receiving unit 1701 is realized by, for example, a program executed by the photodiode 1401 and the microcomputer 1501 (or the CPU 120) in FIG.

  The information extraction unit 1702 is a unit that extracts information (for example, pen identification information, information indicating a drawing state, etc.) included in an optical communication signal received by the optical signal receiving unit 1701. For example, the microcomputer 1501 in FIG. It is realized by a program executed by (or CPU 120).

  In the present embodiment, the data transmission method by optical communication is not particularly limited. For example, standardized standards such as IrDA (registered trademark) (Infrared Data Association) may be used, or unique data transmission may be performed. What uses a method etc. may be used. For example, a data transmission method using optical communication may be one that represents digital values “1” / “0” by turning on / off light of a predetermined wavelength. In this case, the information extraction unit 1702 can acquire information included in a signal of optical communication, for example, by determining the presence or absence of light having a predetermined wavelength at a predetermined sampling rate.

  The drawing data management means 1703 associates the pen identification information of the electronic pen 107 extracted by the information extraction means 1702 with the position (indicated position) of the electronic pen 107 specified by the position specifying means 118, for example, stores The information is stored in the means 1704 and managed. The drawing data management unit 1703 is realized by, for example, a program executed by the CPU 120 (or the microcomputer 1501) in FIG.

  The storage means 1704 is means for storing data (hereinafter referred to as drawing data) managed by the drawing data management means 1703, and is executed by, for example, the storage device 1503 in FIG. 15 and the CPU 120 (or microcomputer 1501) in FIG. This is realized by a program to be executed.

(Electronic pen)
FIG. 18 is an example of a functional block diagram of the electronic pen according to the second embodiment. The electronic pen 107 according to the second embodiment includes a control unit 131, a battery unit 133, a pressure detection unit 134, an optical signal detection unit 1801, an optical communication control unit 1802, a light emission unit 1803, an identification information storage unit 1804, and the like. . Among the functional blocks described above, the functions of the control unit 131, the battery unit 133, and the pressure detection unit 134 are included in the electronic pen 107 according to the first embodiment shown in FIG. The same as the means 133 and the pressure detection means 134. Here, the description will focus on the differences from the first embodiment.

  The optical signal detection means 1801 is a means for detecting a signal of optical communication by the other electronic pen 107, for example, a program executed by the photodiode 1602 in FIG. 16B and the microcomputer 141 in FIG. Etc. For example, the optical signal detection unit 1801 uses the photodiode 1602 to determine whether the light output from the infrared LED 146 of the other electronic pen 107 includes an optical communication signal.

  The optical communication control unit 1802 is a unit that controls the timing at which the electronic pen 107 transmits an optical communication signal based on the detection result of the optical communication signal by the optical signal detection unit 1801. For example, FIG. This is realized by a program executed by the microcomputer 141.

  Preferably, when the electronic pen 107 starts transmitting an optical communication signal, the optical communication control unit 1802 detects an optical communication signal from the other electronic pen 107 by the optical signal detection unit 1801, and the other electronic pen When the optical communication signal 107 is detected, the transmission of the optical communication signal is delayed. For example, when an optical communication signal is detected by another electronic pen 107 when transmitting an optical communication signal, the optical communication control unit 1802 waits for a predetermined time and then transmits the optical communication signal. Try again.

  Further, when transmitting an optical communication signal, the optical communication control unit 1802 turns on / off the light output from the light emitting unit 1803 at a high speed, for example, when the optical communication signal from the other electronic pen 107 is not detected. Therefore, an optical communication signal is transmitted. The optical communication signal output by the optical communication control unit 1802 includes, for example, pen identification information of the electronic pen 107, information indicating a drawing state of the electronic pen, and the like.

  The light emitting unit 1803 is a unit that causes the infrared LED 146 provided in the light source 1611 of the electronic pen 107 to emit light. For example, the infrared LED 146, the microcomputer 141, and the program executed by the microcomputer 141 in FIG. Etc.

  For example, when the pen tip of the electronic pen 107 is pressed against the screen 102, the light emitting unit 1803 starts the emission of the infrared LED 146 according to the control of the control unit 131, and the infrared LED 146 according to the control of the optical communication control unit 1802. Turn on / off. Further, the light emitting means 1803 stops the light emission of the infrared LED 146 according to the control of the control means 131 when the pen tip of the electronic pen 107 leaves the screen 102.

  The identification information storage unit 1804 is a storage unit that pre-stores pen identification information that is identification information for identifying the electronic pen 107. For example, the ROM 1601 in FIG. This is realized by a memory (for example, a flash ROM).

<Process flow>
Next, the flow of processing by the coordinate detection system 201 according to the second embodiment will be described.

(Handling electronic pen)
FIG. 19 is a flowchart illustrating an example of processing of the electronic pen according to the second embodiment.

  In step S1901, when the electronic pen 107 detects that the control unit 131 is drawing using the press detection unit 134, the electronic pen 107 executes the processing from step S1902 onward.

  In step S1902, the control unit 131 of the electronic pen 107 uses the light emitting unit 1803 to start light emission of the infrared LED 146 provided in the light source 1611 at the pen tip.

  In step S1903, the optical communication control unit 1802 of the electronic pen 107 uses the detection result of the optical signal detection unit 1801 to determine whether another electronic pen 107 is in optical communication.

  If another electronic pen 107 is not in optical communication, the optical communication control unit 1802 shifts the processing to step S1904. On the other hand, when another electronic pen 107 is in optical communication, the optical communication control unit 1802 shifts the processing to step S1908.

  In step S1904, the optical communication control unit 1802 uses the light emitting unit 1803 to turn on / off the infrared LED 146 to perform optical communication, whereby the pen identification information stored in the identification information storage unit 1804 and the drawing state are displayed. A signal including information indicating (during drawing) is transmitted.

  In step S1905, the optical communication control unit 1802 waits for a first time (T1-T). T1 is a predetermined transmission period of the optical communication signal, and T is a time required for transmitting the optical communication signal. The transmission cycle T1 and the time T required for transmission will be described later with reference to FIG.

  In step S <b> 1906, the control unit 131 of the electronic pen 107 determines whether the drawing has been detected by the press detected by the press detection unit 134.

  When it is not detected that the drawing has been completed, the control unit 131 of the electronic pen 107 shifts the process to step S1903 and again executes the same process. On the other hand, when the end of drawing is detected, in step S1907, the control unit 131 of the electronic pen 107 stops the light emission of the infrared LED 146 provided in the light source 1611 of the pen tip by the light emitting unit 1803.

  When the process proceeds from step S1903 to step S1908, in step S1908, the optical communication control unit 1802 waits for a second time (T2), and then returns the process to step S1903. T2 is a predetermined standby time.

  Here, the transmission cycle T1, the time T required for transmission, and the standby time T2 will be described with reference to FIG.

  FIG. 20 is a diagram for explaining communication timing of the electronic pen according to the second embodiment. In FIG. 20, the horizontal axis represents time, and “during communication” indicates a period during which the electronic pens 1 to 3 are transmitting optical communication signals.

  In FIG. 20, the electronic pens 1 to 3 basically transmit an optical communication signal at a transmission cycle T1, and it is assumed that time T is required for transmitting the optical communication signal. In this case, after the electronic pens 1 to 3 complete the transmission of the optical communication signal, the first time that is a waiting time until the next transmission of the optical communication signal is started is T1-T.

  In addition, when the electronic pens 1 to 3 transmit optical communication signals, it is determined whether or not the other electronic pens are “in communication”, and if the other electronic pens are not “in communication”, Starts transmission of optical communication signals. In addition, when the other electronic pens are “communication”, the electronic pens 1 to 3 stop transmitting the optical communication signal, wait for the waiting time T2 (second time), and then wait for the optical communication signal. Retry transmission.

  For example, at time t1 in FIG. 20, when the electronic pen 1 transmits an optical communication signal, the electronic pen 1 determines whether or not there is an optical communication signal output from the other electronic pens 2 and 3. Determine whether the electronic pen is in optical communication. The state where there is an optical communication signal indicates a state in which data is transmitted by turning on / off light emitted from the electronic pen 107 at high speed, and the electronic pen 107 simply emits light. The state is not included.

  In the example of FIG. 20, since the other electronic pens 2 and 3 are not “communicating” at the time t1, the electronic pen 1 starts transmitting an optical communication signal. In addition, the electronic pen 1 ends the transmission of the optical communication signal at time t3 when the time T required for transmission has elapsed from time t1. Further, the electronic pen 1 executes the transmission process of the optical communication signal again at the time t5 when the first time (T1-T) has elapsed from the time t3.

  In addition, at time t2 in FIG. 20, when the electronic pen 2 transmits an optical communication signal, the electronic pen 2 determines whether the other electronic pens 1 and 3 are in optical communication.

  In the example of FIG. 20, since the electronic pen 1 is “communication” at the time t2, the electronic pen 2 stops transmitting the optical communication signal, waits for the waiting time T2, and again at the time t4 It is determined whether the electronic pens 1 and 3 are in optical communication.

  In the example of FIG. 20, since the other electronic pens 1 and 3 are not “communicating” at the time t4, the electronic pen 2 starts transmitting an optical communication signal. After starting transmission of the optical communication signal, the electronic pen 2 performs processing similar to that of the electronic pen 1 at time t7 when the first time (T1-T) has elapsed from time t4. The transmission process is executed again.

  At time t5 in FIG. 20, when the electronic pen 3 transmits an optical communication signal, the electronic pen 3 determines whether there is an optical communication signal output from the other electronic pens 1 and 2. Determines whether the optical communication is in progress.

  In the example of FIG. 20, since the other electronic pens 1 and 2 are not “communicating” at the time t5, the electronic pen 3 starts transmitting an optical communication signal. After starting transmission of the optical communication signal, the electronic pen 3 performs processing similar to that of the electronic pen 1 at time t8 when the first time (T1-T) has elapsed from time t5. The transmission process is executed again.

  By repeating the processing as described above, the electronic pen 107 according to the second embodiment avoids interference with the optical communication of the other electronic pen 107, and the optical communication signal includes pen identification information at a predetermined period. Can be sent.

  Preferably, the communication rate of optical communication by the electronic pen 107 is set to a value (for example, 1 MHz) sufficiently higher than the frame rate (for example, 60 Hz) of the camera L103 and the camera R104, thereby reducing the influence on the position detection.

  For example, if the communication rate of optical communication by the electronic pen 107 is 1 MHz and 1 data is 2 bytes, the time required to transmit 1 data is about 16 μs (1/1000 frames), so the influence on position detection is almost eliminated. Can be ignored.

  If the frame rate of the camera L103 and camera R104 is 60 Hz, one frame takes about 16 ms, so if the transmission cycle is less than about 5 ms, optical communication is performed two or three times during one frame. Is transmitted. Therefore, the coordinate detection system 201 can acquire the next data even if the optical communication signal is missed once.

(Processing of coordinate detection system)
FIG. 21 is a flowchart illustrating an example of processing of the coordinate detection system according to the second embodiment. The coordinate detection system 201 according to the second embodiment acquires the drawing data drawn by the electronic pen 107 by repeatedly executing the processing shown in FIG. 21, and stores it in association with the pen identification information of the electronic pen 107. The information is stored in the means 1704 and managed.

  The method for detecting the designated position (coordinates) using the electronic pen 107 or the finger is the same as in the first embodiment, and a detailed description thereof is omitted here.

  In step S2101, the control means 111 of the coordinate detection system 201 acquires a camera image using the first light detection means 113 and the second light detection means 114 (hereinafter referred to as light detection means).

  In step S2102, the determination unit 116 of the coordinate detection system 201 determines a bright part (electronic pen 107) and a dark part (finger) based on the acquired camera image.

  In step S2103, the position specifying unit 118 of the coordinate detection system 201 specifies the electronic pen 107 or finger pointing position (coordinates) determined by the determining unit 116.

  Preferably, in step S2104, the position specifying unit 118 of the coordinate detection system 201 is within a predetermined range from the indicated position of the electronic pen 107 specified in step S2103 among the specified positions of the finger specified in step S2103. The information on the finger pointing position is deleted.

  In step S2105, the drawing data management unit 1703 of the coordinate detection system 201 determines the distance between the designated position (coordinates) of the electronic pen 107 specified in step S2103 and the designated position (coordinates) of the electronic pen 107 that is being tracked. Is calculated.

  In step S2106, the drawing data management unit 1703 of the coordinate detection system 201 determines whether the distance calculated in step S2105 is within a predetermined distance.

  If the calculated distance is within the predetermined distance, the drawing data management unit 1703 shifts the process to step S2107. On the other hand, if the calculated distance is not within the predetermined distance, the drawing data management unit 1703 shifts the process to step S2110. Here, the predetermined distance is determined in advance based on the time interval for executing the processing of FIG.

  In step S2107, the drawing data management unit 1703 of the coordinate detection system 201 determines that the designated position of the electronic pen 107 identified in step S2103 is the designated position of the electronic pen 107 that is being tracked.

  In step S2108, the drawing data management unit 1703 of the coordinate detection system 201 stores the coordinates of the specified position of the specified electronic pen in the drawing data of the electronic pen 107 that is being tracked.

  Table 1 shows an example of the drawing data that the drawing data management unit 1703 stores and manages in the storage unit 1704.

As shown in Table 1, the drawing data includes, for example, information such as “pen identification information”, “drawing data”, “writing date / time”, and “state”.

  “Pen identification information” is identification information for identifying the electronic pen 107.

  “Drawing data” is one or more coordinates indicating the indicated position of the electronic pen 107. For example, when new drawing data by the electronic pen 107 is detected, the drawing data management unit 1703 stores the first coordinates, for example, (X1, Y1) in “drawing data”. Further, the drawing data management unit 1703 stores, as a second coordinate, for example, (X2, Y2), the position indicated by the electronic pen 107 detected within the predetermined distance described above within a predetermined time.

  As described above, the drawing data management unit 1703, for example, determines that the designated position by the electronic pen 107 detected within a predetermined distance within a predetermined time is the same drawing data, and sequentially stores them.

  “Writing date and time” is information indicating the time when the latest coordinates included in “drawing data” are written, for example. For example, the drawing data management unit 1703 sets the “state” to “confirmed” when the new pointing position by the electronic pen 107 is not detected within a predetermined distance within a predetermined time from the “writing date”. Finalize data tracking.

  The “state” is information indicating whether the drawing data has been confirmed or is being drawn (tracking).

  Returning to FIG. 21, the description of the flowchart will be continued.

  In step 2109, the drawing data management means 1703 of the coordinate detection system 201 determines tracking of the electronic pen 107 that has not been detected for a predetermined time. For example, in Table 1, the “state” corresponding to the pen identification information of the electronic pen 107 is set to “confirmed”.

  If the calculated distance is not within the predetermined distance in step S2106, the drawing data management means 1703 of the coordinate detection system 201 starts tracking as new drawing data in step S2110. For example, the drawing data management unit 1703 creates a new entry in the drawing data as shown in Table 1, and stores the coordinates indicating the designated position of the electronic pen 107 specified in step S2103 in “drawing data”.

  In step S2111, the drawing data management unit 1703 of the coordinate detection system 201 manages the newly detected pen identification information notified from the information extraction unit 1702 in association with the drawing data newly created in step S2106. . For example, the drawing data management unit 1703 stores the newly detected pen identification information in the “pen identification information” corresponding to the “drawing data” stored in step S2106.

  In the coordinate detection system 201 according to the second embodiment, the processes shown in steps S2121 to S2123 are executed in parallel with the processes shown in steps S2101 to S2111 in FIG.

  In step S2121, the optical signal receiving unit 1701 of the coordinate detection system 201 acquires an optical communication signal output from the electronic pen 107, and the information extracting unit 1702 displays the pen identification information and the drawing state included in the acquired signal. The information shown is extracted.

  In step S2122, the information extraction unit 1702 of the coordinate detection system 201 determines whether there is newly detected pen identification information in the extracted pen identification information.

  If there is no newly detected pen identification information in the extracted pen identification information, the information extraction unit 1702 returns the process to step S2121 and repeats the same process. On the other hand, when the pen identification information newly detected is included in the extracted pen identification information, the information extraction unit 1702 shifts the processing to step S2123.

  In step S2123, the information extraction unit 1702 of the coordinate detection system 201 notifies the drawing data management unit 1703 of the newly detected identification information of the electronic pen 107.

  Through the above processing, the coordinate detection system 201 can manage the drawing data newly started to be tracked and the newly detected pen identification information of the electronic pen 107 in association with each other.

(Processing of coordinate detection system)
In the above description, management of drawing data by the electronic pen 107 has been described. However, the coordinate detection system 201 according to the second embodiment also manages drawing data by a finger in the same manner as in the first embodiment. be able to.

  FIG. 22 is a sequence diagram illustrating an example of processing of the coordinate detection system according to the second embodiment. Note that the basic process is the same as the process according to the first embodiment shown in FIG.

  In step S2201, the coordinate detection system 201 performs detection LED lighting and luminance adjustment.

  In step S2202, the coordinate detection system 201 starts detection.

  In step S2203, the electronic pen 2 designates a coordinate detection area on the screen 102 of the display device of the coordinate detection system 201. For example, a user who writes on the electronic whiteboard 200 writes on the screen 102 using the electronic pen 2.

  In step S2204, the electronic pen 2 detects that drawing is being performed by pressing, and in step S2205, the infrared LED 146 starts to emit light.

  Similarly, in step S2206, the electronic pen 1 designates a coordinate detection area on the screen 102 of the display device of the coordinate detection system 201. For example, another user who writes on the electronic whiteboard 200 writes on the screen 102 using the electronic pen 1.

  In step S2207, the electronic pen 1 detects that drawing is being performed by pressing, and in step S2208, the infrared LED 146 starts to emit light.

  The electronic pens 1 and 2 repeatedly execute the electronic pen processing 2210 shown in FIG. The electronic pen process 2210 includes, for example, processes shown in steps S2211 to S2214.

  In step S2211, the electronic pen 1 detects the state of optical communication with another electronic pen (for example, the electronic pen 2), and when the other electronic pen is in optical communication, executes the processing of steps S2212, S2213. . For example, in step S2212, the optical communication control unit 1802 of the electronic pen 1 transmits a signal including a drawing state and pen identification information through optical communication, and waits for the first time (T1-T) described above in step S2213.

  On the other hand, in step S2211, the electronic pen 1 detects the state of optical communication with another electronic pen, and when the other electronic pen is not in optical communication, executes the process of step S2214. For example, in step S2214, the optical communication control unit 1802 of the electronic pen 1 waits for the second time T2 described above.

  Note that the electronic pen processing 2210 in FIG. 22 corresponds to the processing in steps S1903 to S1906 and step S1908 in FIG. 19, for example.

  Similarly, the electronic pen 2 repeatedly executes the electronic pen process 2210.

  On the other hand, when the process is started, the coordinate detection system 201 repeatedly executes the process 2220 of the coordinate detection system shown in FIG. Note that the processing 2220 of the coordinate detection system includes, for example, the processing shown in steps S2221 to S2229.

  In step S2221, the coordinate detection system 201 acquires a camera image.

  In step S <b> 2222, the coordinate detection system 201 uses a threshold value to determine a bright part indicating the instruction position by the electronic pens 1 and 2 and a dark part indicating the instruction position by the finger 108.

  In step S <b> 2223, the coordinate detection system 201 specifies the designated positions (coordinates) of the electronic pens 1 and 2 and the finger 108.

  In step S <b> 2224, the coordinate detection system 201 deletes finger information that is information on the designated position of the finger 108 near the designated position of the electronic pens 1 and 2.

  In step S2225, the coordinate detection system 201 compares the indication positions (coordinates) of the electronic pens 1 and 2 and the finger 108 with the coordinates of the electronic pen 107 and the finger 108 that are being tracked. to decide.

  In step S2226, the coordinate detection system 201 stores the new designated position among the designated positions of the electronic pens 1 and 2 and the finger 108 as new drawing data, and newly starts tracking.

  In step S2227, the coordinate detection system 201 manages the instruction position of the electronic pen 107 stored as new drawing data in association with the newly received pen identification information. Note that, for example, identification information indicating the finger 108 or new identification information is generated and assigned to the indicated position of the finger 108 stored as new drawing data.

  In step S2228, the coordinate detection system 201 deletes the tracking information of the electronic pen 107 and the finger 108 that are no longer detected, and confirms the tracking.

  By the above processing, for example, in steps S2229 and S2230, the coordinate detection system 201 can specify the indication positions of the electronic pen 1 and the electronic pen 2 and the pen identification information.

  Note that the processing 2220 of the coordinate detection system in FIG. 22 is obtained by, for example, adding the processing for the finger 108 to the processing of the coordinate detection system shown in FIG.

  In step S2231, for example, when the instruction (drawing) on the screen 102 of the display device of the coordinate detection system 201 by the electronic pen 1 is completed, in step S2232, the electronic pen 1 detects the drawing end by pressing.

  In step S2233, the electronic pen 1 stops the emission of the infrared LED 146.

  Through the above processing, the coordinate detection system 201 according to the second embodiment can perform drawing data drawn by the plurality of electronic pens 107 without performing a pairing operation between the coordinate detection system 201 and the electronic pen 107. This makes it possible to identify the user and improve user convenience.

  The above-described embodiment shows an example of a preferred embodiment of the present invention, and the present invention is not limited thereto, and various modifications can be made without departing from the scope of the invention. is there. For example, in the above-described embodiment, the case of a display device has been described. However, it is needless to say that the present invention is not limited to this and may be a projector device.

101 Electronic whiteboard body 102 Screen (display means)
103 Camera L
104 Camera R
105C, 105L, 105R Infrared LED array 107 Electronic pen 108 Finger 111, 131 Control means 112 First light irradiation means 113 First light detection means 114 Second light detection means 115 Detection level correction means 116 Determination means 117, 132 communication means 118 position specifying means 119 adjustment means 120 CPU
121 ROM
122 RAM
123, 142 Wireless Md
124 I / F
133 Battery means 134 Press detection means 135 Signal intensity detection means 136 Signal transmission means 141 Microcomputer 143 Battery 144 Pressure sensor 145 Infrared sensor 146 Second light irradiation means (infrared LED)
160 Leg 200 Electronic whiteboard (image processing device)
201 Coordinate Detection System 1401 Photodiode 1502 Optical Filter 1602 Photodiode 1603 Optical Filter 1701 Optical Signal Receiving Unit 1702 Information Extracting Unit 1703 Drawing Data Management Unit 1801 Optical Signal Detection Unit 1802 Optical Communication Control Unit

JP 2012-022543 A

Claims (19)

A coordinate detection system that detects coordinates indicated by an electronic pen or a finger that performs an instruction operation on a display screen,
First light irradiation means for irradiating light to a coordinate detection region for detecting coordinates from the outer periphery of the display screen;
A plurality of light detection means for detecting the light emitted from the first light irradiation means and the light emitted from the second light irradiation means emitting light from the electronic pen;
A determination unit that determines whether the electronic pen or the finger is in accordance with a brightness level based on luminance information related to the luminance of light detected by the light detection unit;
Position specifying means for specifying the position of the electronic pen or the finger determined by the determining means;
A coordinate detection system comprising:   The coordinate detection system according to claim 1, wherein the first and second light irradiating means and the light detecting means for detecting the light irradiate and detect infrared light.   The coordinate detection system according to claim 1, wherein the light detection means is an optical camera.   The coordinate detection system according to claim 1, wherein the determination unit has two or more threshold values at a light / dark level according to a light / dark level based on luminance information.   The coordinate detection system according to claim 1, wherein the identification information of the pen is linked to the detected electronic pen or the pointing point of the finger for tracking.   2. The electronic pen according to claim 1, further comprising: a transmission unit configured to transmit pen identification information, press detection information, and light source luminance information of the second light irradiation unit to the display apparatus body using wireless communication. The coordinate detection system described.   The coordinate system according to claim 1, further comprising an adjusting unit that adjusts the light emission luminance of the first light irradiation unit based on light source luminance information of the second light irradiation unit sent from the electronic pen. Detection system.   The coordinate detection system according to claim 1, further comprising a detection level correction unit that corrects a detection level at the time of detecting the instruction unit based on a luminance level detected by the light detection unit. An optical signal receiving means for receiving an optical communication signal for performing communication using light emitted from the second light irradiating means of the electronic pen;
Information extracting means for extracting pen identification information, which is identification information for identifying the electronic pen, included in the optical communication signal received by the optical signal receiving means;
Drawing data management means for managing the pen identification information of the electronic pen extracted by the information extraction means in association with the coordinates indicating the position of the electronic pen specified by the position specifying means;
The coordinate detection system according to claim 1, comprising: The coordinate detection system detects coordinates indicated by the one or more electronic pens;
The electronic pen is
An optical signal detection means for detecting a signal of the optical communication by light emitted from the second light irradiation means of another electronic pen;
An optical communication control means for controlling the timing of transmitting the optical communication signal of the electronic pen based on the detection result of the optical communication signal by the optical signal detection means;
The coordinate detection system according to claim 9. The optical communication control means includes
When the transmission of the optical communication signal is started, the optical signal detected by the other electronic pen is detected by the optical signal detection means, and the optical communication signal is detected by the other electronic pen. The coordinate detection system according to claim 10, wherein transmission of the optical communication signal is stopped, and a predetermined time is waited.   The coordinate detection system according to claim 10 or 11, wherein the light emitted from the second light irradiation means has a wavelength different from that of the light emitted from the first light irradiation means. The optical signal detection means includes
The coordinate detection system according to claim 12, wherein the coordinate detection system is a photodiode provided with an optical filter that transmits light emitted from the second light irradiation unit. The optical signal receiving means includes
The coordinate detection system according to claim 12 or 13, wherein the optical communication signal is received using a photodiode provided with an optical filter that transmits light emitted from the second light irradiation means. The optical communication signal is:
The coordinate detection system according to claim 11, comprising pen identification information of the electronic pen and information indicating a drawing state of the electronic pen. An image processing apparatus that detects coordinates indicated by an electronic pen or a finger that performs an instruction operation on a display screen,
First light irradiation means for irradiating light to a coordinate detection region for detecting coordinates from the outer periphery of the display screen;
A plurality of light detection means for detecting the light emitted from the first light irradiation means and the light emitted from the second light irradiation means emitting light from the electronic pen;
A determination unit that determines whether the electronic pen or the finger is in accordance with a brightness level based on luminance information related to the luminance of light detected by the light detection unit;
Position specifying means for specifying the position of the electronic pen or the finger determined by the determining means;
An image processing apparatus comprising: An optical signal receiving means for receiving an optical communication signal for performing communication using light emitted from the second light irradiating means of the electronic pen;
Information extraction means for extracting pen identification information, which is identification information for identifying the electronic pen, included in the optical communication signal;
Drawing data management means for managing the pen identification information of the electronic pen extracted by the information extraction means in association with the coordinates indicating the position of the electronic pen specified by the position specifying means;
The image processing apparatus according to claim 16. A coordinate detection method executed by a coordinate detection system that detects coordinates indicated by an electronic pen or a finger that performs an instruction operation on a display screen,
Irradiating light to a coordinate detection area for detecting coordinates from the outer periphery of the display screen,
Detecting the irradiated light and the light emitted from the electronic pen;
Determining whether it is the electronic pen or the finger according to the brightness level based on the brightness information relating to the brightness of the detected light,
A coordinate detection method characterized by specifying the determined position of the electronic pen or the finger. A computer-readable program used in a coordinate detection system that detects coordinates indicated by an electronic pen or a finger that performs an instruction operation on a display screen,
A procedure in which the first light irradiation means irradiates light to a coordinate detection region for detecting coordinates from the outer periphery of the display screen;
A procedure in which a plurality of light detection means detect light emitted from the first light irradiation means and light emitted from a second light irradiation means that emits light from the electronic pen;
A procedure for determining whether the determination unit is the electronic pen or the finger according to a brightness level based on luminance information relating to the luminance of light detected by the light detection unit, and a position specifying unit are determined by the determination unit. A procedure for identifying the position of the electronic pen or the finger;
For causing the computer to execute.