JP7143642B2 - shared terminal, writing method, program - Google Patents

Description

The present invention relates to a shared terminal, writing method, and program.

A touch panel that detects the coordinates on the display indicated by the user using a pointing object such as an electronic pen on a display such as a liquid crystal panel, and draws and displays handwritten strokes based on the coordinate data output from the touch panel. Electronic blackboards with controllers and are widely used.

Touch panels are widely used in addition to electronic blackboards, and techniques for assisting writing on touch panels have been devised (see, for example, Patent Document 1). Patent Literature 1 discloses a smartphone that corrects the display position of a drawn image when there is a difference between the writing position of the user and the line-of-sight position in order to accurately adjust the writing position on the touch panel.

However, the conventional technology has a problem that it is not easy for a user to widely use and write on a large-sized display such as an electronic blackboard. Description will be made with reference to FIG.

FIG. 1 shows one scene in which a user writes on an electronic blackboard. As shown in FIG. 1(a), there is also a model equipped with a large size display in the electronic blackboard, and when the user tries to write from one end to the other end of the display 3, the user moves a distance corresponding to the display size. There must be.

In addition, as shown in FIG. 1B, depending on the position of the user, the display 3 may be hidden by the user's body, or the line of sight of another user who is trying to see the screen may be blocked by the user who is the scribe. It is easy to get lost.

For these reasons, conventionally, it has not been easy for the user to widely use the display 3 to write (easy to write from end to end). In order to write from end to end and not block the line of sight, the user has to move from end to end of the display 3 or frequently change the standing position.

SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to provide a shared terminal that facilitates drawing on a wide range of a display.

In view of the above problems, the present invention provides a shared terminal having a display, comprising contact position acquisition means for acquiring a contact position of a pointing object with respect to the display; line-of-sight position detection means for detecting a line-of-sight position of a user operating the pointing object with respect to the display; the contact position acquired by the contact position acquisition means; and the line-of-sight position detected by the line-of-sight position detection means. a drawn image determining means for determining a drawn image around the line-of-sight position with respect to the display, and a drawn image around the line-of-sight position determined by the drawn image determining means is copied to the contact position on the display. and drawn image moving means for moving the drawn image drawn at the contact position by the drawing means to the circumference of the line-of-sight position and displaying it on the display, wherein the drawing means while the line-of-sight position detection means detects the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position, the drawing means displays a drawn image on the contact position on the display, and the line-of-sight position is When the detection means does not detect the user's line-of-sight position with respect to the display within a predetermined distance from the contact position, the drawn image moving means moves the drawn image displayed at the contact position by the drawing means around the line-of-sight position. , and displayed on the display .

It is possible to provide a shared terminal that facilitates drawing on a wide range of display.

It is a figure which shows an example of the scene which a user writes on an electronic blackboard. FIG. 10 is an example of a diagram outlining assistance when a user writes far from the display; It is an example of the schematic perspective view of an electronic blackboard. It is an example of the hardware block diagram of an electronic blackboard. It is an example of the functional block diagram which shows the function which an electronic blackboard has in block form. FIG. 4 is a diagram showing an example of one stroke data; FIG. It is an example of the figure which compares and demonstrates the conventional technique and the technique of this embodiment. It is an example of the flowchart figure which shows the transition to distant writing mode, and the operation|movement procedure of the electronic blackboard in distant writing mode. FIG. 10 is a diagram illustrating a copy position of a drawn image of a line-of-sight area and a destination position of a drawn image of a contact position; It is a figure which shows an example of the setting screen of an electronic blackboard. It is an example of the figure explaining the example of a display of the drawn image already written at the contact position. FIG. 11 is an example of a diagram for explaining a display example of a copied drawn image of a line-of-sight area and a new drawn image at a contact position; It is an example of the figure explaining the example of a display of the drawn image already written at the contact position. FIG. 10 is a diagram showing an example of a drawn image drawn in a line-of-sight area and a drawn image copied and displayed around a contact position; It is an example of the line-of-sight area setting screen on which the user arbitrarily sets the size and shape of the line-of-sight area. It is an example of the line-of-sight area setting screen on which the user arbitrarily sets the size and shape of the line-of-sight area. FIG. 10 is a diagram showing an example of a visual effect of the line-of-sight area; FIG. 10 is a diagram showing an example of the visual effect when the distant writing mode is terminated and the drawn image is reflected in the line-of-sight area; It is a figure which shows the other structural example of an electronic blackboard. It is a figure which shows the other structural example of an electronic blackboard. It is a figure which shows the other structural example of an electronic blackboard. It is a figure which shows the other structural example of an electronic blackboard.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An electronic blackboard and a writing method performed by the electronic blackboard will be described below as an example of a mode for carrying out the present invention with reference to the drawings.

<Outline of the operation of the electronic blackboard of the present embodiment>
The outline of the operation of the electronic blackboard 2 of this embodiment will be described with reference to FIG. FIG. 2 is an example of a diagram outlining assistance when the user writes on the display 3 far away. An operation mode in which a user who is a scribe supports writing in a distant area of the display 3 is referred to as a "distant writing mode".

As shown in FIG. 2(a), "ABC" is already written on the upper right of the electronic blackboard 2, and the user is looking at the drawn image 14 of "ABC". The intersection of the user's line-of-sight direction 16 and the display 3 is referred to as the line-of-sight position 13 . Also, the electronic blackboard 2 determines the line-of-sight area 11 having a predetermined size and shape based on the line-of-sight position 13 .

On the other hand, the user touches the lower left corner of the electronic blackboard 2 with the electronic pen 100 , and the electronic blackboard 2 detects the contact position 17 of the electronic pen 100 . The electronic blackboard 2 determines the drawn image 14 of the line-of-sight area 11 according to the line-of-sight position 13 and the contact position 17 . The contact area 12 shown in FIG. 2 is for indicating the range in which the drawing image 14 of the line-of-sight area 11 is copied (duplicated) and displayed, and the electronic blackboard 2 does not have to determine the contact area 12. .

As shown in FIG. 2(b), the user moves the line-of-sight direction 16 to the contact area 12. As shown in FIG. The electronic blackboard 2 detects that the line-of-sight direction 16 of the user has moved from the line-of-sight area 11 to the touch position 17 (or its surroundings), and copies the drawn image 14 of the line-of-sight area 11 to the surroundings of the contact position 17 . As a result, the user can newly write down the drawn image 15 near the drawn image 14 that has been copied at hand. In FIG. 2(b), a drawn image 15 of "DEF" is written.

As shown in FIG. 2(c), when writing is finished, the electronic pen 100 is separated from the display 3, and the user again moves the line-of-sight direction 16 to the line-of-sight area 11 (even if the user does not move to the line-of-sight area 11, the contact position is 17). When the electronic blackboard 2 detects that the line-of-sight direction 16 of the user has deviated from the contact position 17 , the new drawn image 15 written around the contact position 17 is reflected in the line-of-sight area 11 . Accordingly, the drawn image 15 is displayed below the drawn image 14 in the line-of-sight area 11 .

In this way, the user touches the display 3 with the electronic pen 100 while looking at the position where he or she wants to write (line-of-sight area 11), starts writing from the contact position 17, and moves the electronic pen 100 away from the display 3 after writing. By returning the direction 16 to the line-of-sight area 11, it is possible to write at a distance that cannot be written without moving. In addition, since the user can write while looking at his or her hand, it is easy to write. In addition, since the drawn image 14 at a distance can be displayed at hand, the user can write while considering the relationship with the drawn image 14 . In addition, the user who is the scribe hardly blocks the line of sight of other users with respect to the drawn image 15 moved to the line of sight area 11 . Therefore, the electronic blackboard 2 of the present embodiment can widely utilize the touch panel to assist the user in writing without moving the user from end to end of the display 3 .

<Terms>
The surroundings of the line-of-sight position refer to areas within a predetermined distance from the line-of-sight position 13 , and the line-of-sight area 11 corresponds to the surroundings of the line-of-sight position 13 in this embodiment.

Also, the distance is calculated from the Euclidean distance between two points on the surface of the display 3 . If the unit of the position to be detected is the dot unit, it may be calculated from the coordinates on the screen or the number of dots per inch (dpi).

The periphery of the contact position means within a predetermined distance from the contact position 17 , and the contact area 12 corresponds to the periphery of the contact position 17 in this embodiment.

Depending on the contact position and the line-of-sight position means that it is considered whether or not the contact position and the line-of-sight position are at a predetermined distance or more. Alternatively, it refers to the case where the contact position and the line-of-sight position are at least separated to the extent that it can be determined that they do not match. Considering the accuracy of the line-of-sight position 13, it is sufficient if the contact position and the line-of-sight position are clearly different from each other.

A shared terminal is a terminal or device that is shared by multiple users. In this embodiment, the term "electronic blackboard" will be used. An electronic blackboard has a function of accepting addition of information such as characters, drawings, images, handwritten images, stamps, etc. to a displayed screen from a user. The electronic blackboard also has the function of an information processing device. Electronic blackboards are sometimes called electronic information boards, electronic whiteboards, copyboards, electronic boards, or the like. Note that the whiteboard may be realized by a projector and a whiteboard (a normal blackboard type without a display) other than the type that captures strokes handwritten by a touch panel. Digital signage can be used as a shared terminal other than the electronic blackboard.

An information processing apparatus having a display with a touch panel may serve as an electronic blackboard. There is a tablet terminal as such an information processing device. Although the tablet terminal is not large enough for the user to move for writing, the present embodiment can be preferably applied.

<Configuration example>
FIG. 3 is an example of a schematic perspective view of the electronic blackboard 2. As shown in FIG. The electronic blackboard 2 has a display 3 . The electronic blackboard 2 can display on the display 3 a drawn image drawn by an event caused by the electronic pen 100 or the user's hand. Events generated by the user's hands include gestures such as enlarging or reducing drawn images and turning pages. You can also let

As shown in FIG. 3, the electronic blackboard 2 has a line-of-sight detection camera 19 . In FIG. 3, the line-of-sight detection cameras 19 are arranged at the upper left and the upper right of the display 3, respectively, but the number of the line-of-sight detection cameras 19 may be one, and the arrangement position is not limited to the illustrated position. The mechanism of line-of-sight detection will be described later. The reason why there are two line-of-sight detection cameras 19 is to selectively use the line-of-sight detection camera 19 closer to the user.

Although not shown in FIG. 3, the electronic blackboard 2 is connected to a communication network, and is communicatively connected to any other electronic blackboard 2 via the communication network. An event occurring on the electronic whiteboard 2 at one site is transmitted to the other electronic whiteboard 2 . Therefore, when the user changes the image displayed on the display 3 of the electronic blackboard 2 at one site based on an event generated by the electronic pen 100 or by hand, the event is transmitted to the electronic blackboard 2 at the other site and the electronic The image on the display 3 of the blackboard 2 is also changed.

Furthermore, the electronic blackboard 2 may be connected to a teleconference terminal via a predetermined cable. A video conference terminal exchanges images and sounds with a video conference terminal at another remote site, displays the images on a display device, and outputs audio from a speaker, enabling remote users to communicate with each other. enables remote communication such as conferencing. Therefore, the electronic blackboard 2 connected to the teleconference terminal (or in which the teleconference terminal is built) can display not only drawn images but also images of its own site and other sites on the display 3, and can output sound from the speaker.

It is also possible to connect a PC (Personal Computer) to the electronic blackboard 2 via a cable capable of communicating according to standards such as Display Port (registered trademark), DVI, HDMI (registered trademark) and VGA. The video of the PC can be displayed on the display 3 or captured. Further, the electronic blackboard 2 can transmit event information indicating an event by the electronic pen 100 or by hand to the PC to operate the PC in the same manner as an event from an input device such as a mouse or a keyboard.

<<Hardware configuration example>>
Next, the hardware configuration of the electronic blackboard 2 will be described with reference to FIG. FIG. 4 is an example of a hardware configuration diagram of the electronic blackboard 2. As shown in FIG. As shown in FIG. 4, the electronic blackboard 2 includes a CPU 101 that controls the overall operation of the electronic blackboard 2, a ROM 102 that stores programs used to drive the CPU such as IPL, and a RAM 103 that is used as a work area for the CPU 101. , a network controller 105 that controls communication with the communication network, and an external storage controller 106 that controls communication with the USB memory 5 .

The electronic blackboard 2 also has a capture device 118 that captures still images or moving images displayed on the display 3 of the PC 6 by the PC 6 . It also has an external video interface 111 that captures video from the teleconference terminal 7 . Also, a GPU 112 that specializes in graphics (image processing of the screen displayed on the display 3), and a display controller 113 that controls and manages screen display in order to output video from the GPU 112 to the display 3 and the teleconference terminal 7. It has

Further, the electronic blackboard 2 includes a sensor controller 114 that controls touch panel processing, and a touch panel 115 that detects that the electronic pen 100 has touched the display 3 . The electronic blackboard 2 also includes an electronic pen controller 116 . The electronic pen controller 116 communicates with the electronic pen 100 to determine whether or not the display 3 has been touched with the tip of the electronic pen 100 or with the tail of the pen. It should be noted that the electronic pen controller 116 may determine the presence or absence of touch and pressure on the portion of the electronic pen 100 gripped by the user and other portions of the electronic pen 100 in addition to the pen tip and pen butt.

The electronic blackboard 2 also has two line-of-sight detection cameras 19 connected to the line-of-sight detection camera controller 117 . The line-of-sight detection camera controller 117 detects the line-of-sight position 13 by controlling the line-of-sight detection camera 19 and processing the captured image data.

Further, the electronic blackboard 2 includes a CPU 101, a ROM 102, a RAM 103, a network controller 105, an external storage controller 106, an external video interface 111, a capture device 118, a GPU 112, a sensor controller 114, an electronic pen controller 116, and a line-of-sight detection camera controller 117. are provided with bus lines 120, such as an address bus and a data bus, for electrically connecting them as shown in FIG.

<About functions>
FIG. 5 is an example of a functional block diagram showing functions of the electronic whiteboard 2 in block form. The electronic blackboard 2 includes an input detection unit 21, a line-of-sight position detection unit 22, a drawing control unit 23, a drawn image determining unit 24, a drawn image copying unit 25, a stroke drawing unit 26, a drawn image moving unit 27, a visual effect unit 28, and a UI. It has an image generation unit 29 , a background generation unit 30 , a video superimposition unit 31 , a layout management unit 32 , and a display control unit 33 . These functions possessed by the electronic whiteboard 2 are functions or means realized by any one of the components shown in FIG. is.

The electronic blackboard 2 also has a stroke data DB 35 constructed in the ROM 202, RAM 203, SSD 204, etc. shown in FIG. Information such as the coordinates of each stroke written by the user is stored in the stroke data DB 35 . A collection of one or more strokes (for example, an image that makes sense with one or more strokes, such as a character) is called a drawn image. Details of the stroke data will be described with reference to FIG.

The input detection unit 21 is realized by the touch panel 115, and detects that the electronic pen 100 or the hand H, which is a pointing object, has pointed to the display 3 and the pointed position (coordinates). That is, the user's writing on the touch panel and the contact position 17 are detected. The touched position 17 is the coordinates of the touched position on the touch panel. The coordinate reference point can be anywhere. The electronic blackboard 2 can also be written with the hand H, but for the sake of explanation, it is assumed that the electronic pen 100 is used for drawing. Also, the above gesture input (for example, enlarging or reducing the drawn image/screen, page feed, long press, etc.) is not direct writing, but enlarging or reducing the drawn image is also possible in the remote writing mode. be. However, in this embodiment, the processing at the time of writing will be mainly described, and enlargement/reduction of the drawn image will be described later. The contact position 17 on the touch panel detected by the input detection unit 21 is notified to the drawing control unit 23 .

The line-of-sight position detector 22 detects the line-of-sight direction 16 and the line-of-sight position 13 of the user. Since detection of the line-of-sight direction 16 is performed on the premise of detection of the line-of-sight position 13, hereinafter, the line-of-sight position 13 is simply detected. A plurality of users may exist in front of the electronic blackboard 2, and in that case, the line of sight of the closest user is detected. For example, face recognition is performed to identify a user whose face has the largest circumscribing rectangle, and the line-of-sight position 13 of this user is detected. Alternatively, the user who draws on the display 3 may be identified by recognizing the image of the user's posture or the like to identify the user who writes. Alternatively, a distance sensor such as an ultrasonic sensor may be used to identify the direction in which the closest user is when viewed from the electronic whiteboard 2, and the line-of-sight position 13 may be detected from the user's face in this direction.

The drawing control unit 23 performs overall control of writing on the touch panel. The drawing control unit 23 determines whether or not to perform the long-distance writing mode. A criterion for determining whether or not to perform the long-distance writing mode is that the line-of-sight position 13 and the contact position 17 of the electronic pen 100 are not the same (are separated by a predetermined value or more). When it is determined to operate in the distant writing mode, the drawing control unit 23 causes the drawing image determining unit 24 to determine the line-of-sight area 11 and the drawing image 14 within the line-of-sight area 11 . Further, the drawn image copying unit 25 is made to copy (replicate) the drawn image 14 of the line-of-sight area 11 around the contact position 17 . Further, in the distant writing mode, when the stroke drawing unit 26 draws the drawn image 15 at the contact position 17 and the user's gaze position deviates from the contact position 17, the drawn image moving unit 27 moves the drawn image 15 at the contact position 17. is reflected in the line-of-sight area 11 . Reflecting means moving the drawn image 15 to the line-of-sight area 11 .

Based on the line-of-sight position 13 , the drawn image determination unit 24 determines the line-of-sight area 11 around the line-of-sight position 13 and determines the drawn image 14 within the line-of-sight area 11 . Based on the line-of-sight position 13, the drawing image determination unit 24 obtains the line-of-sight area 11 whose size and shape are determined in advance. The line-of-sight area 11 is circular or rectangular as described above. The determined line-of-sight area 11 is determined with the line-of-sight position 13 as the center. Then, the drawn images included in the line-of-sight area 11 (all if plural) are determined as the drawn images around the line-of-sight position 13 . Although it is not necessary to determine the contact area 12 in the distant writing mode, for convenience of explanation, the size of the contact area 12 and the line-of-sight area 11 are assumed to be the same. For example, the contact area 12 has the same width as the line-of-sight area 11 with the center of the contact position 17 as a reference.

A drawn image copying unit 25 copies the drawn image 14 in the line-of-sight area 11 around the contact position 17 . The position to be copied is above the contact position 17, and the details will be explained with reference to FIG.

The stroke drawing unit 26 draws on the display 3 strokes obtained by connecting the contact positions 17 of the electronic pen 100 detected by the input detection unit 21 . A stroke is a linear object in which a series of coordinates from touchdown to touchup of the electronic pen 100 are connected. The position drawn by the stroke drawing unit 26 does not change whether the distant writing mode is performed or not. It should be noted that not only drawing a stroke but also deleting an existing stroke is included in writing.

Stroke data of each stroke drawn by the stroke drawing section 26 is stored in the stroke data DB 35 . The drawn image moving unit 27 moves the drawn image 15 written at the contact position 17 to the line-of-sight area 11 . During movement, the drawn image 15 is moved to the line-of-sight area 11 so that the relative position of the drawn image 15 with respect to the drawn image 14 around the contact position 17 is maintained in the line-of-sight area 11 as well. Note that the drawn image moving unit 27 erases the drawn image 14 copied around the contact position 17 .

The visual effect unit 28 adds various visual effects to the drawn image 14 and the like for display. For example, visual effects such as drawing a visual effect indicating where the line-of-sight position 13 or the line-of-sight area 11 is by a frame surrounding the line-of-sight area 11, or displaying a mark near the line-of-sight area 11 (within a predetermined distance from the line-of-sight area 11). Use effects. Further, when the drawn image moving unit 27 moves the drawn image 15 to the line-of-sight area 11, animation or a visual effect showing the line-of-sight area 11 with a frame is drawn.

The UI image generator 29 generates a UI (user interface) image in which menu buttons preset on the electronic blackboard 2 are arranged. The background generation unit 30 generates an image such as a solid background or a grid display as the background of the touch panel. A previously captured image can also be a background image.

The layout management unit 32 supplies the image superimposing unit 31 with the drawn images or visual effects drawn by the drawn image copying unit 25, the stroke drawing unit 26, the drawn image moving unit 27, or the visual effects unit 28, and the UI image generating unit 29. Layout information indicating the layout of the UI image to be generated and the background image generated by the background generation unit 30 is managed. As a result, the layout management unit 32 can instruct the image superimposing unit 31 in what order the drawing image, visual effect, UI image, and background image should be superimposed or should be hidden. can.

Based on the layout information output from the layout management unit 32, the image superimposing unit 31 lays out (superimposes) the drawn image, the visual effect, the UI image, and the background image. A superimposed image is called a superimposed image.

The display control unit 33 displays the superimposed image created by the video superimposing unit 31 on the display 3 . In the following description, however, for the sake of simplicity, it is assumed that the drawn image copy unit 25, the stroke drawing unit 26, the drawn image moving unit 27, or the visual effects unit 28 draws and displays the image.

<About stroke data>
Stroke data will be described with reference to FIG. FIG. 6 shows an example of one stroke data. In FIG. 6, one stroke is described in JSON format. The format of stroke data may be XML, CSV or some other format.
・id is identification information for identifying an object (stroke).
・type indicates the type of the object. For strokes, it is "stroke".
・parent is the page number (page identification information) on which the stroke is drawn.
*meeting is identification information of a meeting (from one power-on to power-off).
• owner is the creator of the stroke. For example, it is the ID of the logged-in user.
・width is the width of the line when creating the stroke (the width of one coordinate).
・Height is the height of the line when creating the stroke (height for one coordinate).
・attr is the attribute of the stroke (stroke width, height, color, presence/absence of Bezier, shape).
・points are the coordinates that make up the stroke.
・position specifies how to take the coordinates of points (absolute coordinates, relative coordinates, differential coordinates). "Absolute coordinates" are coordinates with respect to a predetermined origin of the touch panel. "Relative coordinates" are absolute coordinates for the top element, but relative coordinates for other points with the top element as a reference point. "Differential coordinates" indicates that the coordinates of the top element are absolute coordinates, but the other points are the difference values from the immediately preceding point.

Thus, stroke data is stored for each stroke. In the distant writing mode, one or more stroke data constituting the drawn image 14 of the line-of-sight area 11 are copied around the contact position 17, and when the distant writing mode ends, the stroke data written at the contact position 17 are transferred to the line-of-sight area 11. is moved to

<Comparison with conventional technology>
Next, the conventional technology and the technology of this embodiment will be compared with each other with reference to FIG. FIG. 7 is an example of a diagram for explaining a comparison between the conventional technology and the technology of the present embodiment.

First, FIGS. 7A and 7B are diagrams for explaining the prior art. The line-of-sight region 11 and the contact region 12 in FIG. 7A are only shown for explanation, and there is no concept of the line-of-sight region 11 and the contact region 12 in the prior art. As shown in FIG. 7B, the user writes a stroke at the contact position 17 while looking at the line-of-sight area 11, but the drawn image is not displayed at the contact position 17, and the written content is displayed at the end of the line of sight. Only.

7(c) to (e) are diagrams for explaining the long-distance writing mode of the present embodiment. In this embodiment, the electronic blackboard 2 detects the line-of-sight position 13 of the user when the electronic pen 100 contacts the display 3, and as shown in FIG. are the same (compare). When the contact position 17 and the line-of-sight position 13 are at a predetermined distance or more, the electronic blackboard 2 copies the drawn image 14 of the line-of-sight area 11 to the contact area 12 and displays it, as shown in FIG. 7(d).

Also, as shown in FIG. 7D, the user writes by hand while looking at the contact position 17 . When the user's line-of-sight position 13 is removed from the contact position 17 after writing is finished (FIG. 7(e)), the electronic blackboard 2 reflects (moves) a new drawn image 15 on the line-of-sight area 11 .

Therefore, in the conventional technology, the user has to write while looking at a distance, and it is difficult to write at hand. In other words, it cannot be written easily. In this embodiment, writing at hand can be easily realized. Moreover, since the drawn image 14 of the line-of-sight position 13 can be displayed at hand, the user can write while considering the relationship with the drawn image 14 .

<Electronic blackboard operation>
The operating procedure of the electronic blackboard 2 in the remote writing mode will be described with reference to FIG. FIG. 8 is an example of a flow chart diagram showing the transition to the far writing mode and the operation procedure of the electronic blackboard 2 in the far writing mode. The processing in FIG. 8 starts when the electronic blackboard 2 is powered on, for example.

When the user starts writing a stroke with the electronic pen 100, the input detection unit 21 detects the contact position 17 touched by the electronic pen 100 (step S1).

When a plurality of line-of-sight detection cameras 19 are installed, the line-of-sight position detection unit 22 adopts the image data of the line-of-sight detection camera 19 closer to the user. ) is included in the left half (step S2). That is, when the visual axis detection cameras 19 are mounted on the left and right sides of the electronic blackboard 2 as shown in FIG. image data is used (step S3), and in the case of the right half (No in S2), the image data of the right eye line detection camera 19 is used (step S4).

Next, the line-of-sight position detection unit 22 detects the line-of-sight position 13 from the image data captured by the left or right line-of-sight detection camera 19 (step S5).

Then, the contact position 17 of the electronic pen 100 and the line-of-sight position 13 are compared to switch the operation mode. The operation mode includes a normal operation mode (hereinafter referred to as a normal writing mode) and a long-distance writing mode.

The drawing control unit 23 determines whether the contact position 17 of the electronic pen 100 and the line-of-sight position 13 are the same (step S6).

If the contact position 17 of the electronic pen 100 and the line-of-sight position 13 are the same (Yes in step S6), the drawing control unit 23 determines to operate in the normal writing mode (step S7).

The stroke drawing unit 26 draws and displays a stroke at the contact position 17 as usual (step S8). The stroke drawing unit 26 repeats stroke drawing and display while taking in stroke data until the electronic pen 100 moves away from the display 3 (step S9).

If the distance between the contact position 17 of the electronic pen 100 and the line-of-sight position 13 is greater than or equal to the predetermined distance (No in step S6), the drawing control unit 23 determines to operate in the long-distance writing mode (step S10).

The drawing control unit 23 causes the drawn image determination unit 24 to determine the line-of-sight area 11 and the drawn image 14 within the line-of-sight area 11 . Further, the drawn image copy unit 25 is caused to display the drawn image 14 within the line-of-sight area 11 around the contact position 17 (step S11). The perimeter of the contact location 17 does not overlap the contact location 17, but is preferably close to the contact location 17. However, as will be described later, by adjusting the color or the like of the drawn image 14, visibility can be maintained even if the contact position 17 and the drawn image 14 overlap. Since strokes are normally written from left to right and from top to bottom, as an example, the drawn image 14 of the line-of-sight area 11 is displayed above the contact position 17 of the electronic pen 100 . The drawn image copying unit 25 places the upper left corner of the drawn image 14 bounding rectangle above the contact position 17 in consideration of the height and margin of the drawn image 14 bounding rectangle. The drawn image copying unit 25 changes the coordinates of the stroke data according to the amount of movement of the drawn image 14 . Details will be described with reference to FIG.

A depiction 14 of the line-of-sight area 11 is displayed at two locations around the line-of-sight area 11 and the contact position 17 . For this purpose, after changing the coordinates of the stroke, the drawn image moving unit 27 copies the stroke data forming the drawn image 14 in the line-of-sight area 11 and stores it in the stroke data DB 35 . In this case, in order to identify the stroke data of the drawn image 14 of the line-of-sight area 11 (to erase it later), for example, the id of the copied stroke data is changed to the stroke data constituting the drawn image 14 of the line-of-sight area 11 It is given like “id + branch number”.
Stroke data id for viewing direction 16: b518dfe9-10c0-4d05-aff4-5755dbb11d30
Copied stroke data id: b518dfe9－10c0－4d05－aff4－5755dbb11d30α
Therefore, it can be understood that the drawn image moving unit 27 should delete the stroke data to which the branch number (α) is assigned when the distant writing mode ends. By erasing the drawn image 14, it is possible to prevent the same drawn image from being displayed in two places.

When the user writes a stroke, the stroke drawing unit 26 draws the stroke at the contact position 17 (step S12). The stroke drawing unit 26 repeats drawing and displaying strokes while fetching stroke data until the electronic pen 100 moves away from the display 3 (step S13). Stroke data written in the distant writing mode are drawn around the drawn image 14 in the line-of-sight area 11 when the user removes the line of sight from the contact position 17 (when the line-of-sight position 13 is no longer detected within a predetermined distance from the contact position 17). be moved. For this reason, the id of stroke data is given, for example, as the id of any stroke data constituting the drawn image 14 of the line-of-sight area 11 + branch number so that the stroke data can be identified as moving later.
Stroke data id of drawn image 14 in line-of-sight area 11: b518dfe9-10c0-4d05-aff4-5755dbb11d30
id of newly drawn stroke data: b518dfe9-10c0-4d05-aff4-5755dbb11d30β
Therefore, it can be understood that the drawn image moving unit 27 should move the stroke data to which the branch number (β) is assigned when the distant writing mode ends.

In the distant writing mode, the line-of-sight position detector 22 detects the user's line of sight. While the electronic pen 100 is away from the display 3, the drawing control unit 23 determines whether or not the line of sight has left the last contact position 17 of the electronic pen 100 (step S14).

Until the line of sight moves away from the last contact position 17 of the electronic pen 100 (Yes in step S14), the stroke drawing unit 26 repeats drawing and displaying strokes around the line-of-sight position 13 (step S15).

When the line of sight moves away from the last contact position 17 of the electronic pen 100 (No in step S14), the drawing control unit 23 determines that the “distant writing mode” has ended, and the drawing image moving unit 27 moves to the contact position 17. The handwritten stroke data is moved around the drawn image 14 in the line-of-sight area 11 and displayed (S16). The drawn image moving unit 27 specifies the stroke data to be moved by a branch number β, for example, and adjusts the height and margin of the circumscribed rectangle of the drawn image 14 in the line-of-sight area 11 based on the upper left corner of the circumscribed rectangle of the drawn image 14. Considering this, the coordinates of the stroke data of the drawn image 15 drawn around the contact position 17 are changed. Details will be described with reference to FIG. By doing so, the drawn image 15 can be displayed around the drawn image 14 at the line-of-sight position 13 while maintaining the relative positions of the drawn images 14 and 15 in the contact area 12 .

Also, the drawing image moving unit 27 erases the stroke data specified by the branch number α. As a result, the drawn image copied to the contact position 17 can be erased.

<Copy position of drawn image of line-of-sight area and position of destination of moved drawn image of contact position>
The copy position of the drawn image 14 in the line-of-sight area 11 and the destination position of the drawn image 15 in the contact position 17 will be described with reference to FIG. FIG. 9 is a diagram for explaining the copy position of the drawn image 14 in the line-of-sight area 11 and the destination position of the drawn image 15 in the contact position 17 .

FIG. 9(a) is a diagram for explaining the copy position of the drawn image 14 in the line-of-sight area 11. FIG. It is assumed that the line-of-sight area 11 has already been determined. Also, the contact position 17 of the electronic pen 100 is detected. In order to copy and display the drawn image 14 of the line-of-sight area 11, the drawn image copying unit 25 detects the upper left corner P1 of the bounding rectangle of the drawn image 14 of the line-of-sight area 11 and the height h of the bounding rectangle. Then, the upper left corner P2 of the circumscribing rectangle of the drawing image 14 in the line-of-sight area 11 is determined to be a position above the contact position 17 of the electronic pen 100 by the height h and the margin M.

The drawn image copying unit 25 can calculate the difference (ΔX, ΔY) between the upper left corner P1 of the bounding rectangle of the drawn image 14 in the line-of-sight area 11 and the upper left corner P2 of the bounding rectangle of the contact area 12 . Therefore, the drawn image copying unit 25 may change the stroke data of the drawn image 14 by the difference (ΔX, ΔY).

FIG. 9(b) shows the drawing image 15 newly drawn at the contact position 17. FIG. After this, the user looks away from the newly drawn drawing image 15 . To remove the line of sight means that the line of sight position is separated by a predetermined distance or more from the center of the newly drawn drawn image 15 or the last contact position of the electronic pen (the line of sight position is no longer detected within the predetermined distance). .

FIG. 9(c) shows the drawn image 15 written at the contact position 17 moved below the drawn image 14 in the line-of-sight area 11. FIG. The drawn image moving unit 27 determines the upper left corner P3 of the circumscribed rectangle of the drawn image 15 in the line-of-sight area 11 at a position separated by the height h and the margin M downward from the upper left corner P1 of the circumscribed rectangle of the drawn image 14 .

By doing so, the relative positions of the drawn images 14 and 15 in the contact area 12 can be maintained in the line-of-sight area 11 as well. When the height h and the margin M are constant, the coordinate difference (ΔX, ΔY) between the contact position 17 and the upper left corner P3 of the circumscribed rectangle of the drawn image 15 is the coordinate difference (ΔX, ΔY ). Of course, it can also be calculated from FIG. 9(c). The drawn image moving unit 27 may change the stroke data of the drawn image 15 by the difference in coordinates (ΔX, ΔY).

It should be noted that there may be cases where there is no space above the contact position 17 in FIG. 9A, or there are cases where there is no space below the drawn image 14 in the line-of-sight area 11 in FIG. 9C. As is clear from FIG. 9, when the upward direction of the contact position 17 is adopted as the movement destination of the drawn image 14, a space is required below the drawn image 14 in the line-of-sight area 11, and the rightward direction of the contact position 17 is required. as the destination of the drawn image 14, a space is required to the left of the drawn image 14 in the line-of-sight area 11. A space is required to the right of the drawn image 14 in the area 11, and if the downward direction of the contact position 17 is adopted as the movement destination of the drawn image 14, a space is necessary to the upward direction of the drawn image 14 in the line-of-sight area 11. Become.

For this reason, the drawn image copying unit 25 may adopt a direction in which there is space around the contact position 17 according to the order of priority of upward, rightward, leftward, and downward directions.

However, as illustrated in FIGS. 11-13, the remote writing mode can be performed without the space around the contact location 17. FIG.

<Method of Detecting Line of Sight Position>
A known method can be used as a method for detecting the line-of-sight position 13 . For example, there is the following prior art.

The line-of-sight position detector 22 detects the line-of-sight position 13 of the user with respect to the display 3 . The line-of-sight position 13 corresponds to the position on the display 3 when the line of sight extending linearly from the center position of the pupil in the user's eyeball intersects the display 3 .

The line-of-sight position detection unit 22 detects the line-of-sight position 13 by, for example, applying an algorithm based on the corneal reflection method to the image of the user. Specifically, the line-of-sight position detection unit 22 emits infrared rays from the infrared irradiation unit when acquiring an image of the user. The line-of-sight position detection unit 22 identifies the position of the pupil and the position of the corneal reflection of the infrared rays from the acquired image of the user. The line-of-sight position detection unit 22 identifies the line-of-sight direction of the user based on the positional relationship between the position of the pupil and the position of the corneal reflection of infrared rays. The line-of-sight position detection unit 22 determines that the direction of the line of sight is on the outer corner of the eye if the position of the pupil is closer to the outer corner of the eye than the position of the corneal reflection. , the direction of the line of sight is determined to be the inner corner of the eye. The line-of-sight position detection unit 22 calculates the distance between the user's eyeball and the display 3 based on the size of the iris, for example. Based on the direction of the user's line of sight and the distance between the user's eyeball and the display 3, the line-of-sight position detection unit 22 determines whether the line of sight originating from the pupil is detected from the center position of the pupil in the user's eyeball and the display 3. A line-of-sight position 13, which is the position on the display 3 when crossing, is detected.

In the above example, the line-of-sight position detection unit 22 detects the user's line-of-sight position 13 by performing processing using the corneal reflection method, but the present invention is not limited to this example. For example, the line-of-sight position detection unit 22 may detect the line-of-sight position 13 by executing image recognition processing on the image of the user. For example, the line-of-sight position detection unit 22 extracts a predetermined region including the user's eyeball from the user's image, identifies the line-of-sight direction 16 based on the positional relationship between the inner corner of the eye and the iris, and identifies the line-of-sight direction 16, The line-of-sight position 13 is detected based on the distance from the user's eyeball to the display 3 . Alternatively, the line-of-sight position detection unit 22 accumulates, as reference images, images of a plurality of eyeballs when the user browses various parts of the display area of the display 3. By comparing the image of the eyeball with the image of the eyeball, the line-of-sight position 13 is detected.

<Electronic blackboard setting screen>
Next, settings related to the electronic blackboard 2 will be described with reference to FIG. The electronic blackboard 2 can display a setting screen for customizing the electronic blackboard 2 so that the user can use it easily. The visual effect and the size and shape of the line-of-sight area 11 can be set from this setting screen.

FIG. 10 shows an example of a setting screen of the electronic blackboard 2. As shown in FIG. The setting screen of the electronic whiteboard 2 is a screen for setting the operation of the electronic whiteboard 2, such as network settings and authentication (account) settings.

FIG. 10A shows an icon screen 401 on which an icon 402 for displaying a setting screen is displayed. When this icon 402 is selected with the electronic pen 100, the UI image generator 29 displays the setting screen 410 of FIG. 10(b). From a setting screen 410 as shown in FIG. 10B, the user can set parameters (width, height, etc.) related to drawing with the electronic pen 100, visual effects, size and shape of the line-of-sight area 11, and the like. Setting information is stored in the ROM.

<Process when there is no space around the contact position 17>
11 to 13, processing when there is no space around the contact position 17 and there is already a drawn image in the distant writing mode will be described.

<<Change the color of the drawn image that has already been written>>
FIG. 11 is an example of a diagram illustrating a display example of a drawn image that has already been written on the contact position 17. As shown in FIG. When the drawn image copying unit 25 displays the drawn image 14 in the line-of-sight area 11 around the contact position 17 of the electronic pen 100 in the distant writing mode, the drawn image copying unit 25 copies the original drawing around the contact position 17 . The color of the drawn image 18 may be changed.

In FIG. 11( a ), a drawn image 18 of “GHI” already exists around the contact position 17 of the electronic pen 100 . This perimeter is within a predetermined distance from the contact location 17 and is, for example, the contact area 12 . When the drawn image 14 in the line-of-sight area 11 is copied around the contact position 17 of the electronic pen 100 and displayed, a drawn image 18 called "GHI", a drawn image 14, or a newly drawn drawn image 15 called "DEF" is displayed. It may be difficult for the user to see.

Therefore, as shown in FIG. 11B, the drawn image copying unit 25 reduces the brightness of the drawn image 18 named "GHI" to make it different from the drawn images 15 and 14 named "DEF" to be newly drawn. change to color. For example, change to a less noticeable color, such as translucent, yellow, or light gray. To change the color, change the stroke data attribute (attr).

By doing so, the drawn image 14 of the line-of-sight area 11 and the new drawn image 15 can be displayed around the contact position 17 without erasing the drawn image 18 near the contact position 17 of the electronic pen 100 .

<<Change the color of the drawn image 14 in the line-of-sight area 11 and the new drawn image 15>>
Conversely, as shown in FIG. 12, the color of the copied image 14 of the line-of-sight area 11 and the color of the new image 15 at the contact position 17 may be changed. FIG. 12 is an example of a diagram illustrating a display example of the copied drawn image 14 in the line-of-sight area 11 and the new drawn image 15 at the contact position 17 . FIG. 12(a) is similar to FIG. 11(a).

As shown in FIG. 12(b), the drawn image copying unit 25 lowers the brightness of the drawn image 14 "ABC" in the line-of-sight area 11 and the drawn image 15 "DEF" to be newly drawn. is changed to a color different from that of the drawn image 18 of "GHI" drawn from . The drawn image 14 of "ABC" and the drawn image 15 of "DEF" are changed to an inconspicuous color such as translucent, yellow, or light gray. To change the color, change the stroke data attribute (attr).

By doing so, the drawn image 14 in the line-of-sight area and the new drawn image 15 can be displayed around the contact position 17 without erasing the drawn image 18 near the contact position 17 of the electronic pen 100 .

Note that the drawn image 14 of the line-of-sight area and the new drawn image 15 may not have the same color, and the user can distinguish between them by using different colors.

<<Temporarily hide the drawn image that has already been written>>
As shown in FIG. 13, the drawn image 18 originally drawn at the contact position 17 may be temporarily hidden. Temporarily is during the far writing mode. FIG. 13 is an example of a diagram illustrating a display example of a drawn image already written on the contact position 17. FIG. FIG. 13(a) is similar to FIG. 11(a).

As shown in FIG. 13B, the drawn image copying unit 25 hides the drawn image 18 of "GHI" when the distant writing mode is started. Since the stroke data is not erased, the color may be made transparent while leaving the stroke data in the stroke data DB 35 . Or set it to the same color as the background.

After the end of the distant writing mode, the drawn image copying unit 25 restores the color of the drawn image 18 to the original color, so that the drawn image "GHI" can be hidden only temporarily.

By doing so, the drawn image 14 of the line-of-sight area and the new drawn image 15 can be displayed around the contact position 17 .

<User specification of gaze area>
The drawn image 14 of the line-of-sight area 11 is determined by the drawn image determination unit 24 based on the line-of-sight position 13, but there are cases where an appropriate drawn image for the user cannot be determined. Description will be made with reference to FIG.

FIG. 14 shows the drawn image 14 drawn in the line-of-sight area 11 and the drawn image 14 copied and displayed around the contact position 17 . As shown in FIG. 14(a), the line-of-sight area 11 has a drawn image 14 of "ABC" and a drawn image 14-2 of "DEF". Only 14 are displayed. However, there is a case where the user wants to draw a new drawn image 15 by displaying the drawn image 14-2 of "DEF" around the contact position 17 as well.

In such a case, as shown in FIG. 14B, the electronic blackboard 2 can provide a mechanism that allows the user to arbitrarily set the size and shape of the line-of-sight area 11 . Any setting method may be used. For example, the user selects a button for selecting the size and shape of the line-of-sight area 11 from a menu list, selects the size from large, medium, and small, or selects the shape from circle or rectangle. Alternatively, the user may arbitrarily draw a desired size and shape with the electronic pen 100 by selecting a button for arbitrarily designating the size and shape of the line-of-sight area 11 from a menu list. A method for arbitrarily designating the size and shape of the line-of-sight area 11 will be described with reference to FIGS. 15 and 16. FIG.

In FIG. 14B, since the size of the line-of-sight area 11 is increased, the drawn image 14 of "ABC" and the drawn image 14-2 of "DEF" are copied and displayed around the contact position 17. .

Also, as shown in FIG. 14(b), the drawn image 14 of "ABC" and the drawn image 14-2 of "DEF" are copied and displayed around the contact position 17, but as a user, "DEF" is displayed. In some cases, the drawn image 14-2 is not desired to be displayed at the contact position 17. FIG.

Even in such a case, the user can select or arbitrarily set the size and shape of the line-of-sight area 11, as shown in FIG. 14(c). In FIG. 14C, the line-of-sight area 11 has a rectangular shape, and only the drawn image 14 of "ABC" is displayed at the contact position 17. In FIG.

<<How to set the size and shape of the gaze area>>
A specific method for setting the size and shape of the line-of-sight area 11 will be described with reference to FIGS. 15 and 16. FIG. FIG. 15 shows an example of a line-of-sight area setting screen 420 on which the user arbitrarily sets the size and shape of the line-of-sight area 11 . As described with reference to FIG. 14, when the user wishes to adjust the size and shape of the line-of-sight area 11, the user presses a button for arbitrarily designating the size and shape of the line-of-sight area 11 from the menu or the like as described above. select. As a result, the UI image generator 29 displays the line-of-sight area setting screen 420 shown in FIG. 15(a). A line-of-sight area setting screen 420 displays a message 421 that reads, "Please set the area." In FIG. 15A, the user has selected the shape of the rectangular line-of-sight area 11 . The user draws the closing frame 422, for example. By drawing the closing frame 422 , the drawn image determination unit 24 receives designation of the rectangular line-of-sight area 11 .

As shown in FIG. 15B, the user can enlarge the closing frame 422 by dragging the vertex of the closing frame 422 outward. Conversely, the closing frame 422 can be reduced by performing a drag operation toward the center. In this manner, the user can change the size of the line-of-sight area 11 by performing a drag operation with the electronic pen 100, so that the size of the line-of-sight area 11 that is optimal for writing can be intuitively set.

Similarly, FIG. 16 is a diagram for explaining another method for the user to arbitrarily specify the size and shape of the line-of-sight area 11 . 15 except that the shape of the closing frame 422 is different. In FIG. 16A, the user has selected the shape of the circular line-of-sight area 11 . When the user draws the closing frame 423 , the drawing image determining unit 24 receives the setting of the circular line-of-sight region 11 . As shown in FIG. 16B, by dragging a portion of the closing frame 423 outward, the circle that forms the line-of-sight area 11 can be enlarged. A circle can be reduced.

Note that the adjustment of the size and shape of the line-of-sight area 11 described with reference to FIGS. 15 and 16 can be performed by the user by selecting a menu from the setting screen 410 of FIG. 10 as well as in the distant writing mode. Therefore, the user can adjust the size and shape of the line-of-sight area 11 in the initial state in advance.

<Visual effect of line-of-sight area>
Next, the visual effect of the line-of-sight area 11 will be described with reference to FIGS. 17 and 18. FIG. FIG. 17 is a diagram showing an example of a visual effect applied to the line-of-sight area 11. As shown in FIG. The visual effect unit 28 emphasizes the drawn image 14 or line-of-sight region 11 determined by the drawn image determination unit 24 .

In FIG. 17A, a frame 431 and a mark 430 surrounding the line-of-sight area 11 are displayed. In FIG. 17B, a frame 431 surrounding the line-of-sight area 11 is blinking. Mark 430 may blink. As a result, the user can visually grasp where on the display 3 the user is writing while writing on the contact position 17 .

When the drawing control unit 23 notifies the visual effect unit 28 of the start of the distant writing mode, the visual effects unit 28 acquires information about the position of the line-of-sight area 11 from the drawing image determination unit 24 and displays a frame 431 surrounding the line-of-sight area 11 . . The position of the mark 430 is slightly outside the outer edge of the line-of-sight area 11 , for example, in the direction of 2 o'clock with the line-of-sight position 13 as the center.

FIG. 18 shows an example of the visual effect when the distant writing mode is terminated and the drawn image 15 is reflected in the line-of-sight area 11 . In FIG. 18( a ), an arrow animation 432 is displayed from the contact position 17 toward the line-of-sight area 11 . An animation 432 shows how the arrow gradually extends, and arrows of several lengths are prepared in advance. The visual effects unit 28 realizes an animation 432 by switching and displaying gradually longer arrows (the maximum length of the arrow is the length from the contact position 17 to the line-of-sight area 11).

In addition, in FIG. 18B, a frame 431 is displayed and blinked in the line-of-sight area 11 . The display method of the frame 431 may be the same as in FIG.

When the drawing control unit 23 notifies the end of the distant writing mode, the visual effect unit 28 displays an arrow animation 432 from the contact position 17 toward the line-of-sight area 11 or displays a frame 431 in the line-of-sight area 11 . blinks as you move.

Thereby, the user can visually grasp at what timing the drawn image of the contact position 17 was reflected and where it was reflected.

<Summary>
As described above, according to the electronic blackboard 2 of the present embodiment, the user can write while looking at his or her hand, which facilitates writing. Moreover, since the drawn image 14 of the line-of-sight position 13 can be displayed at hand, the user can write while considering the relationship with the drawn image 14 . Further, when the line of sight is removed from the contact position 17, the drawn image 15 can be moved to the line-of-sight area 11 while maintaining the relative positions of the drawn image 14 and the written drawn image 15.例文帳に追加In addition, the user who is the scribe hardly blocks the line of sight of other users with respect to the drawn image 15 moved to the line of sight area 11 . Therefore, it is possible to easily write even in a distant place where writing cannot be done without moving, and it is possible to widely use the touch panel to assist the user in writing.

Although the electronic blackboard 2 of Embodiment 1 is described as having a large touch panel, the electronic blackboard 2 is not limited to having a touch panel.

<<Another example 1 of the configuration of the electronic blackboard>>
FIG. 19 is a diagram showing another configuration example of the electronic blackboard 2. As shown in FIG. In FIG. 19, a projector 411 is installed on the upper side of a normal whiteboard 413 . This projector 411 corresponds to the electronic blackboard 2 . The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which the user directly writes with markers. Note that the whiteboard may be a blackboard, and may be a flat surface that is large enough to project an image.

The projector 411 has an ultra-short focal length optical system, and can project an image with less distortion onto the whiteboard 413 from about 10 cm. This image may be transmitted from a PC connected wirelessly or by wire, or may be stored in the projector 411 .

The user writes by hand on the whiteboard 413 using the dedicated electronic pen 100 . The electronic pen 100 has, for example, a light-emitting portion at its tip portion, which is turned on when the user presses it against the whiteboard 413 for handwriting. Since the wavelength of light is in the near infrared or infrared, it is invisible to the user's eyes. The projector 411 has a camera, captures an image of the light emitting unit, analyzes the image, and identifies the direction of the electronic pen 100 . Also, the electronic pen 100 emits ultrasonic waves together with light emission, and the projector 411 calculates the distance from the arrival time of the ultrasonic waves. The position of the electronic pen 100 can be identified by the direction and distance. A stroke is drawn (projected) at the position of the electronic pen 100 .

Since the projector 411 projects the menu 440, when the user presses a button with the electronic pen 100, the projector 411 identifies the pressed button based on the position of the electronic pen 100 and the ON signal of the switch. For example, when the save button 441 is pressed, the stroke (set of coordinates) handwritten by the user is saved by the projector 411 . Projector 411 stores the handwritten information in predetermined server 412, USB memory 2600, or the like. Handwritten information is stored page by page. Since the coordinates are saved as they are instead of the image data, the user can re-edit them.

Since the line-of-sight detection camera 19 is arranged on the whiteboard 413, the distant writing mode can be executed in the same manner as in the first embodiment.

<<Another example 2 of the configuration of the electronic blackboard>>
FIG. 20 is a diagram showing another configuration example of the electronic blackboard 2. As shown in FIG. In the example of FIG. 20, the electronic blackboard 2 includes a terminal device 600, an image projection device 700A, and a pen motion detection device 810. In the example of FIG.

The terminal device 600 is connected to the image projection device 700A and the pen motion detection device 810 by wire. The image projection device 700A projects image data input from the terminal device 600 onto the screen 800 .

The pen motion detection device 810 communicates with the electronic pen 100 and detects motion of the electronic pen 100 in the vicinity of the screen 800 . Specifically, the electronic pen 100 detects coordinate information indicating a point indicated by the electronic pen 100 on the screen 800 and transmits the coordinate information to the terminal device 600 .

Terminal device 600 generates image data of a stroke image input by electronic pen 100 based on the coordinate information received from pen motion detection device 810, and causes image projection device 700A to draw the stroke image on screen 800. FIG.

The terminal device 600 also generates superimposed image data indicating a superimposed image obtained by synthesizing the background image projected by the image projection device 700A and the stroke image input by the electronic pen 100 .

Since the line-of-sight detection camera 19 is arranged on the screen 800, the distant writing mode can be executed in the same manner as in the first embodiment.

<<Another example 3 of the configuration of the electronic blackboard>>
FIG. 21 is a diagram showing a configuration example of the electronic blackboard 2. As shown in FIG. In the example of FIG. 21, the electronic blackboard 2 includes a terminal device 600, a display 800A, and a pen motion detection device 810A.

Pen motion detection device 810A is arranged near display 800A, detects coordinate information indicating a point indicated by electronic pen 100 on display 800A, and transmits the information to terminal device 600. FIG. Note that in the example of FIG. 21, the electronic pen 100 may be charged by the terminal device 600 via a USB connector.

Terminal device 600 generates image data of a stroke image input by electronic pen 100 based on the coordinate information received from pen motion detection device 810A, and causes display 800A to display the image data.

Since the visual line detection camera 19 is arranged on the display 800A, the distant writing mode can be executed in the same manner as in the first embodiment.

<<Another example 4 of the configuration of the electronic blackboard>>
FIG. 22 is a diagram showing a configuration example of the electronic blackboard 2. As shown in FIG. In the example of FIG. 22, the electronic blackboard 2 includes a terminal device 600 and an image projection device 700A.

The terminal device 600 performs wireless communication (Bluetooth (registered trademark), etc.) with the electronic pen 100 and receives coordinate information of a point indicated by the electronic pen 100 on the screen 800 . Based on the received coordinate information, terminal device 600 generates image data of a stroke image input by electronic pen 100, and causes image projection device 700A to project the stroke image.

The terminal device 600 also generates superimposed image data indicating a superimposed image obtained by synthesizing the background image projected by the image projection device 700A and the stroke image input by the electronic pen 100 .

Since the line-of-sight detection camera 19 is arranged on the screen 800, the distant writing mode can be executed in the same manner as in the first embodiment.

As described above, each of the above-described embodiments can be applied in various system configurations.

<Other application examples>
Although the best mode for carrying out the present invention has been described above using examples, the present invention is by no means limited to such examples, and various modifications can be made without departing from the scope of the present invention. and substitutions can be added.

For example, in the present embodiment, the support for the user to write a new drawn image has been described, but the distant writing mode can also be used to edit (enlarge, reduce, change color, etc.) an already drawn drawn image. . While viewing the drawn image 14 to be edited, the user touches the display 3 with the electronic pen 100 to copy the drawn image 14 at hand. The electronic pen 100 is removed from the contact position 17, but the line-of-sight direction 16 remains to see the drawn image 14 after copying. When the user inputs a gesture to edit the drawn image 14 and removes the line-of-sight direction 16 from the drawn image 14, the electronic blackboard 2 reflects the copied and edited drawn image 14 in the line-of-sight area 11.例文帳に追加In this case, the drawn image 14 originally in the line-of-sight area 11 may be deleted and replaced with the drawn image 14 edited at hand, or the user's editing content may be reflected in the drawn image 14 in the line-of-sight area 11. may

Further, the configuration examples in FIG. 5 and the like are divided according to main functions in order to facilitate understanding of the processing of the electronic blackboard 2 . The present invention is not limited by the division method or name of the unit of processing. Further, the processing of the electronic blackboard 2 can be divided into more processing units according to the content of the processing. Also, one processing unit can be divided to include more processing.

Also, each function of the embodiments described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification means a processor programmed by software to perform each function, such as a processor implemented by an electronic circuit, or a processor designed to perform each function described above. devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), SOC (System on a chip), GPU (Graphics Processing Unit) and conventional circuit modules.

Note that the input detection unit 21 is an example of contact position acquisition means, the stroke drawing unit 26 is an example of drawing means, the line-of-sight position detection unit 22 is an example of line-of-sight position detection means, and the drawn image determination unit 24 is an example of a line-of-sight position detection unit. The drawn image copying unit 25 is an example of the copying unit, the drawn image moving unit 27 is an example of the drawn image moving unit, and the visual effects unit 28 is an example of the visual effects unit.

2 electronic blackboard 3 display 11 line-of-sight area 13 line-of-sight position 14, 15, 18 drawing image 17 contact position 19 line-of-sight detection camera 100 electronic pen

Japanese Patent No. 2016-092440

Claims (10)

A shared terminal having a display,
contact position acquisition means for acquiring a contact position of a pointing object with respect to the display;
drawing means for drawing a drawn image on the display based on the contact position;
line-of-sight position detection means for detecting a line-of-sight position of a user operating the pointing object with respect to the display;
a drawn image determining means for determining a drawn image around the line-of-sight position with respect to the display according to the contact position acquired by the contact position acquisition means and the line-of-sight position detected by the line-of-sight position detecting means;
duplicating means for duplicating the drawn image around the line-of-sight position determined by the drawn image determining means and displaying it around the contact position on the display;
a drawn image moving means for moving the drawn image drawn at the contact position by the drawing means to the vicinity of the line-of-sight position and displaying it on the display;
The drawing means displays a drawn image on the display at the contact position while the line-of-sight position detection means detects the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position,
When the line-of-sight position detecting means stops detecting the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position, the drawn image moving means moves the drawn image displayed at the contact position by the drawing means to the line of sight. A shared terminal characterized by moving around a position and displaying it on the display . The drawn image moving means moves the drawn image drawn at the contact position on the display by the drawing means to the periphery of the line-of-sight position and displays the drawn image, and erases the drawn image displayed at the contact position on the display. 2. The shared terminal according to claim 1 , wherein: When the duplicating means duplicates and displays the drawn image around the line-of-sight position on the display around the contact position, the color of the drawn image that has already been drawn within a predetermined distance from the contact position on the display is reproduced. 3. The shared terminal according to claim 1 , wherein the shared terminal is changed or hidden. When the duplicating means duplicates and displays the drawn image around the line-of-sight position on the display around the contact position, the color of the drawn image that has already been drawn within a predetermined distance from the contact position on the display is reproduced. without changing
The drawing means causes the color of the drawn image displayed at the contact position on the display by the drawing means to differ from the color of the drawn image already drawn within a predetermined distance from the contact position,
The duplicating means makes the color of the drawn image around the line-of-sight position duplicated by the duplicating means and displayed on the display different from the color of the drawn image already drawn within a predetermined distance from the contact position. 4. A shared terminal according to claim 3 . A visual effect means for applying a visual effect for emphasizing the line-of-sight position on the display when the reproduction means duplicates and displays the drawn image around the line-of-sight position on the display around the contact position. The shared terminal according to any one of claims 1 to 4 . When the drawn image moving means moves and displays the drawn image drawn at the contact position on the display around the line-of-sight position on the display,
The visual effect means displays, on the display, an animation indicating that the drawn image displayed at the contact position on the display by the drawing means is reflected at the line-of-sight position, or a frame surrounding the line-of-sight position on the display. 6. The shared terminal according to claim 5 , wherein is displayed. The drawn image determination means receives setting of size and shape of a line-of-sight area corresponding to the circumference of the line-of-sight position on the display, which the drawn image determination means determines to be the circumference of the line-of-sight position on the display. The shared terminal according to any one of claims 1 to 6 , characterized by: 8. The shared terminal according to claim 7 , wherein the drawing image determination means receives setting of the size and shape of the line-of-sight area corresponding to the circumference of the line-of-sight position on the display by dragging the pointing object. . A writing method performed by a shared terminal having a display,
a contact position obtaining means obtaining a contact position of a pointing object with respect to the display;
a drawing means drawing a drawn image on the display based on the contact position;
a line-of-sight position detection means for detecting a line-of-sight position of a user operating the pointing object with respect to the display;
a step in which a drawn image determining means determines a drawn image around the line-of-sight position with respect to the display in accordance with the contact position acquired by the contact position acquiring means and the line-of-sight position detected by the line-of-sight position detecting means; ,
a step of replicating the drawn image around the line-of-sight position determined by the drawn image determining means and displaying it around the contact position on the display;
a drawn image moving means moving the drawn image drawn at the contact position by the drawing means to the vicinity of the line-of-sight position and displaying the drawn image on the display;
The drawing means displays a drawn image on the display at the contact position while the line-of-sight position detection means detects the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position,
When the line-of-sight position detecting means stops detecting the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position, the drawn image moving means moves the drawn image displayed at the contact position by the drawing means to the line of sight. A writing method characterized by moving around a position and displaying it on the display . a shared terminal with a display,
contact position acquisition means for acquiring a contact position of a pointing object with respect to the display;
drawing means for drawing a drawn image on the display based on the contact position;
line-of-sight position detection means for detecting a line-of-sight position of a user operating the pointing object with respect to the display;
a drawn image determining means for determining a drawn image around the line-of-sight position with respect to the display according to the contact position acquired by the contact position acquisition means and the line-of-sight position detected by the line-of-sight position detecting means;
duplicating means for duplicating the drawn image around the line-of-sight position determined by the drawn image determining means and displaying it around the contact position on the display;
functioning as a drawn image moving means for moving the drawn image drawn at the contact position by the drawing means to the periphery of the line-of-sight position and displaying it on the display;
the drawing means displays a drawn image on the display at the contact position while the line-of-sight position detection means detects the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position;
When the line-of-sight position detecting means stops detecting the line-of-sight position of the user with respect to the display within a predetermined distance from the contact position, the drawn image moving means moves the drawn image displayed at the contact position by the drawing means to the line of sight. A program for moving around a location and causing it to appear on the display .

Images